Feature extraction method and system based on 2.5 D convolution

Abstract

The invention discloses a 2.5 D convolution-based feature extraction method and system, and the method comprises the steps: carrying out the useless region cutting of an original 3D image, and obtaining a preprocessed 3D image; determining a plurality of slice directions according to the object contour of the original 3D image; according to the plurality of slice directions, segmenting the preprocessed 3D image into a plurality of 2D image sets; performing fusion processing on all the 2D images in the plurality of 2D image sets to obtain a 2.5 D image output set; multi-channel processing of a convolutional neural network is carried out on the 2.5 D image output set, and internal and external structural features of the object are extracted; and determining an abnormal region of the object according to the structural features. The 2.5 D image set is obtained by carrying out cutting preprocessing and multi-direction slice segmentation on the 3D image, non-time dimension correlation processing on the 3D image is achieved, the calculation complexity and the requirement for computing power and memory resources are reduced, and the image recognition efficiency and credibility are improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a feature extraction method and system based on 2.5D convolution. Background Technology

[0002] 3D convolution is a model that extends traditional 2D convolution operations to the temporal dimension. It can directly manipulate video frame sequences, simultaneously considering temporal and spatial features through 3D convolutional layers to extract spatiotemporal features and capture motion information from video data. 3D convolution is primarily used for dynamic video recognition, requiring greater computational resources and storage space during training and inference. In practical image data processing, not all scenes involve dynamic videos, but only static 3D images. Using 3D convolution would lead to a waste of computational and storage resources, while 2D convolution cannot achieve accurate static 3D image recognition. Therefore, accurately recognizing static 3D images while efficiently utilizing computational and memory resources is crucial for improving image recognition efficiency and reliability. Summary of the Invention

[0003] Considering that existing 3D convolutional recognition of static 3D images leads to a significant waste of computing power and memory resources, while 2D convolutional recognition of static 3D images cannot accurately determine the 3D details, reducing image recognition efficiency and reliability, this invention is proposed to provide a feature extraction method based on 2.5D convolution that overcomes or at least partially solves the above problems, comprising:

[0004] The original 3D image is cropped of its useless regions to obtain a preprocessed 3D image; based on the object outline of the original 3D image, several slicing directions are determined.

[0005] According to the aforementioned slicing directions, the preprocessed 3D image is divided into several 2D image sets; all 2D images within the aforementioned 2D image sets are fused to obtain a 2.5D image output set;

[0006] The 2.5D image output set is processed by a convolutional neural network to extract the structural features of the object's interior and exterior; based on the structural features, the abnormal regions of the object are determined.

[0007] Optionally, the original 3D image is cropped of its useless regions to obtain a preprocessed 3D image; based on the object contours of the original 3D image, several slicing directions are determined, including:

[0008] Multi-channel information distribution recognition is performed on the original 3D image to obtain the information distribution state of the original 3D image; wherein the information distribution state refers to the spatial distribution state of pixel chromaticity information and the spatial distribution state of pixel texture information.

[0009] Based on the information distribution state, the useless regions of the original 3D image are determined; based on the boundaries of the useless regions, the original 3D image is cropped into a preprocessed 3D image with regular boundaries.

[0010] Extract object contour data from the original 3D image, determine the spatial distribution of object contour information in the original 3D image based on the object contour data, and thereby determine several slicing directions; wherein the slicing direction refers to the direction in which the spatial density of object contour information in the original 3D image exceeds a preset density.

[0011] Optionally, the preprocessed 3D image is divided into several 2D image sets according to the several slicing directions; all 2D images within the several 2D image sets are fused to obtain a 2.5D image output set, including:

[0012] The preprocessed 3D image is uniformly divided into several 2D images along each slice direction, and the several 2D images are subjected to noise reduction filtering preprocessing and sequential arrangement preprocessing to obtain a 2D image set; wherein the preprocessed 3D image is divided along each slice direction to obtain a corresponding 2D image set;

[0013] Stack any two adjacent 2D images within each 2D image set to form a 2.5D image, thereby converting each 2D image set into a 2.5D image set. Based on the slice orientation of each 2D image set, perform volume fusion processing and triple attention cross-dimensional interaction mechanism on all 2.5D image sets to obtain a 2.5D image output set.

[0014] Optionally, the 2.5D image output set is subjected to multi-channel processing via a convolutional neural network to extract structural features of the object's interior and exterior; based on these structural features, abnormal regions of the object are determined, including:

[0015] The attention weights of the channels and height, channels and width, and spatial dimensions of the 2.5D image set are captured by several branches of the convolutional neural network; based on the attention weights, key regions are delineated in the 2.5D image output set, and then the structural features of the object's interior and exterior are extracted from the key regions.

[0016] A baseline structural comparison is performed on the internal and external structural features of the object to determine the structural deviations and their distribution locations. Based on the structural deviations and their distribution locations, abnormal regions of the object are determined. The abnormal regions refer to structurally missing areas that exist inside or outside the object.

[0017] As one aspect of the present invention, embodiments of the present invention also provide a feature extraction system based on 2.5D convolution, comprising:

[0018] The image cropping module is used to crop out useless areas from the original 3D image to obtain a preprocessed 3D image.

[0019] The slicing direction determination module is used to determine several slicing directions based on the object outline of the original 3D image;

[0020] The segmentation module is used to segment the preprocessed 3D image into several 2D image sets according to the several slicing directions;

[0021] The fusion module is used to fuse all 2D images within the plurality of 2D image sets to obtain a 2.5D image output set;

[0022] The convolution processing module is used to perform multi-channel processing of the 2.5D image output set using a convolutional neural network to extract the structural features of the object's interior and exterior.

[0023] An abnormal region determination module is used to determine the abnormal region of an object based on the structural features.

[0024] Optionally, the image cropping module is used to crop useless regions from the original 3D image to obtain a preprocessed 3D image, including:

[0025] Multi-channel information distribution recognition is performed on the original 3D image to obtain the information distribution state of the original 3D image; wherein the information distribution state refers to the spatial distribution state of pixel chromaticity information and the spatial distribution state of pixel texture information.

[0026] Based on the information distribution state, the useless regions of the original 3D image are determined; based on the boundaries of the useless regions, the original 3D image is cropped into a preprocessed 3D image with regular boundaries.

[0027] The slicing direction determination module is used to determine several slicing directions based on the object contours of the original 3D image, including:

[0028] Extract object contour data from the original 3D image, determine the spatial distribution of object contour information in the original 3D image based on the object contour data, and thereby determine several slicing directions; wherein the slicing direction refers to the direction in which the spatial density of object contour information in the original 3D image exceeds a preset density.

[0029] Optionally, the segmentation module is used to segment the preprocessed 3D image into several 2D image sets according to the several slicing directions, including:

[0030] The preprocessed 3D image is uniformly divided into several 2D images along each slice direction, and the several 2D images are subjected to noise reduction filtering preprocessing and sequential arrangement preprocessing to obtain a 2D image set; wherein the preprocessed 3D image is divided along each slice direction to obtain a corresponding 2D image set;

[0031] The fusion module is used to fuse all 2D images within the plurality of 2D image sets to obtain a 2.5D image output set, including:

[0032] Stack any two adjacent 2D images within each 2D image set to form a 2.5D image, thereby converting each 2D image set into a 2.5D image set. Based on the slice orientation of each 2D image set, perform volume fusion processing and triple attention cross-dimensional interaction mechanism on all 2.5D image sets to obtain a 2.5D image output set.

[0033] Optionally, the convolution processing module is used to perform multi-channel processing of the 2.5D image output set using a convolutional neural network to extract the structural features of the object's interior and exterior, including:

[0034] The attention weights of the channels and height, channels and width, and spatial dimensions of the 2.5D image set are captured by several branches of the convolutional neural network; based on the attention weights, key regions are delineated in the 2.5D image output set, and then the structural features of the object's interior and exterior are extracted from the key regions.

[0035] The abnormal region determination module is used to determine the abnormal region of the object based on the structural features, including:

[0036] A baseline structural comparison is performed on the internal and external structural features of the object to determine the structural deviations and their distribution locations. Based on the structural deviations and their distribution locations, abnormal regions of the object are determined. The abnormal regions refer to structurally missing areas that exist inside or outside the object.

[0037] The beneficial effects of the above-mentioned technical solutions provided in the embodiments of the present invention include at least the following:

[0038] This invention provides a feature extraction method and system based on 2.5D convolution. The method involves cropping useless regions from an original 3D image to obtain a preprocessed 3D image; determining several slicing directions based on the object contours of the original 3D image; segmenting the preprocessed 3D image into several 2D image sets according to these slicing directions; fusing all 2D images within each 2D image set to obtain a 2.5D image output set; performing multi-channel processing on the 2.5D image output set using a convolutional neural network to extract the internal and external structural features of the object; and identifying abnormal regions of the object based on these structural features. By performing cropping preprocessing and multi-directional slicing on the 3D image to obtain a 2.5D image set, this method achieves non-temporally related processing of the 3D image, reducing computational complexity and the demand for computing power and memory resources, and improving image recognition efficiency and reliability.

[0039] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0040] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0041] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0042] Figure 1 This is a flowchart illustrating the feature extraction method based on 2.5D convolution provided in this embodiment of the invention.

[0043] Figure 2 This is a schematic diagram of the structure of the feature extraction system based on 2.5D convolution provided in an embodiment of the present invention. Detailed Implementation

[0044] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0045] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," "outer," "far," "near," "front," and "rear," etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the accompanying drawings and are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0046] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0047] Please see Figure 1 As shown, an embodiment of this application provides a feature extraction method based on 2.5D convolution. This feature extraction method based on 2.5D convolution includes:

[0048] The useless regions of the original 3D image are cropped to obtain a preprocessed 3D image; based on the object outlines of the original 3D image, several slicing directions are determined.

[0049] Based on several slicing directions, the preprocessed 3D image is divided into several 2D image sets; all 2D images in the several 2D image sets are fused to obtain a 2.5D image output set;

[0050] Multi-channel processing of the 2.5D image output set using a convolutional neural network is performed to extract the structural features of the object's interior and exterior; based on these structural features, abnormal regions of the object are identified.

[0051] The beneficial effects of the above embodiments are that the feature extraction method based on 2.5D convolution obtains a 2.5D image set by preprocessing and multi-directional slicing of 3D images, realizes non-temporal dimension correlation processing of 3D images, reduces computational complexity and computing power and memory resource requirements, and improves image recognition efficiency and reliability.

[0052] In another embodiment, the original 3D image is cropped of its useless regions to obtain a preprocessed 3D image; based on the object contours of the original 3D image, several slicing directions are determined, including:

[0053] Multi-channel information distribution recognition is performed on the original 3D image to obtain the information distribution state of the original 3D image; where the information distribution state refers to the spatial distribution state of pixel chromaticity information and the spatial distribution state of pixel texture information.

[0054] Based on the information distribution, the useless regions of the original 3D image are determined; based on the boundaries of the useless regions, the original 3D image is cropped into a preprocessed 3D image with regular boundaries.

[0055] Extract object contour data from the original 3D image, determine the spatial distribution of object contour information in the original 3D image based on the object contour data, and thus determine several slicing directions; where the slicing direction refers to the direction in which the spatial density of object contour information in the original 3D image exceeds a preset density.

[0056] The beneficial effects of the above embodiments are as follows: For the original 3D image, in addition to the target object, there are other background objects. If the image data of the background objects is also mixed in during the actual image recognition process, it will disrupt the normal recognition of the target object and increase the workload of image recognition. Therefore, before performing feature recognition and extraction on the original 3D image, it is necessary to crop the original 3D image to remove the image portion related to the background objects. Specifically, multi-channel distribution recognition is performed on the original 3D image to obtain the spatial distribution state of pixel chroma information (i.e., the spatial density distribution of pixel chroma information in the entire 3D image's global image area) and the spatial distribution state of pixel texture information (i.e., the spatial density distribution of pixel texture information in the entire 3D image's global image area). It can be understood that the above two information distribution states describe the chroma and texture states of the target object and background objects in the original 3D image, respectively. In the original 3D image, there are significant differences in chroma and texture between the target object and the background object. The target object's chroma and texture information content is much higher than that of the background object. Therefore, based on the spatial distribution of pixel chroma and texture information, useless regions in the original 3D image are determined. These useless regions can be, but are not limited to, areas where the spatial density of pixel chroma information is less than a preset first density threshold or where the spatial density of pixel texture information is less than a preset second density threshold. Then, based on the boundaries of these useless regions, the original 3D image is cropped into a preprocessed 3D image with regular boundaries. For example, the useless regions can be completely removed, and then the image boundaries can be regularized to ensure that the preprocessed 3D image still has regular boundaries such as cubes or cuboids.

[0057] Furthermore, target objects can possess multi-dimensional spatial shapes, resulting in different contour forms in different directions within three-dimensional space. It's understandable that the more contour information a target object possesses in a certain direction of three-dimensional space, the more internal structural information it contains in that direction. To maximize the acquisition of the target object's internal structural information, object contour data is first extracted from the original 3D image to determine the spatial distribution of object contour information in the original 3D image. Then, this spatial distribution is projected across all 360-degree directions in three-dimensional space to obtain the object contour information corresponding to each direction. If the spatial density of object contour information corresponding to a certain direction is greater than a preset third density threshold, that direction is determined as the slicing direction of the original 3D image; that is, several planes perpendicular to that direction are used as slicing planes to segment the preprocessed 3D image. This method ensures that the segmented 2D image reflects the structural information of the target object to the greatest extent possible.

[0058] In another embodiment, the preprocessed 3D image is divided into several 2D image sets according to several slicing directions; all 2D images within the several 2D image sets are fused to obtain a 2.5D image output set, including:

[0059] The preprocessed 3D image is uniformly divided into several 2D images along each slice direction, and the several 2D images are subjected to noise reduction filtering preprocessing and sequential arrangement preprocessing to obtain a 2D image set; wherein the preprocessed 3D image is divided along each slice direction to obtain the corresponding 2D image set.

[0060] Stack any two adjacent 2D images within each 2D image set to form a 2.5D image, thereby converting each 2D image set into a 2.5D image set. Based on the slice orientation of each 2D image set, perform volume fusion processing and triple attention cross-dimensional interaction mechanism on all 2.5D image sets to obtain a 2.5D image output set.

[0061] The beneficial effects of the above embodiments are as follows: considering the determination of multiple slicing directions, segmenting the preprocessed 3D image according to each slicing direction can yield several 2D images. Specifically, for each slicing direction, several planes perpendicular to the slicing direction are used as cutting surfaces to segment the preprocessed 3D image one by one, thereby obtaining several 2D images. For each 2D image corresponding to each slicing direction, noise reduction filtering preprocessing (e.g., filtering out interference noise on each 2D image) and sequential arrangement preprocessing (e.g., arranging the several 2D images according to the slicing order in the corresponding slicing direction) are performed to obtain a 2D image set. It can be understood that each 2D image set corresponds to one slicing direction, and all 2D images within each 2D image set exist in a sequential queue. Then, every two adjacent 2D images within each 2D image set are stacked to form a 2.5D image. Stacking two 2D images into a 2.5D image can be implemented using a convolutional neural network (CNN), thus converting all 2D images within each 2D image set into a 2.5D image set through stacking. As described above, each 2.5D image set corresponds to a slice direction. Based on the spatial layout of all slice directions in the original 3D image, a convolutional neural network (CNN) is used to perform volume fusion processing and a triple attention cross-dimensional interaction mechanism on all 2.5D image sets to obtain a 2.5D image output set. This ensures that the 2.5D image output set comprehensively and accurately represents the construction of the target object in all slice directions. The use of a convolutional neural network (CNN) to perform volume fusion processing and a triple attention cross-dimensional interaction mechanism on all 2.5D image sets is a conventional technique in this field and will not be described in detail here.

[0062] In another embodiment, a multi-channel convolutional neural network is used to process the 2.5D image output set to extract the structural features of the object's interior and exterior; based on these structural features, abnormal regions of the object are determined, including:

[0063] The attention weights of channels and height, channels and width, and spatial dimensions of a 2.5D image set are captured by several branches of a convolutional neural network. Based on the attention weights, key regions are delineated in the 2.5D image output set, and then the structural features of the object's interior and exterior are extracted from the key regions.

[0064] By comparing the structural features of the object's interior and exterior with the benchmark structure, the structural deviations and their distribution locations are determined. Based on the structural deviations and their distribution locations, the abnormal regions of the object are determined. The abnormal regions refer to areas of structural deficiency that exist inside or outside the object.

[0065] The beneficial effects of the above embodiments, in practical operation, involve using several branches of the convolutional neural network model to capture the attention weights of three levels—channel and height, channel and width, and spatial dimension—of the 2.5D image set. The larger the attention weight, the higher the level of attention given to the corresponding level during subsequent feature extraction. Based on the attention weights, key regions are delineated in the 2.5D image output set. Each delineated key region corresponds to a different level of attention at the three levels, thereby extracting the internal and external structural features of the object from these key regions. These structural features comprehensively reflect the structural state of the key regions. Furthermore, a baseline construction comparison is performed on the internal and external structural features of the object to determine the structural deviations and their distribution locations. Regions where the deviation between the shape and size of the object's internal and external structure and the baseline structure exceeds a preset deviation threshold are identified as abnormal regions of the object. This accurately identifies structurally missing regions inside or outside the object, improving the recognition efficiency and reliability of the target object.

[0066] In one embodiment, a baseline structural comparison is performed on the internal and external structural features of an object to determine the structural deviations and their distribution locations. Based on these structural deviations and their distribution locations, abnormal regions of the object are identified. This can be implemented through the following sub-steps:

[0067] S41. Quantize the extracted internal and external structural features of the object into feature maps respectively; and obtain the baseline feature value for each feature channel for comparison.

[0068] Specifically, the extracted internal structural features of the object are quantized into feature maps. Externally constructed features are quantized into feature maps. ,in , The number of feature channels, These represent the spatial dimensions of height, width, and depth; each feature channel is obtained. Corresponding benchmark eigenvalues The aforementioned It is obtained by performing statistical learning on a known normal sample training set in advance, specifically by calculating the statistical average of the output feature values ​​of all normal samples in the training set on the corresponding channel.

[0069] S42. For each spatial location in the feature map, the corresponding internal feature value and external feature value are weighted and fused, and the fused feature value is compared with the reference feature value of each channel to calculate the construction deviation value that reflects the degree of deviation of the spatial location from the normal state.

[0070] In one embodiment, for each location in space The construction deviation value is calculated using the following weighted characteristic deviation formula (1). :

[0071] (1)

[0072] in, and Feature maps and In the passage, location Eigenvalues ​​at; For the first obtained in step S41 Channel reference characteristic value; The weight parameter for fusing internal and external features has a value range of [value range missing]. This is used to adjust the contribution ratio of internal features to external features in the fusion process; For the first The channel importance weight parameters for each feature channel satisfy the following conditions: and , used to characterize the differences in importance of different feature channels for anomaly detection; K is the total number of feature channels; the deviation value at all positions. The construction deviation diagram constitutes the above .

[0073] S43. Calculate the overall dispersion based on the construction deviation values ​​of all spatial locations, and generate an adaptive threshold for judging anomalies based on the overall dispersion.

[0074] In one embodiment, step S43 involves calculating the construction deviation map. Standard deviation and according to Generate adaptive anomaly threshold Specifically, it is calculated according to the following formula (2). Calculate T according to formula (3):

[0075] (2)

[0076] (3)

[0077] in, Deviation diagram The mean, , where is the total number of positions; Deviation diagram The standard deviation is used to measure the overall dispersion of the deviation values; A threshold adjustment parameter greater than zero is used to control the strictness of the anomaly judgment criteria.

[0078] S44. For each spatial location, based on whether its construction deviation value exceeds the adaptive threshold and combined with whether the number of similar abnormal points in its spatial neighborhood meets the continuity requirement, a comprehensive judgment is made to mark the location belonging to the abnormal region.

[0079] In one embodiment, it can be implemented as follows:

[0080] Based on the adaptive threshold obtained in step S43 The deviation diagram is constructed according to the preset judgment rules. Each location in the map is analyzed to generate an anomaly distribution map. The preset judgment rule is as follows:

[0081] For location :

[0082] If its deviation value And in its spatial neighborhood Inside, satisfy The number of points is not less than If so, then the location is determined to be an abnormal region, let Otherwise, it is judged as a normal area, and... .

[0083] in, As preset The spatial neighborhood centered on; is a preset positive integer representing the minimum number of neighboring outliers required to constitute a valid outlier region.

[0084] The intensity of continuous deviations is determined by whether A(x,y,z)=1 or 0. Figure 2 Values ​​are converted to generate an abnormal region mask image, which clearly marks "where the problem area is" (the set of pixels with a value of 1) and where the normal area is (with a value of 0).

[0085] all The set of locations constitutes the final determined abnormal region of the object and its distribution location.

[0086] The above embodiments have the following beneficial effects:

[0087] Through "weighted feature fusion" and "channel importance differentiation" (parameters) This allows for precise targeting of the internal and external structural features most relevant to anomalies, improving the ability to identify real structural defects. Simultaneously, the dual judgment mechanism combining "point-level threshold comparison" and "regional spatial constraints" effectively filters out isolated false alarms caused by noise, ensuring that the detected anomaly areas have reasonable spatial continuity, thereby enhancing the reliability and credibility of the results.

[0088] Anomaly detection threshold It is not a fixed value, but rather a construction of the overall distribution (standard deviation) of the deviation based on the current image. Adaptive generation This allows the system to automatically adapt to input images with different contrasts, lighting conditions, or image qualities, eliminating the need to manually adjust the threshold for each situation and enhancing the system's applicability and stability in real-world, complex scenarios.

[0089] Introduced (Channel weight) (Internal and external feature weights) (Threshold tightness) Several personalized parameters, such as (regional continuity), are available. These parameters provide users with clear adjustment options, allowing for flexible configuration of the model's sensitivity, emphasized features, and result format according to different detection objects and application needs, thus ensuring the technical solution has good interpretability and engineering friendliness.

[0090] In summary, the above embodiments, while ensuring accurate detection of structural anomalies in 3D images, reduce false alarms through their adaptive and robust design.

[0091] Please see Figure 2 As shown, an embodiment of this application provides a feature extraction system based on 2.5D convolution. This feature extraction system based on 2.5D convolution includes:

[0092] The image cropping module is used to crop out useless areas from the original 3D image to obtain a preprocessed 3D image.

[0093] The slicing direction determination module is used to determine several slicing directions based on the object outline of the original 3D image;

[0094] The segmentation module is used to segment the preprocessed 3D image into several 2D image sets according to several slicing directions;

[0095] The fusion module is used to fuse all 2D images within several 2D image sets to obtain a 2.5D image output set.

[0096] The convolution processing module is used to perform multi-channel processing of the 2.5D image output set using a convolutional neural network to extract the structural features of the object's interior and exterior.

[0097] The abnormal region determination module is used to determine the abnormal regions of an object based on its structural features.

[0098] The beneficial effects of the above embodiments are that the feature extraction system based on 2.5D convolution obtains a 2.5D image set by preprocessing and multi-directional slicing of 3D images, realizes non-temporal dimension correlation processing of 3D images, reduces computational complexity and computing power and memory resource requirements, and improves image recognition efficiency and reliability.

[0099] In another embodiment, the image cropping module is used to crop useless regions from the original 3D image to obtain a preprocessed 3D image, including:

[0100] Multi-channel information distribution recognition is performed on the original 3D image to obtain the information distribution state of the original 3D image; where the information distribution state refers to the spatial distribution state of pixel chromaticity information and the spatial distribution state of pixel texture information.

[0101] Based on the information distribution, the useless regions of the original 3D image are determined; based on the boundaries of the useless regions, the original 3D image is cropped into a preprocessed 3D image with regular boundaries.

[0102] The slicing direction determination module is used to determine several slicing directions based on the object contours of the original 3D image, including:

[0103] Extract object contour data from the original 3D image, determine the spatial distribution of object contour information in the original 3D image based on the object contour data, and thus determine several slicing directions; where the slicing direction refers to the direction in which the spatial density of object contour information in the original 3D image exceeds a preset density.

[0104] In another embodiment, the segmentation module is used to segment the preprocessed 3D image into several 2D image sets according to several slicing directions, including:

[0105] The preprocessed 3D image is uniformly divided into several 2D images along each slice direction, and the several 2D images are subjected to noise reduction filtering preprocessing and sequential arrangement preprocessing to obtain a 2D image set; wherein the preprocessed 3D image is divided along each slice direction to obtain the corresponding 2D image set.

[0106] The fusion module is used to fuse all 2D images within several 2D image sets to obtain a 2.5D image output set, including:

[0107] Stack any two adjacent 2D images within each 2D image set to form a 2.5D image, thereby converting each 2D image set into a 2.5D image set. Based on the slice orientation of each 2D image set, perform volume fusion processing and triple attention cross-dimensional interaction mechanism on all 2.5D image sets to obtain a 2.5D image output set.

[0108] In another embodiment, the convolution processing module is used to perform multi-channel processing of the 2.5D image output set using a convolutional neural network to extract the structural features of the object's interior and exterior, including:

[0109] The attention weights of channels and height, channels and width, and spatial dimensions of a 2.5D image set are captured by several branches of a convolutional neural network. Based on the attention weights, key regions are delineated in the 2.5D image output set, and then the structural features of the object's interior and exterior are extracted from the key regions.

[0110] The anomaly region determination module is used to determine the anomaly regions of an object based on its structural features, including:

[0111] By comparing the structural features of the object's interior and exterior with the benchmark structure, the structural deviations and their distribution locations are determined. Based on the structural deviations and their distribution locations, the abnormal regions of the object are determined. The abnormal regions refer to areas of structural deficiency that exist inside or outside the object.

[0112] The feature extraction system based on 2.5D convolution of the present invention operates and has the same effect as the feature extraction method based on 2.5D convolution described above, and will not be described again here.

[0113] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. This disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims. Thus, if these modifications and variations of the invention fall within the scope of the claims of the invention and their equivalents, the invention is also intended to include these modifications and variations.

Claims

1. A feature extraction method based on 2.5D convolution, characterized in that, include: The original 3D image is cropped of its useless regions to obtain a preprocessed 3D image; based on the object outline of the original 3D image, several slicing directions are determined. According to the aforementioned slicing directions, the preprocessed 3D image is divided into several 2D image sets; all 2D images within the aforementioned 2D image sets are fused to obtain a 2.5D image output set; The 2.5D image output set is processed by a convolutional neural network to extract the structural features of the object's interior and exterior; based on the structural features, the abnormal regions of the object are determined.

2. The feature extraction method based on 2.5D convolution as described in claim 1, characterized in that: The original 3D image is cropped of its unwanted regions to obtain a preprocessed 3D image; based on the object contours of the original 3D image, several slicing directions are determined, including: Multi-channel information distribution recognition is performed on the original 3D image to obtain the information distribution state of the original 3D image; wherein the information distribution state refers to the spatial distribution state of pixel chromaticity information and the spatial distribution state of pixel texture information. Based on the information distribution state, the useless regions of the original 3D image are determined; based on the boundaries of the useless regions, the original 3D image is cropped into a preprocessed 3D image with regular boundaries. Extract object contour data from the original 3D image, determine the spatial distribution of object contour information in the original 3D image based on the object contour data, and thereby determine several slicing directions; wherein the slicing direction refers to the direction in which the spatial density of object contour information in the original 3D image exceeds a preset density.

3. The feature extraction method based on 2.5D convolution as described in claim 1, characterized in that: Based on the aforementioned slicing directions, the preprocessed 3D image is divided into several 2D image sets; all 2D images within the aforementioned 2D image sets are fused to obtain a 2.5D image output set, including: The preprocessed 3D image is uniformly divided into several 2D images along each slice direction, and the several 2D images are subjected to noise reduction filtering preprocessing and sequential arrangement preprocessing to obtain a 2D image set; wherein the preprocessed 3D image is divided along each slice direction to obtain a corresponding 2D image set; Stack any two adjacent 2D images within each 2D image set to form a 2.5D image, thereby converting each 2D image set into a 2.5D image set. Based on the slice orientation of each 2D image set, perform volume fusion processing and triple attention cross-dimensional interaction mechanism on all 2.5D image sets to obtain a 2.5D image output set.

4. The feature extraction method based on 2.5D convolution as described in claim 1, characterized in that: The 2.5D image output set is processed by a convolutional neural network in a multi-channel manner to extract the structural features of the object's interior and exterior; based on these structural features, abnormal regions of the object are determined, including: The attention weights of the channels and height, channels and width, and spatial dimensions of the 2.5D image set are captured by several branches of the convolutional neural network; based on the attention weights, key regions are delineated in the 2.5D image output set, and then the structural features of the object's interior and exterior are extracted from the key regions. A baseline structural comparison is performed on the internal and external structural features of the object to determine the structural deviations and their distribution locations. Based on the structural deviations and their distribution locations, abnormal regions of the object are determined. The abnormal regions refer to structurally missing areas that exist inside or outside the object.

5. A feature extraction system based on 2.5D convolution, characterized in that, include: The image cropping module is used to crop out useless areas from the original 3D image to obtain a preprocessed 3D image. The slicing direction determination module is used to determine several slicing directions based on the object outline of the original 3D image; The segmentation module is used to segment the preprocessed 3D image into several 2D image sets according to the several slicing directions; The fusion module is used to fuse all 2D images within the plurality of 2D image sets to obtain a 2.5D image output set; The convolution processing module is used to perform multi-channel processing of the 2.5D image output set using a convolutional neural network to extract the structural features of the object's interior and exterior. An abnormal region determination module is used to determine the abnormal region of an object based on the structural features.

6. The feature extraction system based on 2.5D convolution as described in claim 5, characterized in that: The image cropping module is used to crop useless regions from the original 3D image to obtain a preprocessed 3D image, including: Multi-channel information distribution recognition is performed on the original 3D image to obtain the information distribution state of the original 3D image; wherein the information distribution state refers to the spatial distribution state of pixel chromaticity information and the spatial distribution state of pixel texture information. Based on the information distribution state, the useless regions of the original 3D image are determined; based on the boundaries of the useless regions, the original 3D image is cropped into a preprocessed 3D image with regular boundaries. The slicing direction determination module is used to determine several slicing directions based on the object contours of the original 3D image, including: Extract object contour data from the original 3D image, determine the spatial distribution of object contour information in the original 3D image based on the object contour data, and thereby determine several slicing directions; wherein the slicing direction refers to the direction in which the spatial density of object contour information in the original 3D image exceeds a preset density.

7. The feature extraction system based on 2.5D convolution as described in claim 5, characterized in that: The segmentation module is used to segment the preprocessed 3D image into several 2D image sets according to the several slice directions, including: The preprocessed 3D image is uniformly divided into several 2D images along each slice direction, and the several 2D images are subjected to noise reduction filtering preprocessing and sequential arrangement preprocessing to obtain a 2D image set; wherein the preprocessed 3D image is divided along each slice direction to obtain a corresponding 2D image set; The fusion module is used to fuse all 2D images within the plurality of 2D image sets to obtain a 2.5D image output set, including: Stack any two adjacent 2D images within each 2D image set to form a 2.5D image, thereby converting each 2D image set into a 2.5D image set. Based on the slice orientation of each 2D image set, perform volume fusion processing and triple attention cross-dimensional interaction mechanism on all 2.5D image sets to obtain a 2.5D image output set.

8. The feature extraction system based on 2.5D convolution as described in claim 5, characterized in that: The convolutional processing module is used to perform multi-channel processing of the 2.5D image output set using a convolutional neural network to extract the structural features of the object's interior and exterior, including: The attention weights of the channels and height, channels and width, and spatial dimensions of the 2.5D image set are captured by several branches of the convolutional neural network; based on the attention weights, key regions are delineated in the 2.5D image output set, and then the structural features of the object's interior and exterior are extracted from the key regions. The abnormal region determination module is used to determine the abnormal region of the object based on the structural features, including: A baseline structural comparison is performed on the internal and external structural features of the object to determine the structural deviations and their distribution locations. Based on the structural deviations and their distribution locations, abnormal regions of the object are determined. The abnormal regions refer to structurally missing areas that exist inside or outside the object.

Images