Unified shape representation and transformation method, system and equipment and storage medium

Abstract

The invention provides a unified shape representation and transformation method, system and device and a storage medium, and relates to the technical field of image processing, and the method comprises the steps: obtaining original image data; processing each piece of original image data based on a preset detection model to obtain corresponding key point data; inputting all the key point data into a preset shape representation and transformation unification model for processing to obtain categories of different shapes in the original image data; wherein the shape representation and transformation unification model is used for identifying different shapes in the original image data and determining corresponding categories. Through shape representation and transformation unified model processing, the influence of different image sources and quality on shape recognition accuracy can be reduced, and the consistency of recognition results is improved. Meanwhile, the shape representation and transformation unified model can process image data of various types and sources, and the generalization ability of the model to new data is enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a unified shape representation and transformation method, system, device and storage medium. BACKGROUND

[0002] In the field of computer vision and image processing, the description and analysis of object features is one of the core tasks. From a visual perspective, various attributes can be used to describe the features of an object, such as shape, position, color, and lighting, etc. Among them, position information changes with the change of the position or direction of the target, and color and lighting features are closely related to the scene in which the object is located. In contrast, shape describes the structural features of the target, has strong stability, and does not change with the position, size and brightness of the object, and is one of the essential features of the object and one of the most difficult features to describe in an image.

[0003] Shape analysis refers to the process of extracting feature information of the shape of a target from an image, storing it in a specific data structure, and using it for comparison, recognition, classification and retrieval operations. Shape analysis plays an important role in computer vision and medical imaging, and is widely used in image recognition, text processing, biological science and medical diagnosis related fields. For example, in medical imaging, shape analysis can be used to identify and classify diseased tissues to assist doctors in diagnosis; in computer vision, it can be used for object recognition and scene understanding.

[0004] Currently, shape representation and transformation are widely used in shape matching. Shape matching based on target morphology requires shape similarity calculation, which usually depends on the parameter system of shape transformation. However, most existing methods use separate models for shape representation and shape transformation. Specifically, one model is proposed for shape representation, while another model is proposed for shape transformation. This separate model not only increases the complexity of shape analysis, but also limits the synergy between shape representation and transformation, reducing the efficiency and accuracy of shape matching. SUMMARY

[0005] The present application solves one or more of the above related technical problems.

[0006] To solve the above problems, the present application provides a unified shape representation and transformation method, system, device and storage medium.

[0007] In a first aspect, the present application provides a unified shape representation and transformation method, comprising: obtaining original image data; processing each of the original image data based on a preset detection model to obtain corresponding key point data; inputting all the key point data into a preset shape representation and transformation unified model to obtain categories of different shapes in each of the original image data; The shape representation and transformation unified model is configured to identify different shapes in each of the original image data and determine corresponding categories.

[0008] Optionally, the preset detection model comprises a feature extraction module, an improved attention module, and a key point detection module; the original image data is processed based on the preset detection model to obtain key point data, which comprises: preprocessing each of the original image data to obtain temporary image data; processing the temporary image data through the feature extraction module to obtain corresponding fusion feature maps; enhancing the fusion feature maps through the improved attention module to obtain enhanced feature maps; processing the enhanced feature maps through the key point detection module to obtain the key point data.

[0009] Optionally, the feature extraction module comprises an HRNet module, a deformable convolution module, and a fusion module; the temporary image data is processed through the feature extraction module to obtain corresponding fusion feature maps, which comprises: extracting features of the temporary image data through the HRNet module to obtain multi-scale feature maps; convolving the multi-scale feature maps through the deformable convolution module to obtain temporary feature maps; fusing the multi-scale feature maps and the temporary feature maps through the fusion module to obtain corresponding fusion feature maps.

[0010] Optionally, the preset shape representation and transformation unified model comprises a shape expression module, a shape change module, and a shape similarity comparison module; all the key point data is inputted into the preset shape representation and transformation unified model to obtain categories of different shapes in each of the original image data, which comprises: processing the key point data in each of the original image data through the shape expression module to obtain corresponding representation data of the different shapes; calculating the representation data of the different shapes corresponding to different original image data through the shape change module to obtain geodesic distances between the different shapes; performing cluster analysis on the geodesic distances between the different shapes through the shape similarity comparison module to obtain categories of different shapes in each of the original image data.

[0011] Optionally, the step of processing the key point data in each of the original image data through the shape representation module to obtain the corresponding representation data of different shapes includes: Based on a preset Gaussian mixture algorithm, the key point data is processed to obtain representation data of different shapes; The preset Gaussian mixture algorithm is as follows: ; in, It is a set composed of the key point data. It is the number of key points. It corresponds to any point in the original image data. It is the first Location data of key points; For variance data, The probability density data for the x-th point is the representation data of the different shapes.

[0012] Optionally, the step of transforming the representation data of the different shapes through the shape transformation module to obtain the geodesic distance between the different shapes includes: The corresponding metric tensor data is obtained by calculating the probability density data in each of the original image data. Geodesics are determined based on the metric tensor data of different original image data, and the geodesic distances between different shapes are obtained based on the geodesics.

[0013] Optionally, the step of performing cluster analysis on the geodesic distances between different shapes using the shape similarity comparison module to obtain the categories of different shapes in each of the original image data includes: Construct a similarity matrix based on the distances between the geodesics described above; Cluster analysis is performed on the similarity matrix according to a preset clustering algorithm to obtain the categories corresponding to different shapes in each of the original image data.

[0014] Secondly, the present invention provides a unified shape representation and transformation system, comprising: The acquisition module is used to acquire raw image data; The processing module is used to process each of the original image data based on a preset detection model to obtain the corresponding key point data; The processing module is further configured to input all the key point data into a preset shape representation and transformation unified model for processing to obtain categories of different shapes in each of the original image data.

[0015] In a third aspect, the present application provides a shape representation and transformation unified device, comprising a memory and a processor; the memory is configured to store a computer program; the processor is configured to implement the shape representation and transformation unified method according to the first aspect when executing the computer program.

[0016] In a fourth aspect, the present application provides a computer readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the shape representation and transformation unified method according to the first aspect is implemented.

[0017] The shape representation and transformation unified method, system, device and storage medium of the present application have the following advantages: First, a series of original image data containing different target shapes are obtained. These images can be medical scan images, satellite images, industrial detection images, etc., depending on the application scenario. The original image data can contain single or multiple shape targets.

[0018] Next, a preset detection model is used to process each original image. This model may include deep learning networks or other image processing techniques for identifying and locating key points in the image. The output of the detection model is the data of the key points in each image, which usually includes the coordinates, colors or other characteristics of the key points.

[0019] Then, the key point data extracted from all original images are integrated and input into a shape representation and transformation unified model. This shape representation and transformation unified model is a preset processing flow for converting key point data in different images into a unified shape representation. The shape representation and transformation unified model may include data preprocessing, feature extraction, shape modeling, etc. to ensure that key point data from different sources can be uniformly understood and processed. At the same time, the shape representation and transformation unified model analyzes the input key point data, identifies the shapes in the image, and determines the category of each shape. This process may involve shape feature extraction, shape similarity measurement, and classification algorithms based on these measurements. Finally, the shape representation and transformation unified model outputs the category labels of each shape, which can be based on the geometric features, texture features or other related features of the shape.

[0020] Therefore, the present application can reduce the influence of different image sources and qualities on shape recognition accuracy through shape representation and transformation unified model processing, and improve the consistency of recognition results. At the same time, the shape representation and transformation unified model can process multiple types and sources of image data, enhancing the model's generalization ability to new data. In addition, integrating key point detection and shape classification into a unified process simplifies the complexity of shape analysis, making the entire process more efficient and easy to manage. Due to the versatility and flexibility of the method, it can be applied to various fields and scenarios, such as medical image analysis, object recognition, industrial detection, etc. Through automated shape recognition and classification, manual intervention can be reduced, processing speed can be improved, and the development of automation and intelligence in related fields can be promoted. At the same time, standardized shape category information is provided, facilitating further statistical analysis, pattern recognition, and decision support. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 A flowchart of a unified shape representation and transformation method according to an embodiment of the present application; Figure 2 A structure diagram of a preset detection model according to an embodiment of the present application; Figure 3 A diagram of different shapes in different images according to an embodiment of the present application; Figure 4 A diagram of different shape categories according to an embodiment of the present application; Figure 5 A diagram of a unified shape representation and transformation system according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] To make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings. Although some embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms, and should not be interpreted as being limited to the embodiments described herein, on the contrary, these embodiments are provided to more thoroughly and completely understand the present application. It should be understood that the drawings and embodiments of the present application are only for illustrative purposes, and are not intended to limit the scope of protection of the present application.

[0023] The term “include,” and variations thereof, as used in this document, is open-ended and means “comprising” or “including” but not limited to; the term “based on” means “based, at least in part, on;” the term “one embodiment” means “at least one embodiment;” the term “another embodiment” means “at least one additional embodiment;” the term “some embodiments” means “at least some embodiments;” and the term “optional” means “optional in at least one embodiment.” Related terms shall be construed accordingly. It is to be noted that the terms “first,” “second,” and the like used in this document do not necessarily have an ordinal meaning with respect to one another, and are used to distinguish one element from another.

[0024] The names of the messages or information exchanged between the plurality of devices in the embodiments of the present application are only for illustrative purposes, and are not used to limit the scope of the messages or information.

[0025] The embodiments of the present application provide a unified shape representation and transformation method, system, device and storage medium.

[0026] As shown in Figure 1 The unified shape representation and transformation method provided by the embodiments of the present application comprises: Step S100, acquiring original image data.

[0027] Specifically, according to the application requirements, the original image data is collected from different data sources. These data sources can include digital image databases, online resources, scanning devices, etc. And the original image data is preprocessed accordingly, such as image enhancement and denoising operations. Through the acquisition and preprocessing of the original image data, a solid foundation is laid for the subsequent key point detection, shape representation and transformation steps, ensuring the smooth progress of the entire process.

[0028] Step S200, processing each of the original image data based on a preset detection model to obtain corresponding key point data.

[0029] Specifically, a detection model is pre-trained, which can identify and locate the key points in the image. The model can be a neural network based on deep learning (such as convolutional neural network CNN), or a traditional image processing technology (such as edge detection, feature matching, etc.). The selection of the model depends on the application scenario, computing resources and accuracy requirements, etc. Load the pre-trained detection model. If the model needs to be trained from scratch, the training data set needs to be prepared, and the model training and verification are performed.

[0030] The original image data is preprocessed, including size adjustment, normalization, enhancement, etc. to adapt to the input requirements of the model. Preprocessing can improve the robustness and accuracy of the model.

[0031] The pre-processed image is input into a detection model, which outputs the positions of key points in the image and possibly other features such as color, texture, etc. Key points can be corner points, edges, center points, etc. of the object, which are representative feature points.

[0032] The key point data output by the model is post-processed, including denoising, filtering, sorting, etc. to improve the accuracy and reliability of the key points. Post-processing can remove incorrectly detected key points and retain the most representative key points. The detected key point data is stored in a database or file system for subsequent shape analysis and processing.

[0033] By using a pre-set detection model to automatically detect key points, human error can be reduced, and the accuracy and consistency of shape recognition can be improved. In addition, automated key point detection can significantly improve processing speed, especially when processing large amounts of image data, which can greatly save time and labor costs. In addition to the positions of key points, the model can also extract other features such as color, texture, etc. to provide more rich information for subsequent shape analysis. Providing accurate and reliable key point data can simplify subsequent shape representation, transformation and classification steps, thereby improving the efficiency and accuracy of the entire process.

[0034] In summary, based on the pre-set detection model, the original image data is processed to obtain the corresponding key point data, which is a key step in the shape analysis process. Through automated and high-precision key point detection, high-quality input data can be provided for subsequent shape analysis, thereby improving the efficiency and accuracy of the entire process.

[0035] In step S300, all the key point data is input into a pre-set shape representation and transformation unified model for processing to obtain the categories of different shapes in each of the original image data. The shape representation and transformation unified model is used to identify different shapes in each of the original image data and determine the corresponding categories.

[0036] Specifically, all the key point data extracted from step S200 is integrated together. These data may include the coordinates of key points, color, texture, etc. features, and their positions in the original image.

[0037] The integrated key point data is input into a pre-set shape representation and transformation unified model. This model is a system that integrates shape representation and transformation functions for processing and analyzing key point data.

[0038] The unified model first performs shape representation on the input keypoint data. This may involve mapping the keypoints into a shape space, such as using a Gaussian Mixture Model (GMM) or other probabilistic model to represent the probability density function of the shapes.

[0039] Next, the model calculates transformations between different shapes, such as geodesic distances. These transformations are used to measure the similarity or difference between shapes and serve as the basis for shape matching and classification.

[0040] Based on the results of shape representation and transformation, the unified model classifies the shapes. This may involve grouping shapes using clustering algorithms (such as K-means, hierarchical clustering, etc.) or labeling shapes using classification algorithms (such as support vector machines, neural networks, etc.).

[0041] Finally, the model outputs the class labels for each shape. These labels represent the class to which the shape belongs and can be based on geometric features, texture features, or other relevant features of the shape.

[0042] The above process handles keypoint data through a unified model, which can reduce the impact of different image sources and quality on shape recognition accuracy, improving the consistency of recognition results. The unified model can handle multiple types and sources of image data, enhancing the model's generalization ability for new data. By integrating keypoint detection and shape classification into a unified process, the complexity of shape analysis is simplified, making the entire process more efficient and easier to manage. At the same time, due to the universality and flexibility of this method, it can be applied to various fields and scenarios, such as medical image analysis, object recognition, industrial detection, etc.

[0043] Through automated shape recognition and classification, manual intervention can be reduced, processing speed can be improved, and the development of automation and intelligence in related fields can be promoted. The data processed by the unified model provides standardized shape class information, which facilitates further statistical analysis, pattern recognition, and decision support.

[0044] Therefore, step S300 processes keypoint data through a shape representation and transformation unified model, achieving accurate recognition and classification of shapes, providing a solid foundation for subsequent shape analysis and applications.

[0045] In this embodiment, first, a series of original image data containing different target shapes is obtained. These images can be medical scan images, satellite images, industrial detection images, etc., depending on the application scenario. The original image data can contain single or multiple shape targets.

[0046] Next, a pre-set detection model is used to process each raw image. This model can include deep learning networks or other image processing techniques for identifying and locating key points in the images. The output of the detection model is key point data for each image, which typically includes coordinates, colors, or other features of the key points.

[0047] Then, the key point data extracted from all raw images is integrated and input into a shape representation and transformation unification model. This shape representation and transformation unification model is a pre-set processing flow for converting key point data from different images into a unified shape representation. The shape representation and transformation unification model can include data preprocessing, feature extraction, shape modeling, etc. steps to ensure that key point data from different sources can be uniformly understood and processed. At the same time, the shape representation and transformation unification model analyzes the input key point data, identifies the shapes in the images, and determines the category of each shape. This process can involve feature extraction of shapes, similarity measurement between shapes, and classification algorithms based on these measurements. Finally, the shape representation and transformation unification model outputs the category labels of each shape, which can be based on the geometric features, texture features or other related features of the shapes.

[0048] Therefore, by processing through the shape representation and transformation unification model, the present application can reduce the influence of different image sources and qualities on shape recognition accuracy, and improve the consistency of recognition results. At the same time, the shape representation and transformation unification model can process multiple types and sources of image data, enhancing the model's generalization ability for new data. In addition, integrating key point detection and shape classification into a unified process simplifies the complexity of shape analysis, making the entire process more efficient and easier to manage. Due to its versatility and flexibility, this method can be applied to various fields and scenarios, such as medical image analysis, object recognition, industrial detection, etc. Through automated shape recognition and classification, manual intervention can be reduced, processing speed can be improved, and the development of automation and intelligence in related fields can be promoted. At the same time, standardized shape category information is provided, facilitating further statistical analysis, pattern recognition and decision support.

[0049] Optionally, the pre-set detection model includes a feature extraction module, an improved attention module and a key point detection module; the processing of each raw image data based on the pre-set detection model to obtain key point data includes: preprocessing each raw image data to obtain temporary image data; processing the temporary image data through the feature extraction module to obtain the corresponding fusion feature map; enhancing the fusion feature map through the improved attention module to obtain an enhanced feature map; The key point detection module processes the enhanced feature map to obtain the key point data.

[0050] Optionally, the feature extraction module includes an HRNet module, a deformable convolution module, and a fusion module; the processing of the temporary image data by the feature extraction module to obtain the corresponding fusion feature map includes: The HRNet module extracts features from the temporary image data to obtain a multi-scale feature map; The deformable convolution module convolves the multi-scale feature map to obtain a temporary feature map; The fusion module fuses the multi-scale feature map and the temporary feature map to obtain the corresponding fusion feature map.

[0051] Specifically, as shown in Figure 3 The preset detection model is based on the HRNet framework combined with deformable convolution and an improved attention module to obtain key point data. The input data of the model is original image data, which may have different sizes, pixel value ranges, and may contain noise and other interference factors.

[0052] First, the original image data is preprocessed, such as size adjustment: the image is uniformly adjusted to the input size required by the HRNet module, such as common sizes of 256x256, 384x288, etc. This is usually achieved through image scaling algorithms such as interpolation to ensure that the model can process fixed-size input data.

[0053] Normalization process: normalize the pixel values of the image from the original range (such as [0, 255]) to a specific interval (such as [0, 1] or [-1, 1]), by subtracting the mean and dividing by the standard deviation, etc. to make the data distribution of different images relatively consistent, which helps to speed up the convergence process of model training, improve training efficiency and stability.

[0054] Data augmentation process: random rotation, translation, flipping, scaling, cropping, and possibly color jittering (such as adjusting brightness, contrast, saturation, etc.) are performed on the image. The purpose is to increase the diversity and richness of the data set, so that the model can learn more robust features, and improve the model's ability to adapt to images under different poses, angles, and lighting conditions, so that it can more accurately detect key points in actual applications.

[0055] The image data (temporary image data) processed as described above has a uniform size, a normalized pixel value range, and the scale of the data set is expanded after data augmentation, and the diversity of the samples is also improved, preparing for subsequent feature extraction and other operations.

[0056] HRNet includes multiple parallel convolution branches (convolutional networks), each corresponding to a different resolution. In the initial stage, different size convolution kernels (such as 3x3 convolution kernels, etc.) are used to perform convolution operations on the input image (input data: temporary image data) to extract feature maps of different scales. These branches start parallel processing at the shallow stage, allowing the network to capture local detail features (in high-resolution branches) and global context semantic features (in low-resolution branches) at an early stage. In this process, each branch has multiple layers of convolution, batch normalization (Batch Normalization, BN), and activation functions (such as ReLU) for gradually extracting and enhancing features.

[0057] At the end of each stage, the feature maps of each branch are fused and interacted and fused at the same resolution through upsampling or downsampling operations. For example, the high-resolution feature map is downsampled and added or spliced with the low-resolution feature map element by element, and then upsampled to restore the high resolution, or the low-resolution feature map is upsampled and fused with the high-resolution feature map. This fusion method allows different resolution branches to complement each other's information, effectively fusing multi-scale features, allowing the network to utilize features of different scales to better understand image content and provide more rich feature information for subsequent key point detection.

[0058] After multi-stage parallel multi-resolution branch convolution extraction and feature fusion, multi-scale feature maps are obtained. These feature maps contain various levels of image feature information from local to global and from fine to coarse, which can comprehensively represent the shape structure and other features in the image, laying a good feature foundation for subsequent key point detection tasks.

[0059] The input data of the deformable convolution module is a part of the feature map output by the HRNet module (usually some key branches, such as the feature map in the high-resolution branch responsible for extracting finer features).

[0060] The core of deformable convolution is to introduce an additional offset learning mechanism. First, the input feature map is processed through a convolution layer to obtain the offset parameters at each position. These offset parameters determine the shape change of the convolution kernel at that position, i.e., the position of each convolution kernel is dynamically adjusted according to the learned offset, so as to adaptively focus on the key area in the complex shape structure and better capture the features of irregular shapes.

[0061] After obtaining the offset, the offset is applied to the traditional convolution operation, so that the convolution kernel can dynamically change the sampling position during feature extraction according to the shape structure of the image. Specifically, for each sampling point in the convolution kernel, the actual sampling position in the input feature map is determined according to the learned offset, and then the feature value at that position is obtained through methods such as bilinear interpolation, and then the convolution operation is completed to extract features that can better adapt to complex shape changes.

[0062] After the deformable convolution processing, the feature maps have enhanced expression ability for complex shape structures compared to traditional convolution feature maps, and can more accurately reflect the key part features of irregular shapes in the image, providing more discriminative feature information for subsequent feature fusion and key point detection.

[0063] The input data of the multi-scale feature fusion module (fusion module) is the feature maps of each resolution branch processed by the HRNet module (multi-scale feature maps) and the enhanced feature maps output by the deformable convolution module (temporary feature maps).

[0064] Data processing process: Adjust the feature maps of different resolutions to the same resolution through upsampling or downsampling operations to facilitate subsequent fusion operations. Upsampling generally uses methods such as bilinear interpolation and deconvolution to increase the spatial size of the feature map, while downsampling can reduce the spatial size of the feature map through methods such as max pooling, average pooling, or convolution with a stride greater than 1.

[0065] The feature maps adjusted to the same resolution are spliced to integrate feature information from different sources. Then, the spliced feature map is further processed through convolution layers and other operations to reduce the number of feature channels and enhance the expression ability of the features. This process can be regarded as a comprehensive utilization of multi-scale features, enabling the model to utilize both global context information and local fine detail information, while combining the shape features enhanced by deformable convolution, thereby improving the detection performance of shape key points.

[0066] Finally, the fused feature map is output, which integrates multi-scale feature information and shape feature enhanced by deformable convolution, can more comprehensively and accurately represent the key region features of the shape in the image, and is an important basis for subsequent key point prediction.

[0067] Input data of the improved attention module: feature map output by the multi-scale feature fusion module (fused feature map).

[0068] Data processing process: first, the input feature map is subjected to global average pooling operation to obtain a global feature vector for each channel. Then, the vector is processed through two fully connected layers to compress and activate the features respectively, obtaining a channel attention weight vector. This weight vector represents the importance of each channel in the current feature map for the key point detection task. By performing a channel-by-channel multiplication operation between the channel attention weight vector and the original feature map, the feature channels that are important for shape key point detection can be enhanced, and the channels with low task relevance can be suppressed.

[0069] Based on the feature map after channel attention processing, spatial attention is further calculated. By performing maximum pooling and average pooling operations on the feature map in the channel dimension, two intermediate feature maps are obtained. Then, the two feature maps are spliced and processed through convolution layers and activation functions to obtain a spatial attention weight map. This spatial attention weight map emphasizes the key region positions in the feature map related to the shape key points, and highlights the features of the key regions by performing element-by-element multiplication operation with the original feature map, so that the model can focus more on the key parts of the shape and enhance the extraction effect of complex structure features such as cervical vertebrae.

[0070] After obtaining the feature map after channel attention and spatial attention processing, additional nonlinear transformation and feature interaction operations (such as convolution layers, activation functions, etc.) are introduced to further enhance the features, so that the features can better represent the surrounding information of the shape key points and improve the feature extraction ability of the model for complex structures and the accuracy of key point detection.

[0071] The feature map enhanced by the improved attention module is optimized in both channel and spatial dimensions, focusing more on the key points of the shape and their surrounding key regions, highlighting the feature information that contributes significantly to key point detection, and providing higher quality feature input for the final key point prediction.

[0072] Input data of the key point prediction module: feature map processed by the attention module (enhanced feature map).

[0073] Data processing process: usually one or more convolutional layers are used to process the input feature map, and the output channel number of the convolutional layer is equal to the number of key point categories (for example, for human pose estimation tasks, key point categories may include head, shoulder, elbow, etc., and the output channel number is the number of these key point categories). During convolution, each convolution kernel learns how to convert the features in the feature map into a probability response map for the corresponding key point category.

[0074] After the convolution operation, an activation function (such as a Sigmoid function) is used to map the convolution output value to the interval [0,1], indicating the probability of each pixel position belonging to the corresponding key point category. For example, in human pose estimation, the pixel value close to 1 on each key point probability map indicates that the position is likely to correspond to the corresponding human key point.

[0075] The probability map of key point prediction, each probability map corresponds to a key point category, indicating the probability of the key point appearing at each pixel position in the image. By setting a threshold (such as 0.5) or using non-maximum suppression, etc., the specific position coordinates of each key point are determined from these probability maps, thus completing the key point detection task and obtaining the shape key point data in the image.

[0076] The entire process described above works through the cooperation of various modules, starting from the original image data, and through preprocessing, feature extraction, feature enhancement, feature fusion, and key point prediction, etc. multiple steps, ultimately realizing the automatic detection of shape key points in the image, obtaining the accurate position data of the key points, and through the introduction of deformable convolution and CAE attention module, etc. Improvement measures, enhance the model's feature extraction ability for complex structure of shape, improve the accuracy and robustness of key point detection.

[0077] Optionally, the preset shape representation and transformation unified model includes a shape representation module, a shape change module, and a shape similarity comparison module; the key point data of all the original image data is input into the preset shape representation and transformation unified model for processing to obtain the categories of different shapes in each original image data, including: The key point data in each of the original image data is processed by the shape representation module to obtain the corresponding representation data of the different shapes; The shape change module calculates the representation data of the different shapes corresponding to different original image data to obtain the geodesic distance between the different shapes; The shape similarity comparison module clusters and analyzes the geodesic distance between the different shapes to obtain the categories of different shapes in each original image data.

[0078] Optionally, the step of processing the key point data in each of the original image data through the shape representation module to obtain the corresponding representation data of different shapes includes: Based on a preset Gaussian mixture algorithm, the key point data is processed to obtain representation data of different shapes; The preset Gaussian mixture algorithm is as follows: ; in, It is a set composed of the key point data. It is the number of key points. It corresponds to any point in the original image data. It is the first Location data of key points; For variance data, The probability density data for the x-th point is the representation data of the different shapes.

[0079] Specifically, such as Figure 3 Schematic diagrams of different shapes in different images. Figure 3 The image includes a straight line with 5 key points (the initial shape, corresponding to the circles), and the shape of the straight line with 5 key points rotated around a point (corresponding to the squares; the rotated shape comes from different images, i.e., other image data). In other words, the original image consists of two images: one containing an initial shape (the straight line), and the other containing a rotated shape (such as the shape of the straight line rotated 90 degrees around a certain point).

[0080] In the shape representation module, both the initial shape (corresponding to the circles) and the rotated shape (corresponding to the squares) are represented using a Gaussian Mixture Model (GMM). Each keypoint corresponds to the mean of a Gaussian distribution, and all Gaussian distributions share the same variance. That is, the shape representation module processes the keypoints in each original image. For example, if the initial shape has 5 keypoints, then the GMM will have 5 Gaussian distributions, and the position of each keypoint is the mean of the corresponding Gaussian distribution. The keypoint (square) positions of the rotated shape are also used to construct a new Gaussian Mixture Model, reflecting the characteristics of the rotated shape. In other words, the keypoint data for each shape is converted into a probability density function.

[0081] When using Gaussian Mixture Models (GMMs) for shape representation, the choice of keypoint locations has a significant impact on the model's accuracy. As the mean of various Gaussian distributions within the GMM, the accuracy of keypoint locations directly affects the model's representation of the shape. If the keypoint locations deviate from the center of the true distribution, the model may fail to accurately capture the shape's features, leading to distorted probability density estimation. The following is a detailed explanation of the evaluation and optimization methods: In some embodiments, the goodness of fit of GMM to real data under different mean selection is evaluated by computing the log-likelihood values on the test set. The log-likelihood value measures the explanatory power of the model to the observed data, and a higher log-likelihood value usually indicates a better fit of the model to the data. The specific steps are as follows: Prepare test set: Select a set of test data with known shape features.

[0082] Fit GMM: Train GMM models for different mean initialization methods.

[0083] Compute log-likelihood values: Use the trained GMM models to compute the log-likelihood values of the test set data.

[0084] Compare results: Compare the log-likelihood values under different mean initialization methods and select the model with higher log-likelihood values.

[0085] Optimization strategies: Adaptive initialization: Use K-means++ initialization method instead of traditional uniform sampling. K-means++ reduces the possibility of the model falling into local optimum by selecting initial cluster centers, ensuring that the means cover the main features of the shape.

[0086] Curvature-based key point detection method: For complex shape regions (high curvature regions), increase the density of key points to ensure that the features of these regions are fully captured.

[0087] Hierarchical GMM: First use coarse-grained mean initialization to construct a simple GMM model that can capture the general features of the shape. According to the fitting residual of the model, dynamically add more Gaussian components in the regions with larger residuals, gradually refine the model, and improve the ability to capture shape details.

[0088] Joint optimization: Treat the mean position as a learnable parameter and optimize it end-to-end by combining the EM (Expectation Maximization) algorithm and gradient descent algorithm. Under the framework of the EM algorithm, use the gradient descent method to update the mean position to maximize the log-likelihood function and improve the model's fitting ability to the data.

[0089] The position of key points as means in GMM is crucial to shape representation accuracy. Through adaptive initialization, hierarchical GMM and joint optimization strategies, the accuracy of GMM representation of shape can be improved, avoiding the model falling into local optimum, so as to more accurately capture shape features and improve the accuracy of probability density estimation. The evaluation method helps to select the optimal mean initialization method by comparing the log-likelihood values.

[0090] Optionally, the transforming the representation data of different shapes by the shape change module to obtain the geodesic distance between different shapes comprises: According to the probability density data in each of the original image data, corresponding metric tensor data is calculated; According to the metric tensor data of different original image data, geodesics are determined, and corresponding geodesic distances between different shapes are obtained according to the geodesics.

[0091] Specifically, the metric tensor data of each shape is calculated using the Fisher-Rao metric tensor formula. The shape parameter space is regarded as a manifold, and the metric tensor is used to define the geometric structure of the manifold. By solving the geodesic equation, the geodesic path between two shapes is determined. And by integrating the metric tensor (metric tensor data) on the geodesic path, the geodesic distance between two shapes is calculated.

[0092] As shown in the initial path (dashed line) in the figure: represents the position of the initial shape in the shape space, and the simple transformation path (such as straight line path) when the structure of the shape manifold is not considered. This can be regarded as an initial guess or direct interpolation path of shape transformation. Figure 3

[0093] Geodesic path: the shortest path calculated by the shape change module that meets the statistical manifold arc line properties, Figure 1 The optimal transformation path of the shape from the initial state to the rotated state is shown in the figure. It reflects the real gradual change process of the shape in the manifold space. From the results, it can be seen that the obtained geodesic has a smooth change, which can reflect the shape transformation process to a certain extent. The geodesic path shows how the initial shape is gradually transformed into the rotated shape in the shape space (manifold defined by GMM parameters). This path is obtained by optimizing the shortest path between two points on the manifold.

[0094] Optionally, the shape similarity comparison module is used to cluster analyze the geodesic distance between different shapes to obtain the categories of different shapes in each of the original image data, comprising: A similarity matrix is constructed according to each of the geodesic distances; According to a predetermined clustering algorithm, the similarity matrix is clustered analyzed to obtain the corresponding categories of different shapes in each of the original image data.

[0095] ​Specifically, the geodesic distances between all pairs of shapes are obtained from the shape variation module. These distances reflect the similarity or difference between different shapes. These geodesic distances are organized into a matrix form, denoted as a similarity matrix. The rows and columns of the matrix correspond to different shapes, and the elements in the matrix represent the geodesic distances between the corresponding two shapes. A suitable clustering algorithm is selected, such as K-means, hierarchical clustering, DBSCAN, etc. Different clustering algorithms are suitable for different data distributions and clustering requirements.

[0096] The similarity matrix is input into the clustering algorithm to perform clustering analysis. The clustering algorithm groups similar shapes into the same category and different shapes into different categories based on the distance information in the similarity matrix, as shown in Figure 4 The schematic diagram of different shape categories after processing the 9 original image data (each original image data contains one shape) by the unified shape representation and transformation method. shape1, shape8, and shape5 are grouped into the same category, shape2, shape3, and shape6 are grouped into the same category, and shape4, shape7, and shape9 are grouped into different categories, respectively.

[0097] After the clustering algorithm is executed, the category labels of each shape are output. These labels divide different shapes in the original image data into different categories. According to the category labels, the relationship between different shapes can be further analyzed to support subsequent applications such as image recognition, classification, retrieval, etc.

[0098] The above process can systematically classify shapes by constructing a similarity matrix and applying a clustering algorithm, avoiding the subjectivity and inconsistency of manual classification. The clustering algorithm can efficiently process a large amount of shape data, quickly group similar shapes into a category, and improve the efficiency of shape classification.

[0099] The clustering analysis based on geodesic distance can more accurately reflect the similarity between shapes, and compared with traditional clustering methods based on Euclidean distance, it can better capture the essential differences between shapes.

[0100] This method can be adjusted according to different clustering algorithms and clustering parameters to adapt to different application scenarios and data characteristics, and has strong adaptability.

[0101] In some embodiments, multiple indicators can be used to evaluate shape similarity, which measure the similarity between the transformed shape and the original shape from different angles: Perimeter Similarity: This is used to compare the change in perimeter between the transformed shape and the original shape. The value is defined as the ratio of the transformed perimeter to the original perimeter. If this value is close to 1, it indicates that the change in perimeter is small, and the shape similarity is high.

[0102] Area Similarity: This is used to compare the change in area between the transformed shape and the original shape. The value is defined as the ratio of the transformed area to the original area. If this value is close to 1, it means that the change in area is small, and the shape similarity is high.

[0103] Radial Length Similarity: This is used to compare the change in radial length sum between the transformed shape and the original shape. Radial length refers to the distance from a point on the tumor edge to the tumor center point. If this similarity value is close to 1, it indicates that the change in radial length sum is small, and the shape similarity is high.

[0104] In addition, other indicators can also be used for further measurement: Shape Coverage: Calculate the volume ratio of the union of shapes in the same cluster to the convex hull. The closer this value is to 1, the higher the overlap between shapes, and the greater the similarity.

[0105] Average Symmetric Surface Distance (ASSD): Calculate the average distance between the surface point sets of two shapes in the same cluster. The smaller the distance, the more similar the shapes.

[0106] Similarity based on deep features: Use a pre-trained deep learning network (such as PointNet or ResNet) to extract shape features, and calculate the average cosine similarity of the features within the cluster. The higher the average similarity, the closer the shapes are in the feature space, and the stronger the similarity.

[0107] By comprehensively using these indicators, the similarity between shapes can be more comprehensively evaluated.

[0108] In some embodiments, in medical image analysis, identifying lesion shapes is a crucial task that can assist doctors in early diagnosis and treatment planning of diseases. The unified shape representation and transformation method process is as follows: Collect medical image data, such as CT scans, MRI images, etc. These images contain lesion tissues of different patients, such as tumors, cysts, etc.

[0109] Use a pre-set detection model to process the medical images and identify and locate the key points of the lesion shape. These key points can be boundary points, center points, etc. of the lesion tissue.

[0110] Input all key point data into a pre-set shape representation and transformation unified model for processing. The processing process includes: The extracted key point data is input into the unified model, and the key point data is converted into a probabilistic density representation of shape (such as a Gaussian mixture model) through the shape representation module.

[0111] The geodesic distance between different lesion shapes is calculated through the shape variation module, reflecting their similarity.

[0112] The geodesic distance is clustered and analyzed through the shape similarity comparison module, and similar lesion shapes are classified into the same category. For example, benign and malignant tumor shapes can be distinguished.

[0113] In medical image analysis, through automated shape analysis, human error can be reduced, and the accuracy and consistency of lesion shape recognition can be improved, thereby providing quantitative analysis results for doctors to assist in developing treatment plans.

[0114] In other embodiments, in industrial detection, the classification of product shapes is an important link to ensure product quality and consistency.

[0115] The process of the unified shape representation and transformation method is as follows: Collect product images on the production line, which contain products of different shapes, such as screws, nuts, gears, etc.

[0116] Use a pre-set detection model to process the product images, identify and locate the key points of the product shape. These key points can be edge points, hole points, etc.

[0117] The extracted key point data is input into the unified model (which includes the shape representation module, the shape variation module, and the shape similarity comparison module), and the key point data is converted into a probabilistic density representation of shape (such as a Gaussian mixture model) through the shape representation module.

[0118] The geodesic distance between different product shapes is calculated through the shape variation module, reflecting their similarity.

[0119] The geodesic distance is clustered and analyzed through the shape similarity comparison module, and similar product shapes are classified into the same category. For example, the shapes of qualified and unqualified products can be distinguished.

[0120] In industrial detection, this method can significantly improve detection speed through automated shape classification, meeting the needs of large-scale production. At the same time, it helps to identify and classify unqualified products, improving product quality.

[0121] Whether it is medical image analysis or industrial detection, the above shape representation and transformation method can achieve automatic recognition, classification, and analysis of shapes through a unified model. This method has the advantages of improving accuracy, efficiency, and adaptability, providing strong technical support for related fields.

[0122] As Figure 5 shown, the embodiment of the present application provides a unified shape representation and transformation system, comprising: An acquisition module is configured to acquire original image data. A processing module is configured to process each of the original image data based on a preset detection model to obtain corresponding key point data. The processing module is further configured to input all the key point data into a preset shape representation and transformation unified model for processing to obtain categories of different shapes in each of the original image data; wherein the shape representation and transformation unified model is configured to identify different shapes in each of the original image data and determine corresponding categories.

[0123] The embodiment of the present application provides a unified shape representation and transformation device, comprising a memory and a processor; the memory is configured to store a computer program; the processor is configured to implement the unified shape representation and transformation method as described above when the computer program is executed.

[0124] The embodiment of the present application provides a computer readable storage medium, and the storage medium stores a computer program. When the computer program is executed by a processor, the unified shape representation and transformation method as described above is implemented.

[0125] A unified shape representation and transformation system that can be a server or a client of the present application will now be described, which is an example of a hardware device that can be applied to various aspects of the present application. The unified shape representation and transformation system is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The unified shape representation and transformation system can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are merely examples and are not intended to limit the implementations described and / or claimed in this document.

[0126] Although the present application is disclosed as above, the protection scope of the present application is not limited to this. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present application, and these changes and modifications shall fall within the protection scope of the present application.

Claims

1. A unified shape representation and transformation method, characterized by, The method comprises: obtaining original image data; processing each of the original image data based on a preset detection model to obtain corresponding key point data; inputting all the key point data into a preset shape representation and transformation unified model for processing to obtain categories of different shapes in each of the original image data; wherein the shape representation and transformation unified model is used to identify different shapes in each of the original image data and determine corresponding categories.

2. The unified shape representation and transformation method of claim 1, wherein, The preset detection model comprises a feature extraction module, an improved attention module and a key point detection module; the processing of each of the original image data based on the preset detection model to obtain key point data comprises: preprocessing each of the original image data to obtain temporary image data; processing the temporary image data through the feature extraction module to obtain corresponding fusion feature maps; enhancing the fusion feature maps through the improved attention module to obtain enhanced feature maps; processing the enhanced feature maps through the key point detection module to obtain the key point data.

3. The unified shape representation and transformation method of claim 2, wherein, The feature extraction module comprises an HRNet module, a deformable convolution module and a fusion module; the processing of the temporary image data through the feature extraction module to obtain corresponding fusion feature maps comprises: extracting features of the temporary image data through the HRNet module to obtain multi-scale feature maps; convolving the multi-scale feature maps through the deformable convolution module to obtain temporary feature maps; fusing the multi-scale feature maps and the temporary feature maps through the fusion module to obtain corresponding fusion feature maps.

4. The unified shape representation and transformation method of claim 1, wherein, The preset shape representation and transformation unified model comprises a shape representation module, a shape change module and a shape similarity comparison module; the processing of all the key point data into the preset shape representation and transformation unified model to obtain categories of different shapes in each of the original image data comprises: processing the key point data in each of the original image data through the shape representation module to obtain corresponding representation data of the different shapes; calculating the representation data of the different shapes corresponding to different original image data through the shape change module to obtain geodesic distance between the different shapes; performing cluster analysis on the geodesic distance between the different shapes through the shape similarity comparison module to obtain categories of different shapes in each of the original image data.

5. The unified shape representation and transformation method of claim 4, wherein, The processing of the key point data through the shape representation module to obtain corresponding representation data of the different shapes comprises: processing the key point data based on a preset Gaussian mixture algorithm to obtain the representation data of the different shapes; wherein the preset Gaussian mixture algorithm is: ； in, It is a set composed of the key point data. It is the number of key points. It corresponds to any point in the original image data. It is the first Location data of key points; For variance data, The probability density data for the x-th point is the representation data of the different shapes.

6. The unified shape representation and transformation method of claim 5, wherein, the transformation processing of the representation data of the different shapes through the shape change module to obtain geodesic distance between the different shapes comprises: calculating corresponding metric tensor data according to the probability density data in each of the original image data; Geodesic lines are determined according to the metric tensor data of different original image data, and the geodesic line distances between different shapes are obtained according to the geodesic lines.

7. The unified shape representation and transformation method of claim 6, wherein, The geodesic line distances between different shapes are clustered by the shape similarity comparison module, and the categories of different shapes in each original image data are obtained, including: A similarity matrix is constructed according to each geodesic line distance; The similarity matrix is clustered according to a preset clustering algorithm, and the categories corresponding to different shapes in each original image data are obtained.

8. A unified shape representation and transformation system, characterized by, Including: An acquisition module is configured to acquire original image data; A processing module is configured to process each original image data based on a preset detection model to obtain corresponding key point data; The processing module is further configured to input all the key point data into a preset shape representation and transformation unified model for processing to obtain the categories of different shapes in each original image data; wherein the shape representation and transformation unified model is used to identify different shapes in each original image data and determine the corresponding categories.

9. A uniform shape representation and transformation device, characterized by The memory is configured to store a computer program; and the processor is configured to implement the unified shape representation and transformation method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The storage medium has a computer program stored thereon, and when the computer program is executed by the processor, the unified shape representation and transformation method according to any one of claims 1 to 7 is implemented.

Images