Multi-label image spatial localization and enhancement method based on polyhedral image features

Abstract

The invention discloses a multi-label image spatial positioning and enhancement method based on polyhedral image features, and relates to the technical field of image processing. In the enhancement stage, the invention presets a series of Gaussian kernels of different sizes for sub-region images corresponding to each target object, calculates horizontal and vertical gradients after convolution with the sub-region images, obtains gradient information at different Gaussian kernel scales, performs comprehensive evaluation to obtain a comprehensive evaluation index, and then determines the weights of each scale for linear fusion to obtain fused gradient information. This comprehensive evaluation and weight distribution method fully considers the quantity and quality of edges, so that the fused gradient information can better reflect the edge features of the target object, provides high-quality input data for subsequent enhancement operations such as sharpening, helps to improve the pertinence and effectiveness of the enhancement effect, and overcomes the problem of poor enhancement effect caused by the lack of reasonable evaluation and fusion of edge features of different scales in traditional methods.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a multi-label image spatial positioning and enhancement method based on polyhedral image features. Background Art

[0002] In today's digital age, image data is showing explosive growth. Whether in security monitoring, autonomous driving, virtual reality or image editing and many other fields, there is an extremely urgent need for the precise spatial positioning of target objects in the image and the enhancement of image quality.

[0003] However, the existing multi-label image spatial positioning and enhancement methods still have the following shortcomings:

[0004] Most of them process the whole image uniformly, but fail to divide the multi-label image into sub-region images according to the location information of the target object obtained by positioning, and process each sub-region image separately, which makes it impossible to perform personalized enhancement operations according to the characteristics of different target objects and adapt to the diverse needs of different targets in multi-label images;

[0005] In addition, the enhancement stage lacks multi-scale comprehensive evaluation based only on a single indicator or scale, that is, reasonable evaluation of edge features at different scales and evaluation of sharpening effects at different scales, resulting in unsatisfactory enhancement effects.

[0006] Therefore, a multi-label image spatial positioning and enhancement method based on polyhedral image features is proposed. Summary of the invention

[0007] The purpose of the present invention is to propose a multi-label image spatial positioning and enhancement method based on polyhedral image features in order to solve the problem.

[0008] The object of the present invention can be achieved by the following technical solution: A multi-label image spatial positioning and enhancement method based on polyhedral image features, comprising:

[0009] Image preprocessing: collect multi-label image data and perform preliminary processing. After the preliminary processing is completed, polyhedral image features are extracted from the multi-label image data, each group of polyhedral image features is assigned a unique number and a feature database is established; the number is represented by i, i=1, 2, ..., k; k is the total number of polyhedral image features;

[0010] Positioning stage: access the established feature database, extract each group of polyhedral image features from it, set the calibration parameters of the image acquisition device, and after the setting is completed, execute the corresponding steps to determine the position information of each target object in the multi-label image in the three-dimensional space; based on the position information of each target object in the multi-label image in the three-dimensional space, further analyze and determine the relative position relationship between different target objects and construct a spatial position map of the multi-label image;

[0011] Enhancement stage: According to the position information of each target object in the multi-label image obtained by positioning in three-dimensional space, the multi-label image is divided into sub-region images corresponding to each target object, and corresponding steps are performed to process the sub-region images corresponding to each target object. After the processing is completed, an enhanced multi-label image is obtained;

[0012] Output display: Output the spatial positioning information and enhanced image corresponding to the multi-label image and store them.

[0013] As a preferred implementation of the present invention, after the setting is completed, corresponding steps are performed to determine the position information of each target object in the multi-label image in the three-dimensional space, specifically:

[0014] S1: extracting each group of polyhedral image features from the feature database, and sorting them according to the numbers of each group of polyhedral image features to obtain a feature data set formed by each group of polyhedral image features;

[0015] S2: Obtain each vertex of each group of polyhedral image features, represented by (x, y, z), use linear algebra methods to determine the three-dimensional coordinates of each vertex, and substitute it into the pinhole camera model equation for calculation to obtain the position and shape of the target object corresponding to each group of polyhedral features in three-dimensional space.

[0016] As a preferred embodiment of the present invention, the relative position relationship between different target objects is determined and a spatial position map of a multi-label image is constructed, specifically:

[0017] S3: Based on the three-dimensional coordinates of each vertex of each group of polyhedral image features in step S2, the average values ​​of the coordinates of all vertices of each group of polyhedral image features in each dimension are calculated to obtain the coordinates of the center point of each group of polyhedral image features, and then the distance between the target objects corresponding to each group of polyhedral image features is calculated using the Euclidean distance formula for the coordinates of the center point of each group of polyhedral image features;

[0018] Calculate the direction vector between each group of polyhedral image features, that is, for the coordinates of the center point of feature number 1 and the coordinates of the center point of feature number 2, calculate the direction vector by (x1-x2, y1-y2, z1-z2), and then calculate the angle between the direction vector and the positive direction vector of the coordinate axis to determine the direction between each group of polyhedral image features corresponding to the target object;

[0019] The target objects corresponding to each group of polyhedral image features are segmented by using an image segmentation algorithm. After the segmentation is completed, if it is found that two groups of target objects partially overlap in the image, the z coordinates between the relevant vertices of the two groups of polyhedral image features are further compared. If the z coordinates of the relevant vertices of one group of polyhedral image features are smaller than the z coordinates of the relevant vertices of the other group of polyhedral image features, the front and back occlusion relationship and position order between the target objects corresponding to each group of polyhedral image features are determined;

[0020] Obtain the face information of each group of polyhedral image features and the vertex coordinates corresponding to the face information, compare the vertex coordinates of each group of polyhedral image feature face information, preset the coordinate error reference value corresponding to the face information, and similarly for the vertex coordinates of the face information of feature number 1 and the vertex coordinates of the face information of feature number 2, if there is , and counting the number of vertex coordinates that satisfy the comparison relationship. If the counted number is greater than a preset reference number, it is determined that the surface information corresponding to feature number 1 and feature number 2 belongs to a shared surface; wherein r represents a coordinate error reference value corresponding to the preset surface information;

[0021] Obtaining edge information of each group of polyhedral image features and vertex coordinates corresponding to the edge information, comparing the vertex coordinates of each group of polyhedral image feature edge information, and presetting a coordinate error reference value corresponding to the edge information; if the direction vector difference between the vertex coordinates corresponding to one group of edge information of two groups of polyhedral image features is less than g, and the number of vertex coordinates less than g is greater than a preset reference number, it is determined that the edge information corresponding to the two groups of polyhedral features belongs to a shared edge; wherein g represents the preset coordinate error reference value corresponding to the surface information;

[0022] S4: Determine the relative position relationship between different target objects in the multi-label image according to the distance, direction, front and rear occlusion relationship, position order, shared surface and shared edge determined in step S3.

[0023] As a preferred implementation of the present invention, corresponding steps are performed to process the sub-region images corresponding to each target object, specifically:

[0024] M1: For the sub-region images corresponding to each group of target objects, a series of Gaussian kernels of different sizes are preset; for each preset Gaussian kernel, it is convolved with each sub-region image. After the convolution is completed, the gradient calculation method is used to calculate the horizontal and vertical gradients of the convolved sub-region image, so as to obtain the gradient information of each sub-region image under different Gaussian kernel scales;

[0025] M2: For the gradient information of each sub-region image under different Gaussian kernel scales, first calculate the gradient amplitude of each sub-region image under different Gaussian kernel scales, and preset the reference evaluation value corresponding to the gradient amplitude, compare the gradient amplitude of each sub-region image under different Gaussian kernel scales with the preset reference evaluation value, and count the number of pixels whose gradient amplitude of each sub-region image under different Gaussian kernel scales is greater than the preset reference evaluation value as the effect evaluation index.

[0026] As a preferred embodiment of the present invention, executing corresponding steps to process the sub-region images corresponding to each target object also includes:

[0027] M3: For each pixel point of each sub-region image under different Gaussian kernel scales and the set neighborhood range, calculate the gradient direction variance of each pixel point in each sub-region image and its neighborhood; take the mean of each group of gradient direction variances in each sub-region image to obtain the consistency evaluation index of each sub-region image under different Gaussian kernel scales;

[0028] The effect evaluation index and consistency evaluation index of each sub-region image under different Gaussian kernel scales are marked as te and tu respectively, and substituted into the formula A weighted calculation is performed to obtain the comprehensive evaluation index tp of each sub-region image under different Gaussian kernel scales; and They represent the expected effect evaluation index and the expected consistency evaluation index corresponding to each sub-region image respectively; a1 and a2 are the influencing weight factors of the effect evaluation index te and the consistency evaluation index tu respectively;

[0029] The average of the comprehensive evaluation index tp of each sub-region image under different Gaussian kernel scales is taken as the enhancement performance index of different Gaussian kernel scales for multi-label images; the sum of the enhancement performance index of each group is taken as the sum of the indexes, and the proportion of the enhancement performance index corresponding to different Gaussian kernel scales in the sum of the indexes is calculated respectively as the weight corresponding to different Gaussian kernel scales;

[0030] M4: Based on the weights corresponding to different Gaussian kernel scales, a weighted linear fusion strategy is used to fuse the images to obtain the fused gradient information corresponding to the multi-label image.

[0031] As a preferred embodiment of the present invention, executing corresponding steps to process the sub-region images corresponding to each target object also includes:

[0032] M5: Select high-pass filters of different radii; after the selection is completed, based on the fused gradient information corresponding to the multi-label image obtained in step M4, use high-pass filters of different radii to sharpen each sub-region image of the multi-label image, calculate the gradient amplitude of each sub-region image after sharpening by high-pass filters of different radii, and count the average value of the gradient amplitude in each sub-region image as the sharpening effect index of high-pass filters of different radii;

[0033] Performing discrete Fourier transform on each sub-region image after sharpening by high-pass filters of different radii, converting the image into the frequency domain, obtaining the frequency domain image of each sub-region image, presetting the cutoff frequency corresponding to each sub-region image, marking the area above the cutoff frequency as the high-frequency area, and calculating the energy evaluation index of the high-frequency area corresponding to each sub-region image;

[0034] Calculate the difference between each sub-region image after sharpening by high-pass filters of different radii and each sub-region image corresponding to the fused gradient information in step M4, that is, calculate the structural similarity index corresponding to each sub-region image;

[0035] The sharpening effect index, energy evaluation index and structural similarity index corresponding to each sub-region image after sharpening processing by high-pass filters of different radii are marked as va, vb and vc respectively;

[0036] Substitute the sharpening effect index va, energy evaluation index vb and structural similarity index vc corresponding to each sub-region image after sharpening by high-pass filters of different radii into the formula A weighted calculation is performed to obtain the sharpening processing index vt of each sub-region image after sharpening processing by high-pass filters of different radii; wherein and are the expected sharpening effect index and expected energy evaluation index corresponding to the image sharpening effect index va and energy evaluation index vb of each sub-region respectively; is the expected minimum difference corresponding to the image structure similarity index of each sub-region; b1, b2 and b3 are the influence weight factors of the sharpening effect index va, the energy evaluation index vb and the structure similarity index vc respectively;

[0037] For each sub-region image after sharpening processing by high-pass filters of different radii, the sub-region image with the highest sharpening processing index vt is selected as the final sub-region image after sharpening processing.

[0038] As a preferred embodiment of the present invention, after the processing is completed, the enhanced multi-label image is obtained, specifically:

[0039] M6: After post-processing based on steps M1-M5, an enhanced multi-label image is obtained.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] The present invention extracts the polyhedral image features from the feature database in the positioning stage, first uses a linear algebra method to determine the three-dimensional coordinates of each vertex, and substitutes them into the pinhole camera model equation to calculate the position and shape of the target object in the three-dimensional space. Then, through a series of calculations and analyses, the geometric features, spatial position relationship and occlusion relationship of the target object are comprehensively considered to comprehensively determine the position information of the target object in the three-dimensional space from multiple dimensions. Compared with the traditional single-factor positioning method, the positioning accuracy is greatly improved, and the spatial position relationship between different target objects in the multi-label image can be more accurately described.

[0042] The present invention presets a series of Gaussian kernels of different sizes for the sub-region images corresponding to each target object in the enhancement stage, calculates the horizontal and vertical gradients after convolution with the sub-region images, obtains the gradient information at different Gaussian kernel scales, and performs comprehensive evaluation to obtain the comprehensive evaluation index, and then determines the weights of each scale for linear fusion to obtain the fused gradient information. This comprehensive evaluation and weight distribution method fully considers the quantity and quality of the edges, so that the fused gradient information can better reflect the edge features of the target object, and provides high-quality input data for subsequent enhancement operations such as sharpening, which helps to improve the pertinence and effectiveness of the enhancement effect, and overcomes the problem of poor enhancement effect caused by the lack of reasonable evaluation and fusion of edge features of different scales in traditional methods.

[0043] The present invention selects high-pass filters of different radii to sharpen the sub-region image that integrates gradient information, calculates the sharpening effect index, high-frequency region energy evaluation index and structural similarity index of the sharpened image, obtains the sharpening processing index through weighted calculation, and selects the sub-region image with the highest sharpening processing index through multi-scale sharpening and comprehensive evaluation indexes, thereby avoiding the one-sidedness of judging the sharpening effect based on only a single indicator, achieving a better balance in multiple aspects such as edge enhancement and detail retention, improving the overall sharpening effect of the image, and improving the unsatisfactory enhancement effect caused by the lack of multi-scale comprehensive evaluation in traditional sharpening methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0045] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0046] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] See also Figure 1 As shown, the multi-label image spatial positioning and enhancement method based on polyhedral image features includes:

[0048] Image preprocessing: Collect multi-label image data and perform preliminary processing; preliminary processing includes denoising, grayscale, image size normalization and other operations to improve the accuracy and efficiency of subsequent processing;

[0049] It should be noted that appropriate image acquisition equipment (such as cameras, scanners, etc.) is used to acquire image data, and then denoising algorithms such as Gaussian filtering are used to remove noise interference in the image, the color image is converted into a grayscale image to simplify the calculation, and finally the image size is adjusted to a uniform standard size.

[0050] After the preliminary processing is completed, the Canny edge detection algorithm is used to extract polyhedral image features from the multi-label image data, and each group of polyhedral image features is assigned a unique number and a feature database is established; the feature database stores the vertex coordinates, face information, and identification data of each group of polyhedral features for subsequent query and processing; the number is represented by i, i=1, 2, ..., k; k is the total number of polyhedral image features;

[0051] It should be noted that the Canny edge detection algorithm is used to detect the edge of the preprocessed grayscale image, and the appropriate threshold parameters are set to accurately extract the edge information in the image. The extracted edges are connected and fitted. Through morphological processing (such as expansion, corrosion, etc.) and line segment fitting algorithms, the edges are combined into faces and vertices of polyhedrons. For targets with complex shapes, multi-scale analysis technology is used to extract features at different scales, and then fusion is performed to ensure the integrity of the polyhedron features.

[0052] Positioning stage: access the established feature database, extract each group of polyhedral image features from it, and set the calibration parameters of the image acquisition device; the image acquisition device is a high-definition camera, and the calibration parameters include the focal length of the device, the coordinates of the principal point, etc. These parameters can be obtained through a preset calibration process, such as using a chessboard calibration plate for calibration calculation, placing the chessboard at different positions and angles within the camera's field of view, and taking multiple groups of images; using a corner detection algorithm (such as Harris corner detection) to find the position of the corner points in the chessboard in the image, and according to the known chessboard size and the image coordinates of the corner points, calculate the calibration parameters of the camera through geometric relationships such as the homography matrix; after the setting is completed, execute the corresponding steps to determine the position information of each target object in the multi-label image in the three-dimensional space; based on the position information of each target object in the multi-label image in the three-dimensional space, further analyze and determine the relative position relationship between different target objects and construct a spatial position map of the multi-label image; select a professional three-dimensional visualization library to construct the spatial position map, integrate the spatial position information of each target object, and form a complete spatial layout description corresponding to the multi-label image;

[0053] Specifically:

[0054] S1: extracting each group of polyhedron image features from the feature database, and sorting them according to the numbers of each group of polyhedron image features to obtain a feature data set formed by each group of polyhedron image features; for example, all vertex coordinates, face information, etc. of the polyhedron numbered "1" are sorted together to facilitate subsequent calculation and analysis of the spatial position of each polyhedron;

[0055] S2: Obtain each vertex of each group of polyhedral image features, represented by (x, y, z), use linear algebra to determine the three-dimensional coordinates of each vertex, and substitute it into the pinhole camera model equation for calculation to obtain the position and shape of the target object corresponding to each group of polyhedral features in three-dimensional space; the pinhole camera model describes the projection relationship from a three-dimensional space point to a two-dimensional image plane;

[0056] It should be noted that after obtaining the vertex coordinates of the polyhedral image features, outlier detection and processing are first performed, and by setting a reasonable coordinate range threshold, those abnormal vertex coordinates that obviously deviate from the normal range are identified and eliminated.

[0057] S3: Based on the three-dimensional coordinates of each vertex of each group of polyhedral image features in step S2, the average values ​​of the coordinates of all vertices of each group of polyhedral image features in each dimension are calculated to obtain the coordinates of the center point of each group of polyhedral image features, and then the distance between the target objects corresponding to each group of polyhedral image features is calculated using the Euclidean distance formula for the coordinates of the center point of each group of polyhedral image features;

[0058] Calculate the direction vector between each group of polyhedral image features, that is, for the coordinates of the center point of feature number 1 and the coordinates of the center point of feature number 2, calculate the direction vector by (x1-x2, y1-y2, z1-z2), and then calculate the angle between the direction vector and the positive direction vector of the coordinate axis to determine the direction between each group of polyhedral image features corresponding to the target object;

[0059] The target objects corresponding to each group of polyhedral image features are segmented by using an image segmentation algorithm. After the segmentation is completed, if it is found that two groups of target objects partially overlap in the image, the z coordinates between the relevant vertices of the two groups of polyhedral image features are further compared. If the z coordinates of the relevant vertices of one group of polyhedral image features are smaller than the z coordinates of the relevant vertices of the other group of polyhedral image features, the front and back occlusion relationship and position order between the target objects corresponding to each group of polyhedral image features are determined;

[0060] Obtain the face information of each group of polyhedral image features and the vertex coordinates corresponding to the face information, compare the vertex coordinates of each group of polyhedral image feature face information, preset the coordinate error reference value corresponding to the face information, and similarly for the vertex coordinates of the face information of feature number 1 and the vertex coordinates of the face information of feature number 2, if there is , and counting the number of vertex coordinates that satisfy the comparison relationship. If the counted number is greater than a preset reference number, it is determined that the surface information corresponding to feature number 1 and feature number 2 belongs to a shared surface; wherein r represents a coordinate error reference value corresponding to the preset surface information;

[0061] Obtaining edge information of each group of polyhedral image features and vertex coordinates corresponding to the edge information, comparing the vertex coordinates of each group of polyhedral image feature edge information, and presetting a coordinate error reference value corresponding to the edge information; if the direction vector difference between the vertex coordinates corresponding to one group of edge information of two groups of polyhedral image features is less than g, and the number of vertex coordinates less than g is greater than a preset reference number, it is determined that the edge information corresponding to the two groups of polyhedral features belongs to a shared edge; wherein g represents the preset coordinate error reference value corresponding to the surface information;

[0062] S4: Determine the relative position relationship between different target objects in the multi-label image according to the distance, direction, front and rear occlusion relationship, position order, shared surface and shared edge determined in step S3;

[0063] It should be noted that through the above complete process, the distance and direction relationship obtained by calculating the center point coordinates, the occlusion relationship determined by depth information and image analysis, and the shared face / edge relationship determined based on geometric feature comparison are comprehensively considered, so that the relative position relationship between these different polyhedral target objects can be determined more comprehensively and accurately.

[0064] Enhancement stage: According to the position information of each target object in the multi-label image obtained by positioning in three-dimensional space, the multi-label image is divided into sub-region images corresponding to each target object, and corresponding steps are performed to process the sub-region images corresponding to each target object. After the processing is completed, an enhanced multi-label image is obtained;

[0065] Specifically:

[0066] M1: For the sub-region images corresponding to each group of target objects, a series of Gaussian kernels of different sizes are preset; the presets are made according to the detail scale ranges corresponding to the sub-region images that are expected to be detected; for each preset Gaussian kernel, it is convolved with each sub-region image, and after the convolution is completed, the gradient calculation method is used to calculate the horizontal and vertical gradients of the sub-region images after the convolution, so as to obtain the gradient information of each sub-region image under different Gaussian kernel scales;

[0067] M2: For the gradient information of each sub-region image under different Gaussian kernel scales, first calculate the gradient amplitude of each sub-region image under different Gaussian kernel scales, and preset the reference evaluation value corresponding to the gradient amplitude; the reference evaluation value is set according to the expectation of the edge strength of the image; compare the gradient amplitude of each sub-region image under different Gaussian kernel scales with the preset reference evaluation value, and count the number of pixels whose gradient amplitude of each sub-region image under different Gaussian kernel scales is greater than the preset reference evaluation value as the effect evaluation index;

[0068] M3: For each pixel point of each sub-region image at different Gaussian kernel scales and the set neighborhood range, calculate the variance of the gradient direction of each pixel point in each sub-region image and its neighborhood; the smaller the variance, the more consistent the gradient direction; take the mean of the variance of each group of gradient directions in each sub-region image to obtain the consistency evaluation index of each sub-region image at different Gaussian kernel scales;

[0069] The effect evaluation index and consistency evaluation index of each sub-region image under different Gaussian kernel scales are marked as te and tu respectively, and substituted into the formula A weighted calculation is performed to obtain the comprehensive evaluation index tp of each sub-region image under different Gaussian kernel scales; and They represent the expected effect evaluation index and the expected consistency evaluation index corresponding to each sub-region image respectively; a1 and a2 are the influencing weight factors of the effect evaluation index te and the consistency evaluation index tu respectively;

[0070] The average of the comprehensive evaluation index tp of each sub-region image under different Gaussian kernel scales is taken as the enhancement performance index of different Gaussian kernel scales for multi-label images; the sum of the enhancement performance index of each group is taken as the sum of the indexes, and the proportion of the enhancement performance index corresponding to different Gaussian kernel scales in the sum of the indexes is calculated respectively as the weight corresponding to different Gaussian kernel scales;

[0071] M4: Based on the weights corresponding to different Gaussian kernel scales, a weighted linear fusion strategy is used to fuse the images, thereby obtaining the fused gradient information corresponding to the multi-label image.

[0072] It should be noted that by presetting Gaussian kernels of different sizes in the M1 step, and performing convolution and subsequent gradient calculation with the sub-region image, it is possible to mine the edge information in the image from multiple scales. In the subsequent steps M2 to M4, the richness of the edges that can be effectively detected at each scale is first considered through the effect evaluation index (based on the number of pixels whose gradient amplitude is greater than the reference evaluation value), and then the consistency evaluation index (based on the mean of the gradient direction variance) is used to measure the coherence and reliability of the edge. The comprehensive evaluation index is then calculated by weighted calculation and the weight is determined, and finally weight-based linear fusion is achieved, which integrates the advantages of different scales. The fused gradient information not only contains rich edge details, but also ensures that these edges have good consistency in direction, which can more realistically and accurately reflect the edge features of the target object, and provide high-quality input data for subsequent image enhancement operations.

[0073] M5: Select high-pass filters of different radii; the specific values ​​of different radii are flexibly set according to the desired degree of sharpening; after the selection is completed, based on the fused gradient information corresponding to the multi-label image obtained in step M4, the sub-region images of the multi-label image are sharpened using high-pass filters of different radii, the gradient amplitude of each sub-region image after sharpening by high-pass filters of different radii is calculated, and the average value of the gradient amplitude in each sub-region image is counted as the sharpening effect index of high-pass filters of different radii; the higher the average value, the more obvious the edge enhancement effect after sharpening by the radius filter;

[0074] Perform discrete Fourier transform on each sub-region image after sharpening with high-pass filters of different radii, convert the image to the frequency domain, obtain the frequency domain image of each sub-region image, preset the cutoff frequency corresponding to each sub-region image, mark the area above the cutoff frequency as the high-frequency area, and calculate the energy evaluation index of the high-frequency area corresponding to each sub-region image; the higher the energy, the more high-frequency details are retained, and the better the effect;

[0075] The difference between each sub-region image after sharpening by high-pass filters of different radii and each sub-region image corresponding to the fused gradient information in step M4 is calculated, that is, the structural similarity index corresponding to each sub-region image is calculated; the closer the structural similarity index is to 1, the more similar the sub-region images after sharpening are to the sub-region images corresponding to the fused gradient information in structure and details, indicating that the details are better preserved;

[0076] The sharpening effect index, energy evaluation index and structural similarity index corresponding to each sub-region image after sharpening processing by high-pass filters of different radii are marked as va, vb and vc respectively;

[0077] Substitute the sharpening effect index va, energy evaluation index vb and structural similarity index vc corresponding to each sub-region image after sharpening by high-pass filters of different radii into the formula A weighted calculation is performed to obtain the sharpening processing index vt of each sub-region image after sharpening processing by high-pass filters of different radii; wherein and are the expected sharpening effect index and expected energy evaluation index corresponding to the image sharpening effect index va and energy evaluation index vb of each sub-region respectively; is the expected minimum difference corresponding to the image structure similarity index of each sub-region; b1, b2 and b3 are the influence weight factors of the sharpening effect index va, the energy evaluation index vb and the structure similarity index vc respectively;

[0078] For each sub-region image after sharpening by high-pass filters of different radii, the sub-region image with the highest sharpening index vt is selected as the final sub-region image after sharpening;

[0079] It should be noted that by calculating the sharpening index, we can avoid the one-sidedness of judging the sharpening effect based on a single indicator, and comprehensively measure the performance of sub-region images in multiple key aspects after sharpening by high-pass filters of different radii. Theoretically, the sub-region image with the highest sharpening index can indeed achieve a relatively better sharpening effect overall, and achieve a better balance in terms of edge enhancement, detail retention, etc.

[0080] M6: After post-processing based on steps M1-M5, an enhanced multi-label image is obtained;

[0081] It should be noted that, by default, after processing in steps M1-M5, basic color correction, denoising, etc. are performed.

[0082] Output display: output the spatial positioning information and enhanced image corresponding to the multi-label image, and store them;

[0083] It should be noted that the spatial position information of each target object in the multi-label image is output in the form of a text file, recording the target number, three-dimensional coordinates, direction and other information to facilitate subsequent data analysis and processing; at the same time, the spatial distribution of the target is displayed in the form of a visual chart (such as a three-dimensional coordinate chart) to intuitively present the spatial structure of the multi-label image;

[0084] The enhanced image is displayed on the display screen, and the user can visually observe the improvement effect of the image quality. The user can choose to save the enhanced image in a common image format (such as JPEG, PNG, etc.) according to the needs, so as to view and use it in other devices or software, or transmit the spatial positioning results and enhanced image data to a remote server or other related image processing system through a network interface for further analysis and processing;

[0085] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A multi-label image spatial positioning and enhancement method based on polyhedral image features, characterized in that: include: Image preprocessing: collect multi-label image data and perform preliminary processing. After the preliminary processing is completed, polyhedral image features are extracted from the multi-label image data, each group of polyhedral image features is assigned a unique number and a feature database is established; the number is represented by i, i=1, 2, ..., k; k is the total number of polyhedral image features; Positioning stage: access the established feature database, extract each group of polyhedral image features from it, set the calibration parameters of the image acquisition device, and after the setting is completed, perform the corresponding steps to determine the position information of each target object in the multi-label image in the three-dimensional space; Based on the position information of each target object in the multi-label image in three-dimensional space, further analyze and determine the relative position relationship between different target objects and construct a spatial position map of the multi-label image; Enhancement stage: According to the position information of each target object in the multi-label image obtained by positioning in three-dimensional space, the multi-label image is divided into sub-region images corresponding to each target object, and corresponding steps are performed to process the sub-region images corresponding to each target object. After the processing is completed, an enhanced multi-label image is obtained; Output display: Output the spatial positioning information and enhanced image corresponding to the multi-label image and store them.

2. The multi-label image spatial positioning and enhancement method based on polyhedral image features according to claim 1 is characterized in that: After the setting is completed, the corresponding steps are performed to determine the position information of each target object in the multi-label image in the three-dimensional space, specifically: S1: extracting each group of polyhedral image features from the feature database, and sorting them according to the numbers of each group of polyhedral image features to obtain a feature data set formed by each group of polyhedral image features; S2: Obtain each vertex of each group of polyhedral image features, represented by (x, y, z), use linear algebra methods to determine the three-dimensional coordinates of each vertex, and substitute it into the pinhole camera model equation for calculation to obtain the position and shape of the target object corresponding to each group of polyhedral features in three-dimensional space.

3. The multi-label image spatial positioning and enhancement method based on polyhedral image features according to claim 2 is characterized in that: Determine the relative position relationship between different target objects and construct a spatial position map of multi-label images, specifically: S3: Based on the three-dimensional coordinates of each vertex of each group of polyhedral image features in step S2, the average values ​​of the coordinates of all vertices of each group of polyhedral image features in each dimension are calculated to obtain the coordinates of the center point of each group of polyhedral image features, and then the distance between the target objects corresponding to each group of polyhedral image features is calculated using the Euclidean distance formula for the coordinates of the center point of each group of polyhedral image features; Calculate the direction vector between each group of polyhedral image features, that is, for the coordinates of the center point of feature number 1 and the coordinates of the center point of feature number 2, calculate the direction vector by (x1-x2, y1-y2, z1-z2), and then calculate the angle between the direction vector and the positive direction vector of the coordinate axis to determine the direction between each group of polyhedral image features corresponding to the target object; The target objects corresponding to each group of polyhedral image features are segmented by using an image segmentation algorithm. After the segmentation is completed, if it is found that two groups of target objects partially overlap in the image, the z coordinates between the relevant vertices of the two groups of polyhedral image features are further compared. If the z coordinates of the relevant vertices of one group of polyhedral image features are smaller than the z coordinates of the relevant vertices of the other group of polyhedral image features, the front and back occlusion relationship and position order between the target objects corresponding to each group of polyhedral image features are determined; Obtain the face information of each group of polyhedral image features and the vertex coordinates corresponding to the face information, compare the vertex coordinates of each group of polyhedral image feature face information, preset the coordinate error reference value corresponding to the face information, and similarly for the vertex coordinates of the face information of feature number 1 and the vertex coordinates of the face information of feature number 2, if there is , and counting the number of vertex coordinates that satisfy the comparison relationship. If the counted number is greater than a preset reference number, it is determined that the surface information corresponding to feature number 1 and feature number 2 belongs to a shared surface; wherein r represents a coordinate error reference value corresponding to the preset surface information; Obtaining edge information of each group of polyhedral image features and vertex coordinates corresponding to the edge information, comparing the vertex coordinates of the edge information of each group of polyhedral image features, and presetting a coordinate error reference value corresponding to the edge information; If the direction vector difference between the vertex coordinates corresponding to one set of edge information of two sets of polyhedral image features is less than g, and the number of vertex coordinates less than g is greater than a preset reference number, it is determined that the edge information corresponding to the two sets of polyhedral features belongs to a shared edge; wherein g represents a preset coordinate error reference value corresponding to the surface information; S4: Determine the relative position relationship between different target objects in the multi-label image according to the distance, direction, front and rear occlusion relationship, position order, shared surface and shared edge determined in step S3.

4. The multi-label image spatial positioning and enhancement method based on polyhedral image features according to claim 3 is characterized in that: Execute corresponding steps to process the sub-region images corresponding to each target object, specifically: M1: For the sub-region images corresponding to each group of target objects, a series of Gaussian kernels of different sizes are preset; for each preset Gaussian kernel, it is convolved with each sub-region image. After the convolution is completed, the gradient calculation method is used to calculate the horizontal and vertical gradients of the convolved sub-region image, so as to obtain the gradient information of each sub-region image under different Gaussian kernel scales; M2: For the gradient information of each sub-region image under different Gaussian kernel scales, first calculate the gradient amplitude of each sub-region image under different Gaussian kernel scales, and preset the reference evaluation value corresponding to the gradient amplitude, compare the gradient amplitude of each sub-region image under different Gaussian kernel scales with the preset reference evaluation value, and count the number of pixels whose gradient amplitude of each sub-region image under different Gaussian kernel scales is greater than the preset reference evaluation value as the effect evaluation index.

5. The multi-label image spatial positioning and enhancement method based on polyhedral image features according to claim 4 is characterized in that: Executing corresponding steps to process the sub-region images corresponding to each target object also includes: M3: For each pixel point of each sub-region image under different Gaussian kernel scales and the set neighborhood range, calculate the gradient direction variance of each pixel point in each sub-region image and its neighborhood; take the mean of each group of gradient direction variances in each sub-region image to obtain the consistency evaluation index of each sub-region image under different Gaussian kernel scales; The effect evaluation index and consistency evaluation index of each sub-region image under different Gaussian kernel scales are marked as te and tu respectively, and substituted into the formula A weighted calculation is performed to obtain the comprehensive evaluation index tp of each sub-region image under different Gaussian kernel scales; and They represent the expected effect evaluation index and the expected consistency evaluation index corresponding to each sub-region image respectively; a1 and a2 are the influencing weight factors of the effect evaluation index te and the consistency evaluation index tu respectively; The average of the comprehensive evaluation index tp of each sub-region image under different Gaussian kernel scales is taken as the enhancement performance index of different Gaussian kernel scales for multi-label images; the sum of the enhancement performance index of each group is taken as the sum of the indexes, and the proportion of the enhancement performance index corresponding to different Gaussian kernel scales in the sum of the indexes is calculated respectively as the weight corresponding to different Gaussian kernel scales; M4: Based on the weights corresponding to different Gaussian kernel scales, a weighted linear fusion strategy is used to fuse the images to obtain the fused gradient information corresponding to the multi-label image.

6. The multi-label image spatial positioning and enhancement method based on polyhedral image features according to claim 5, characterized in that: Executing corresponding steps to process the sub-region images corresponding to each target object also includes: M5: Select high-pass filters of different radii; after the selection is completed, based on the fused gradient information corresponding to the multi-label image obtained in step M4, use high-pass filters of different radii to sharpen each sub-region image of the multi-label image, calculate the gradient amplitude of each sub-region image after sharpening by high-pass filters of different radii, and count the average value of the gradient amplitude in each sub-region image as the sharpening effect index of high-pass filters of different radii; Performing discrete Fourier transform on each sub-region image after sharpening by high-pass filters of different radii, converting the image into the frequency domain, obtaining the frequency domain image of each sub-region image, presetting the cutoff frequency corresponding to each sub-region image, marking the area above the cutoff frequency as the high-frequency area, and calculating the energy evaluation index of the high-frequency area corresponding to each sub-region image; Calculate the difference between each sub-region image after sharpening by high-pass filters of different radii and each sub-region image corresponding to the fused gradient information in step M4, that is, calculate the structural similarity index corresponding to each sub-region image; The sharpening effect index, energy evaluation index and structural similarity index corresponding to each sub-region image after sharpening processing by high-pass filters of different radii are marked as va, vb and vc respectively; Substitute the sharpening effect index va, energy evaluation index vb and structural similarity index vc corresponding to each sub-region image after sharpening by high-pass filters of different radii into the formula A weighted calculation is performed to obtain the sharpening processing index vt of each sub-region image after sharpening processing by high-pass filters of different radii; wherein and are the expected sharpening effect index and expected energy evaluation index corresponding to the image sharpening effect index va and energy evaluation index vb of each sub-region respectively; is the expected minimum difference corresponding to the image structure similarity index of each sub-region; b1, b2 and b3 are the influence weight factors of the sharpening effect index va, the energy evaluation index vb and the structure similarity index vc respectively; For each sub-region image after sharpening processing by high-pass filters of different radii, the sub-region image with the highest sharpening processing index vt is selected as the final sub-region image after sharpening processing.

7. The multi-label image spatial positioning and enhancement method based on polyhedral image features according to claim 6 is characterized in that: After the processing is completed, the enhanced multi-label image is obtained, specifically: M6: After post-processing based on steps M1-M5, an enhanced multi-label image is obtained.

Images