A method for image clarity processing by fitting multiple samples

Abstract

The present invention discloses a method for image clarity processing by fitting multiple samples. This method performs fitting of multiple-sample images by combining reference points, block sampling, and edges. On the basis of ensuring accurate identification of reference points, it can achieve high fitting accuracy and complete recognition results through a point-to-plane positioning and fitting method, making this method more scientific. At the same time, aiming at the problem of large edge ambiguity, by optimizing long straight lines, it can quickly extract and fit a complete edge structure, greatly reducing the amount of data calculation and reducing the probability of problems such as misclassification caused by image distortion and defect, and the recognition results not meeting the subjective and objective consistency.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a method for image sharpening processing by fitting multiple samples. Background Art

[0002] At present, image processing methods are becoming increasingly perfect, and their applications are widely implemented in various working scenarios, such as image processing, face image recognition, urban satellite image fitting, etc., which have an impact on all aspects of existing life. In image processing, the research on methods for image sharpness is also diverse, with each having its own focus.

[0003] Specifically, in the aspect of processing images based on deep learning, multiple samples are often used for fitting. There are many such methods, which can be mainly summarized into three types: methods based on statistical or decision theory, methods based on structure or syntax, and methods based on neural networks.

[0004] However, the above methods still have some problems: due to the uncertainty of image data and uneven sample distribution, the accuracy of image recognition is greatly affected; and the recognition is easily affected by abnormal images, such as individual distorted images in the recognition image, resulting in misclassification, and the recognition results do not conform to the subjective and objective consistency, etc.

[0005] Therefore, it is necessary to provide a technical solution to solve the above technical problems. Summary of the Invention

[0006] In view of this, an embodiment of the present invention provides a method for image sharpening processing by fitting multiple samples.

[0007] A first aspect of an embodiment of the present invention provides a method for image sharpening processing by fitting multiple samples, the method comprising the following steps:

[0008] Preprocess the image to be fitted to obtain a first fitted image; wherein, the preprocessing includes processing different illumination intensities in the image to be fitted and normalizing the image parameters of the image to be fitted;

[0009] Perform path processing on the first fitted image to obtain a feature map; wherein, the path processing is convolution processing by N convolutional layers, where N is a preset integer;

[0010] According to the feature map, determine the corresponding landmark point information in each first fitted image; and according to the landmark point information, determine the reference point of the first fitted image;

[0011] Based on the reference point, perform multi-density block sampling, and by setting a loss function, obtain a second fitted image;

[0012] Perform edge feature processing on the second fitted image to obtain a third fitted image.

[0013] In addition to one or more features described herein, further real-time solutions may include:

[0014] The normalization processing is specifically as follows:

[0015]

[0016] where data.min() represents the minimum value of the HU value of the image to be fitted; data.max() represents the maximum value of the HU value of the image to be fitted, and the value i represents the HU value in the i-th identification point; the Value i represents the HU value of the i-th image to be fitted after the normalization processing.

[0017] In addition to one or more features described herein, further real-time solutions may include:

[0018] The determination of the reference point of the first fitted image according to the identification point information includes:

[0019] The identification point information includes determining the field strength distribution map of the identification point according to the curve connected by the identification points;

[0020] According to the field strength distribution map, the identification points with corresponding field strength greater than the preset field strength are determined as the reference points of the first fitted image.

[0021] In addition to one or more features described herein, further real-time solutions may include:

[0022] The field strength formula corresponding to the field strength is:

[0023]

[0024] where F ik is the field strength received by any identification point k from the identification point field source i, P is the field strength of the identification point field source i, and d ik is the distance from the any identification point k to the identification point field source i; 1 ≤ i ≤ N; 1 ≤ k ≤ N; N is the total number of identification points;

[0025] According to the field strength formula, determine the field strength radiated from each identification point field source i to each identification point, that is, the field strength distribution map F i .

[0026] In addition to one or more features described herein, further real-time solutions may include:

[0027] The edge feature processing of the second fitting image includes:

[0028] Traverse the extracted straight line segments, and merge two straight lines with a distance less than a preset distance between the straight line segments to obtain the merged straight line segments;

[0029] The distance between the straight line segments includes the spatial distance between two straight lines and the straight line distance. The corresponding preset distances are the first spatial distance threshold and the second straight line distance threshold respectively;

[0030] The straight line distance is the maximum value of the distances from the two endpoints of one straight line segment to the two endpoints of the other straight line segment.

[0031] In addition to one or more features described herein, further real-time solutions may include:

[0032] The merging process includes: deleting the shorter straight line segment among the two straight lines; processing according to the positions of the two feet of the perpendiculars from the two endpoints of the shorter straight line segment to the longer straight line segment and the positions of the two endpoints of the longer straight line segment: if the two foot positions are on the longer straight line segment, then use the longer straight line segment as the new straight line segment; if the foot positions are outside the longer straight line segment, then use the feet outside the longer straight line segment as the endpoints of the new straight line segment.

[0033] In addition to one or more features described herein, further real-time solutions may include:

[0034] After obtaining the merged straight line segments, it further includes: traversing the merged straight line segments, and extending two straight line segments with a spatial distance less than the second spatial distance threshold and a slope less than a preset slope value between the straight line segments; the extension process includes: connecting the nearest endpoints of the two straight line segments.

[0035] The beneficial effects of the embodiments of the present invention compared with the prior art are:

[0036] The present invention uses a method of combining reference points, block sampling, and edges for multi-sample image fitting. On the basis of ensuring accurate identification of reference points, it can adopt a point-to-plane positioning and fitting method, resulting in high fitting accuracy and complete recognition results, making the method more scientific; at the same time, aiming at the problem of large edge ambiguity, by optimizing long straight lines, it can quickly extract and fit a complete edge structure, greatly reducing the data calculation amount and reducing the probability of problems such as incorrect classification caused by image distortion and defect, and the recognition result not meeting the subjective and objective consistency.

[0037] On the other hand, by adopting the multi-sample fitting image sharpening method of the present invention, it is possible to better BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0039] Figure 1 FIG. is a schematic flowchart of an implementation of a multi-sample fitting image sharpening method provided in Embodiment 1 of the present invention.

[0040] Figure 2 FIG. is a schematic diagram of two straight line segments ab and cd provided in Embodiment 1 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0041] In the following description, specific details such as specific system structures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present invention. However, those skilled in the art should clearly understand that the present invention can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present invention.

[0042] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0043] It should also be further understood that the term "and / or" used in the specification and claims of the present invention refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0044] In order to illustrate the technical solutions described in the present invention, the following will be described through specific embodiments.

[0045] Refer to Figure 1 , which is a multi-sample fitting image sharpening method provided in Embodiment 1 of the present invention. As Figure 1 shown, the processing method includes the following steps:

[0046] Preprocess the image to be fitted to obtain the first fitted image; wherein, the preprocessing includes processing different illumination intensities in the image to be fitted and normalizing the image parameters of the image to be fitted;

[0047] Among them, the preprocessing specifically includes setting gamma transformation, wherein the gamma transformation is:

[0048]

[0049]

[0050] Among them, γ represents the histogram offset in the target image. When γ < 1, the histogram shifts to the right, and when γ > 1, the histogram shifts to the left. In this paper, the mean value of the image on each channel is used to replace γ, so that after gamma transformation, the histograms of images with different illuminations tend to be consistent. mean() represents the bar average. I in is the input parameter of the image to be fitted.

[0051] Perform path processing on the first fitted image to obtain a feature map; wherein, the path processing is convolution processing by N convolutional layers, where N is a preset integer;

[0052] Among them, considering the running time, the path processing in this paper is set relatively simply and only includes traditional 3*3 convolutions. The path processing of the network includes 4 stages. The first stage includes 2 convolutional layers, and the stride of each convolutional layer is 2. The remaining stages all include 3 convolutional layers, and only the stride of the first convolutional layer is 2, and the remaining strides are 1. The feature maps generated in the 2nd, 3rd, and 4th stages. Therefore, the path processing has two functions: First, use path processing to obtain detailed information to prepare for the final multi-sample image fitting; second, provide high-quality feature maps for subsequent landmark determination.

[0053] According to the feature map, determine the corresponding landmark information in each first fitted image; and according to the landmark information, determine the reference point of the first fitted image;

[0054] Among them, in the feature map, by setting the landmark field strength threshold, determine the landmark with a large field strength, and based on this landmark, realize the determination of the reference point of the first fitted image. That is, to a certain extent, the landmark with a large field strength can be understood as the reference point of the first fitted image. The landmark information may include landmark position information and landmark field strength data information.

[0055] Based on the reference point, perform multi-density block sampling, and by setting a loss function, obtain the second fitted image;

[0056] Among them, for each reference point in the first fitted image, multi-density block sampling is performed centered on each reference point to predict and fit the image around the reference point. By setting this variable, a binary classifier can be trained to easily screen out the sampling blocks beyond a certain distance from the reference point and not consider their predictions for the reference point. The binary classifier is a simple task, equivalent to determining whether the center of the target reference point is within a circle with a certain diameter, so that extremely high prediction accuracy can be obtained quickly.

[0057] For the loss function, in this paper, the prediction loss is combined with the auxiliary loss to train the network of the block sampling model. The loss function is shown as follows:

[0058] loss = A·predict_loss + B·auxiliary_loss

[0059] Among them, loss represents the total loss, and A and B are the weights of each loss, which can be adjusted by oneself and can be selected in the present invention. A = B = 1.

[0060] Edge feature processing is performed on the second fitted image to obtain a third fitted image.

[0061] Among them, based on the second fitted image, edge features are extracted and fitted to realize edge processing of the fitted image. Specific edge optimizations include merging processing and extending processing (corresponding to step 1 and step 2 respectively). As other embodiments, only the straight line segments can be merged without extending. Of course, the specific merging means and extending means proposed in this embodiment may not be adopted either. For example, the specific merging means and extending means are both to fit two straight line segments into one straight line segment. Using the edge optimization method proposed in the present invention, long and continuous straight line features can be formed by fitting different straight line segments, and the edge structure of the fitted image can be extracted and expressed more completely, providing a true and reliable basis for subsequent processing of the image.

[0062] In addition to one or more features described in this paper, further real-time solutions may include:

[0063] The normalization processing is specifically:

[0064]

[0065] Among them, data.min() represents the minimum value of the HU value of the image to be fitted; data.max() represents the maximum value of the HU value of the image to be fitted, and the value i represents the HU value in the i-th identification point; the Value iIt represents the HU value of the i-th image to be fitted after the normalization process.

[0066] In addition to one or more features described herein, further real-time solutions may include:

[0067] Determining the fiducial point of the first fitted image according to the fiducial point information includes:

[0068] The fiducial point information includes determining the field strength distribution map of the fiducial point according to the curve connected by the fiducial points;

[0069] According to the field strength distribution map, the fiducial points corresponding to the field strength greater than the preset field strength are determined as the fiducial points of the first fitted image.

[0070] Among them, the field strength is proportional to the importance degree of the fiducial point and inversely proportional to the square of the distance between two fiducial points. Using the field strength formula, the influence size of each fiducial point in the image area under study can be calculated respectively, so as to determine the importance of the fiducial point. Combining the number and position of the fiducial points of the image to be fitted, the fiducial point of the first fitted image is determined, that is, the point positioning of the first fitted image is realized.

[0071] In addition to one or more features described herein, further real-time solutions may include:

[0072] The field strength formula corresponding to the field strength is:

[0073]

[0074] Among them, F ik is the field strength received by any fiducial point k from the fiducial point field source i, P is the field strength of the fiducial point field source i, and d ik is the distance from the any fiducial point k to the fiducial point field source i; 1 ≤ i ≤ N; 1 ≤ k ≤ N; N is the total number of the fiducial points;

[0075] According to the field strength formula, determining the field strength radiated from each fiducial point field source i to each fiducial point, that is, the field strength distribution map F i .

[0076] Among them, this field strength formula realizes the accurate calculation of the influence of the fiducial point field strength and ensures the basis and unity of the calculation.

[0077] In addition to one or more features described herein, further real-time solutions may include:

[0078] Performing edge feature processing on the second fitted image includes:

[0079] Traverse the extracted straight line segments, and merge two straight lines with a distance less than a preset distance between the straight line segments to obtain the merged straight line segments;

[0080] The distance between the straight line segments includes the spatial distance and the straight line distance between two straight lines, and the corresponding preset distances are the first spatial distance threshold and the second straight line distance threshold respectively;

[0081] The straight line distance is the maximum value of the distances from the two endpoints of one straight line segment to the two endpoints of the other straight line segment.

[0082] In addition to one or more features described herein, further real-time solutions may include:

[0083] The merging process includes: deleting the shorter straight line segment among the two straight lines; processing according to the positions of the two perpendicular feet of the two endpoints of the shorter straight line segment to the longer straight line segment and the positions of the two endpoints of the longer straight line segment: if the two perpendicular foot positions are on the longer straight line segment, then use the longer straight line segment as the new straight line segment; if the perpendicular foot positions are outside the longer straight line segment, then use the perpendicular feet outside the longer straight line segment as the endpoints of the new straight line segment.

[0084] S01: Merging process: Traverse the first set of straight line segments obtained after filtering by the combined filter operator, and determine whether there is a situation where the spatial distance between two straight line segments is less than the set first spatial distance threshold and the straight line distance between two straight line segments is less than the second straight line distance threshold. If this situation occurs, merge these two straight line segments to obtain the merged straight line segments, and continuously repeat this step S01 until such a situation does not exist in the first set of straight line segments, and finally form the second set of straight line segments.

[0085] Among them, the spatial distance between the two straight line segments defined here is: the maximum value of the distances from the two endpoints of the shorter straight line segment among the two straight line segments to the longer straight line segment. For example, as Figure 2 shown, it is a schematic diagram of two straight line segments ab and cd provided in the first embodiment of the present invention. The two straight line segments are ab and cd respectively, and ab is the longer straight line segment among the two straight line segments. Then the spatial distance between the two straight line segments is: the maximum value of the distance from endpoint c to the straight line segment ab and the distance from endpoint d to the straight line segment ab.

[0086] The straight line distance between the two straight line segments defined here is: the maximum value of the distances between the two endpoints of one straight line segment to the two endpoints of the other straight line segment. For example, as Figure 2 shown, the straight line distance between the straight line segments ab and cd is: the maximum value of ac, ad, bc, and bd. The merged straight line segment is a new line segment.

[0087] When determining that two straight line segments belong to the same category, the specific merging method is as follows: draw a perpendicular line from the shorter straight line segment to the longer straight line segment to obtain two foot of perpendiculars, and compare the positions of the two foot of perpendiculars and the positions of the two endpoints of the longer straight line segment: if the positions of both foot of perpendiculars are between the two endpoints of the longer straight line segment, then retain the original longer straight line segment as the new straight line segment and delete the original shorter straight line segment; if the position of the foot of perpendicular is outside the two endpoints of the longer straight line segment, then use the position of the foot of perpendicular outside the two endpoints of the longer straight line segment as the endpoints of the new straight line segment, thereby forming a new straight line segment, that is, the longer straight line segment is extended according to the positions of the two foot of perpendiculars and the positions of the two endpoints of the longer straight line segment. That is: use the point with the smallest X coordinate among these four points as the starting point of the new straight line segment, use the point with the largest X coordinate as the end point of the new straight line segment, and delete the shorter straight line segment.

[0088] In addition to one or more features described in this article, further real-time solutions may include:

[0089] After obtaining the merged straight line segment, it further includes: traversing the merged straight line segment, and extending two straight line segments whose spatial distance between the straight line segments is less than the second spatial distance threshold and whose slope of the straight line segment is less than the preset slope value; the extension process includes: connecting the nearest endpoints of the two straight line segments.

[0090] Among them, S02: Extension process: Traverse the second set of straight line segments, and judge whether there is a situation where the spatial distance between two straight line segments is less than the set second spatial distance threshold (which can be set to 80 pixels) and the slope difference between these two straight line segments is less than the set slope threshold (which can be set to 0.01). If this situation occurs, then merge these two straight line segments. Keep repeating this step until such a situation does not exist in the second set of straight line segments.

[0091] The present invention uses a method that combines reference points, block sampling, and edges for multi-sample image fitting. On the basis of ensuring accurate identification of reference points, it can achieve high fitting accuracy and complete recognition results through a point-to-plane positioning and fitting method, making this method more scientific; at the same time, aiming at the problem of large edge ambiguity, by using the optimization method of long straight lines, it can quickly extract and fit a complete edge structure, greatly reducing the amount of data calculation and reducing the probability of problems such as incorrect classification caused by image distortion and defect, and the recognition result not conforming to the subjective and objective consistency.

[0092] The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for processing the clarity of a multi-sample fitting image, characterized in that: Preprocess the image to be fitted to obtain a first fitted image; wherein, the preprocessing includes processing different illumination intensities in the image to be fitted and normalizing the image parameters of the image to be fitted; Perform path processing on the first fitted image to obtain a feature map; wherein, the path processing is convolution processing by N convolutional layers, where N is a preset integer; According to the feature map, determine the corresponding landmark point information in each of the first fitted images; and according to the landmark point information, determine the reference point of the first fitted image; Based on the reference point, perform multi-density block sampling, and obtain a second fitted image by setting a loss function; Perform edge feature processing on the second fitted image to obtain a third fitted image; The determining the reference point of the first fitted image according to the landmark point information includes: The landmark point information includes determining the field strength distribution map of the landmark point according to the curve connected by the landmark points; According to the field strength distribution map, determine the landmark points with corresponding field strengths greater than the preset field strength as the reference points of the first fitted image.

2. The method for processing the clarity of a multi-sample fitting image according to claim 1, characterized in that: The normalization processing is specifically: Among them, data.min() represents the minimum value of the HU values of the image to be fitted; data.max() represents the maximum value of the HU values of the image to be fitted, and value i represents the HU value in the i-th identification point; Value i represents the HU value of the i-th image to be fitted after normalization processing.

3. The method for processing the clarity of a multi-sample fitting image according to claim 2, characterized in that: The field strength formula corresponding to the field strength is: Among them, F ik is the field strength received by any identification point k from the identification point field source i, and P i is the field strength of the identification point field source i, and d ik is the distance from any identification point k to the identification point field source i; 1 ≤ i ≤ N; 1 ≤ k ≤ N; N is the total number of identification points; According to the field strength formula, determine the field strength radiated from the field source i of each identification point to each identification point, that is, the field strength distribution map F i .

4. The method for processing the clarity of a multi-sample fitting image according to claim 1, characterized in that: The edge feature processing on the second fitted image includes: Traverse the extracted straight line segments, and merge two straight lines with a distance less than a preset distance between the straight line segments to obtain a merged straight line segment; The distance between the straight line segments includes the spatial distance between two straight lines and the straight line distance, and the corresponding preset distances are the first spatial distance threshold and the second straight line distance threshold respectively; The straight line distance is the maximum value of the distances from the two endpoints of one straight line segment to the two endpoints of the other straight line segment.

5. The method for processing the clarity of a multi-sample fitting image according to claim 4, characterized in that: The merging process includes: deleting the shorter straight line segment among the two straight lines; and processing according to the positions of the two feet of the shorter straight line segment to the longer straight line segment and the positions of the two endpoints of the longer straight line segment: if the two foot positions are on the longer straight line segment, then use the longer straight line segment as the new straight line segment; if the foot positions are outside the longer straight line segment, then use the feet outside the longer straight line segment as the endpoints of the new straight line segment.

6. The method for processing the clarity of a multi-sample fitting image according to claim 4, wherein: The merged straight line segment is a new line segment.

Images