Image processing method, device and electronic equipment

Abstract

The present invention provides a method, device and electronic device for image processing, wherein the method includes: obtaining a first image and a second image to be spliced; generating an antigen based on a first feature point in the first image and generating an antibody based on a second feature point in the second image; cyclically executing multiple rounds of clonal selection operations for antigen variation until the iteration ends and the global optimal antibody is determined; the first feature point of the antigen corresponding to the global optimal antibody and the second feature point corresponding to the global optimal antibody are used as alignment point pairs for alignment. The image processing method, device and electronic device provided by the embodiments of the present invention use a clonal selection algorithm to determine the alignment point pairs without the need for secondary alignment, and have good adaptability and robustness; the antigen is mutated, and the local optimal antibody determined in each round is memorized through multiple rounds of clonal selection operations, which has better adaptability and can more accurately determine the alignment point pairs to prevent local optimality.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method, device, electronic device and computer-readable storage medium. Background Art

[0002] Images can provide rich, intuitive information, but due to the limitations of imaging equipment, captured images are often only partial. Image stitching technology can generate panoramic images, providing a comprehensive view of the target. For example, in CTL (cervical, lumbar, and thoracic) spine and whole-body MR (Magnetic Resonance) image acquisition, stitching is essential for better diagnosis and treatment.

[0003] Image stitching is a method of combining multiple image blocks to obtain a complete image. Image stitching technology usually has four steps: image preprocessing, image registration, modeling and coordinate transformation, fusion and reconstruction. Among them, the quality of the image registration process will directly affect the success and speed of image stitching. Stitching algorithms can be divided into the following categories according to the different image information used in the registration process: (1) transform domain-based methods, such as the extended phase correlation method proposed by De Castro and Morandi and the fast Fourier transform-based method proposed by Reddy and Chatterji, etc., (2) spatial domain-based methods, such as the feature-based Harris algorithm, Susan algorithm, SIFT algorithm, SURF algorithm, etc. Feature-based methods are more robust to scene rotation, faster, and can skillfully and automatically determine the overlapping relationship between a set of disordered images. After extracting feature points, feature-based methods need to further search and select relevant matching feature points, confirm the transformation relationship of the images to be stitched, and complete image stitching. Among them, determining the registration point pairs is one of its key parts.

[0004] At present, some schemes match feature point pairs by the distance between feature points in the overlapping area of ​​the two images to be stitched. This method has poor adaptability and robustness. In addition, secondary registration can also be used, that is, further registration is performed on the basis of coarse registration. However, when the noise is large and the coarse registration effect is poor, the secondary registration will also show increased deviation and poor stitching accuracy. Summary of the Invention

[0005] To solve existing technical problems, embodiments of the present invention provide an image processing method, apparatus, electronic device, and computer-readable storage medium.

[0006] In a first aspect, an embodiment of the present invention provides an image processing method, comprising:

[0007] Acquire a first image and a second image to be stitched, and extract first feature points in the first image and second feature points in the second image;

[0008] generating an antigen based on the first feature point, and generating an antibody based on the second feature point;

[0009] Repeat multiple rounds of clone selection until the iteration ends and the global optimal antibody is determined;

[0010] registering the first image and the second image by using a first feature point corresponding to the antigen matching the global optimal antibody and a second feature point corresponding to the global optimal antibody as a registration point pair;

[0011] The clone selection operation includes:

[0012] Determine the local optimal antibody that matches the current round antigen based on the clonal selection algorithm, and memorize the local optimal antibody; the global optimal antibody is the optimal antibody determined based on multiple local optimal antibodies;

[0013] When further iteration is required, the antigens of the current round are mutated to generate antigens for performing the next round of clonal selection operations.

[0014] In a possible implementation, determining the local optimal antibody that matches the current round antigen includes:

[0015] The affinity between the current round antigen and the current round antibody is determined, and the current round antibody with the highest affinity is used as the local optimal antibody matching the current round antigen.

[0016] In one possible implementation, determining the affinity between the current round antigen and the current round antibody includes:

[0017] Determining an affinity domain, wherein the affinity domain is used to represent the scope of action when determining the affinity between the antibody and the antigen;

[0018] Determine the characteristic difference between the current round antigen and the current round antibody, and determine the similarity between the current round antigen and the current round antibody within the affinity scope, and determine the affinity between the current round antigen and the current round antibody based on the characteristic difference and the similarity; the affinity is negatively correlated with the characteristic difference, and the affinity is positively correlated with the similarity.

[0019] In one possible implementation, determining the affinity scope includes:

[0020] An affinity scope of a corresponding shape is selected according to the splicing features of the first image and the second image, and the shapes of the affinity scope include: rectangle, cross, line, ellipse, and circle.

[0021] In one possible implementation, determining the characteristic difference between the current round antigen and the current round antibody includes:

[0022] Determine the characteristic value A(i) of the first characteristic point corresponding to the current round antigen i, determine the characteristic value B(j) of the second characteristic point corresponding to the current round antibody j, and use the absolute value of the difference between the characteristic value A(i) and the characteristic value B(j) as the characteristic difference between the current round antigen i and the current round antibody j;

[0023] Determining the similarity between the current round antigen and the current round antibody within the affinity domain comprises:

[0024] Determining pixel values ​​of a plurality of pixels within a first window in the first image, and determining pixel values ​​of a plurality of pixels within a second window in the second image; the first window is a window based on a first feature point corresponding to the current round antigen i, and the second window is a window based on a second feature point corresponding to the current round antibody j, and both the first window and the second window match the affinity scope; and

[0025] The similarity between the current round antigen i and the current round antibody j is determined according to the pixel values ​​of the multiple pixel points in the first window and the pixel values ​​of the multiple pixel points in the second window.

[0026] In one possible implementation, the affinity between the current round antigen and the current round antibody satisfies:

[0027]

[0028] Among them, f(i, j) represents the affinity between the current round antigen i and the current round antibody j, NCC(i, j) represents the similarity between the current round antigen i and the current round antibody j, b is an adjustment coefficient not less than 1, and a1 and a2 both represent weighting coefficients.

[0029] In one possible implementation, the similarity NCC(i, j) between the current round antigen i and the current round antibody j satisfies:

[0030]

[0031] Among them, W i (x k ,y k) indicates that the first window is located at position (x k ,y k ) pixel value at the pixel point, W j (x k ,y k ) indicates that the second window is located at position (x k ,y k ) pixel value at the pixel point, u i represents the mean value of the pixel values ​​of multiple pixels in the first window, u j Represents the average pixel value of multiple pixels in the second window.

[0032] In a possible implementation, the method further includes:

[0033] The registered first image and the registered second image are stitched together to form a fused image.

[0034] In a possible implementation, the number of the registration point pairs is at least three;

[0035] The first image and the second image after stitching and registration include:

[0036] Based on at least three of the registration point pairs, a clonal selection algorithm is used to perform an optimization solution to determine coordinate transformation parameters between the first image and the second image;

[0037] Coordinate transformation is performed on one of the first image and the second image based on the coordinate transformation parameters, and then the other of the first image and the second image is stitched together.

[0038] In a second aspect, an embodiment of the present invention further provides an image processing apparatus, comprising:

[0039] a preprocessing module, configured to obtain a first image and a second image to be stitched, and extract first feature points in the first image and second feature points in the second image;

[0040] a generating module, configured to generate an antigen based on the first feature point, and generate an antibody based on the second feature point;

[0041] The iteration module is used to cyclically execute multiple rounds of clone selection operations until the iteration ends and the global optimal antibody is determined;

[0042] a registration module, configured to register the first image and the second image by using a first feature point corresponding to the antigen matching the global optimal antibody and a second feature point corresponding to the global optimal antibody as a registration point pair;

[0043] Wherein, the iteration module includes:

[0044] a clonal selection unit, configured to determine a local optimal antibody that matches the current round of antigens based on a clonal selection algorithm, and memorize the local optimal antibody; the global optimal antibody is the optimal antibody determined based on multiple local optimal antibodies;

[0045] The antigen mutation unit is used to mutate the antigen of the current round when further iteration is required to generate an antigen for performing the next round of clonal selection operation.

[0046] In a third aspect, an embodiment of the present invention provides an electronic device comprising a bus, a transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the transceiver, the memory, and the processor are connected via the bus, and the computer program, when executed by the processor, implements the steps of any one of the above-described image processing methods.

[0047] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps in any one of the above-mentioned image processing methods.

[0048] The image processing method, apparatus, electronic device, and computer-readable storage medium provided by embodiments of the present invention convert feature points in the two images to be stitched into antigens and antibodies, respectively, and use a clonal selection algorithm to determine alignment point pairs, eliminating the need for secondary alignment. This method exhibits excellent adaptability and robustness. The method also mutates the antigens, memorizing the locally optimal antibodies determined in each round through multiple rounds of clonal selection. This method exhibits enhanced adaptability and, due to the presence of immune memory, enables better matching results and more accurate determination of alignment point pairs. Furthermore, it can, to a certain extent, prevent the search for the optimal antibody from being limited to a local optimum. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the background technology, the drawings required for use in the embodiments of the present invention or the background technology will be described below.

[0050] Figure 1 A flowchart of an image processing method provided by an embodiment of the present invention is shown;

[0051] Figure 2 A schematic diagram showing the shape of the affinity scope provided by an embodiment of the present invention is shown;

[0052] Figure 3A A schematic diagram of images to be stitched provided by an embodiment of the present invention is shown;

[0053] Figure 3B A schematic diagram of region division of images to be stitched provided by an embodiment of the present invention is shown;

[0054] Figure 4 Two images to be stitched provided by an embodiment of the present invention are shown;

[0055] Figure 5A shows an affinity trend graph of a rectangular scope provided by an embodiment of the present invention;

[0056] Figure 5B shows an affinity trend diagram of a cross-shaped scope provided by an embodiment of the present invention;

[0057] Figure 6 The fused image obtained by stitching two images using the traditional scheme is shown;

[0058] Figure 7 shows the content and image obtained by the image processing method provided by the embodiment of the present invention;

[0059] Figure 8 A schematic diagram showing the structure of an image processing apparatus provided by an embodiment of the present invention is shown;

[0060] Figure 9 A schematic structural diagram of an electronic device for executing an image processing method provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0061] Artificial immune systems, inspired by immunology, have high information processing capabilities and have become another research hotspot in intelligent computing, following neural networks and genetic algorithms. Artificial immune algorithms have been widely used in medical image processing.

[0062] The clonal selection algorithm describes the cloning, death, and memory processes of antibodies, some of the key factors in adaptive immunity, in response to antigenic stimulation. By determining affinity, it clones and mutates antibodies, enabling recognition of a wide range of antigens. This algorithm offers excellent adaptability and versatility, but traditional clonal selection algorithms also have limited adaptability and versatility. The image processing method provided in this embodiment of the present invention utilizes an improved clonal selection algorithm to achieve image registration, offering enhanced adaptability.

[0063] The embodiments of the present invention are described below with reference to the accompanying drawings.

[0064] Figure 1 FIG. 1 is a flow chart of an image processing method provided by an embodiment of the present invention. Figure 1 As shown, the method includes:

[0065] Step 101: Acquire a first image and a second image to be stitched, and extract first feature points in the first image and second feature points in the second image.

[0066] In an embodiment of the present invention, when multiple images need to be stitched together, two adjacent images can be selected as the first image and the second image, respectively. Furthermore, feature point extraction is performed on the images to be stitched together, and feature points are extracted from the images to be stitched together. For ease of distinction, this embodiment refers to feature points in the first image as first feature points, and feature points in the second image as second feature points. Generally, there are multiple first feature points and second feature points. Image registration is achieved by searching for matching registration points among the first and second feature points to form registration point pairs.

[0067] Among them, before extracting feature points, the first image and the second image can be preprocessed, for example, extracting the image foreground, filtering, and other preprocessing, and extracting feature points from the preprocessed first image and second image, so that the first feature points and the second feature points can be extracted better and more accurately.

[0068] Step 102: Generate an antigen based on the first feature point, and generate an antibody based on the second feature point.

[0069] In an embodiment of the present invention, registration points are determined based on a clonal selection algorithm; antigens are generated based on first feature points in a first image, and antibodies are generated based on second feature points in a second image. Specifically, the first feature point can be encoded to generate a corresponding antigen, and the second feature point can be encoded to generate a corresponding antibody; multiple antibodies can be generated to form an initial antibody population. In this embodiment, a corresponding antigen or antibody is generated based on at least one feature point; for example, a first feature point is encoded to generate an antigen, and multiple second feature points are encoded separately to generate multiple antibodies.

[0070] Step 103: Repeat multiple rounds of clone selection operations until the iteration ends and the global optimal antibody is determined.

[0071] Furthermore, the “clone selection operation” in step 103 includes:

[0072] Step 1031: Determine the local optimal antibody that matches the current round of antigens based on the clonal selection algorithm, and memorize the local optimal antibody; the global optimal antibody is the optimal antibody determined based on multiple local optimal antibodies.

[0073] Step 1032: If further iteration is required, the current round antigen is mutated to generate an antigen for performing the next round of clonal selection operation.

[0074] In an embodiment of the present invention, after determining the initial antigen and the initial antibody, a first round of clonal selection operation is performed, using the initial antigen as the current round antigen and the initial antibody as the current round antibody, and selecting the optimal antibody, i.e., the local optimal antibody, from multiple antibodies based on the clonal selection algorithm, and memorizing the local optimal antibody, for example, storing the local optimal antibody in a memory bank. Afterwards, the initial antigen is mutated to generate a new antigen, which is used to perform the next round of clonal selection operation, i.e., in the second round of clonal selection operation, the new antigen is used as the current round antigen, and step 1031 is continued to form an iteration. If further iteration is not required (for example, the number of iterations reaches a preset threshold), there is no need to mutate the antigen and the clonal selection operation is no longer performed.

[0075] In the process of determining the optimal antibody based on the clonal selection algorithm, this embodiment selects the optimal antibody based on the affinity between the antigen and the antibody. Optionally, the above step 1031 "determining the local optimal antibody that matches the current round of antigens" includes the following steps A1:

[0076] Step A1: Determine the affinity between the current round antigen and the current round antibody, and use the current round antibody with the highest affinity as the local optimal antibody matching the current round antigen.

[0077] In the embodiments of the present invention, the affinity used in the clonal selection algorithm is the affinity between the antigen and the antibody. Specifically, for each round of clonal selection, this affinity is the affinity between the antigen and the antibody in the current round. A higher affinity indicates a more likely optimal antibody. After the iterative process in the clonal selection algorithm completes, the optimal antibody for the current round, i.e., the local optimal antibody, is determined. Because the clonal selection algorithm clones and mutates antibodies, the antibodies in the current round in this embodiment can include both the initial antibodies determined at the beginning of the clonal selection process and new antibodies determined after cloning and mutation.

[0078] For example, during the current round of clonal selection, the above step 1031 of "determining the local optimal antibody matching the current round antigen based on the clonal selection algorithm" may specifically include the following steps:

[0079] Step B1: Determine the current round antigen and multiple initial antibodies, wherein the initial antibody may also be the current round antibody, and the multiple initial antibodies may form an initial antibody population.

[0080] Step B2: Determine the affinity between the current round antigen and each initial antibody, and determine the N best antibodies in the antibody population based on the affinity. Other antibodies with low affinity may be discarded.

[0081] Step B3: Clone (copy) the N best antibodies to generate a temporary clone population C;

[0082] Step B4: Apply mutation operation to the cloned population C to generate a new antibody population C'.

[0083] Step B5: Select the N best antibodies with the highest affinity to the current round antigen from the antibody population C', and discard antibodies with low affinity.

[0084] Step B6: Determine whether to stop iteration; if not, proceed to step B3; if not, proceed to step B7.

[0085] Step B7: The best antibody with the highest affinity is used as the local optimal antibody for the current round.

[0086] In step B6, whether the iterations need to be stopped can be determined by determining whether the number of iterations has reached the required number or whether the affinity has exceeded a preset threshold (i.e., whether the affinity is sufficiently large). Those skilled in the art will appreciate that the embodiments of the present invention involve two iterative processes, namely, the iterative process of mutating the antigen in step 1032 and the iterative process of cloning and selecting the antibody in step B6; wherein the iterative process in step B6 is the iterative process in the clonal selection algorithm.

[0087] Step 104: Using the first feature point corresponding to the antigen matching the global optimal antibody and the second feature point corresponding to the global optimal antibody as a registration point pair, the first image and the second image are registered.

[0088] In an embodiment of the present invention, a corresponding local optimal antibody can be determined based on each round of clonal selection operations. After performing multiple rounds of clonal selection operations, multiple local optimal antibodies can be determined, and a global optimal antibody can be selected from them; for example, the global optimal antibody is the local optimal antibody with the highest affinity. Further, an antigen that matches the global optimal antibody can be determined (for example, the affinity between the antigen and the global optimal antibody is the highest), and then the first feature point corresponding to the antigen can be determined, and the second feature point corresponding to the global optimal antibody can be determined. In this embodiment, the first feature point and the second feature point determined in this step are used as two alignment points to form a pair of alignment points. For example, the antigen and the global optimal antibody can be decoded to determine the corresponding first feature point and second feature point.

[0089] In general, multiple registration point pairs need to be determined to accurately determine the stitching orientation of the first and second images, achieve image registration, and facilitate subsequent stitching. If the global optimal antibody corresponds to a second feature point, multiple global optimal antibodies can be determined to determine multiple registration point pairs.

[0090] The image processing method provided by the embodiments of the present invention converts feature points in the two images to be stitched into antigens and antibodies, respectively, and uses a clonal selection algorithm to determine alignment point pairs, eliminating the need for secondary alignment. This method exhibits excellent adaptability and robustness. The method also mutates the antigens, memorizing the locally optimal antibodies determined in each round through multiple rounds of clonal selection. This method offers enhanced adaptability and, thanks to the immune memory, enables better matching results and more accurate determination of alignment point pairs. Furthermore, it can, to a certain extent, prevent the search for the optimal antibody from being limited to a local optimum.

[0091] Optionally, the above step A1 "determining the affinity between the current round antigen and the current round antibody" includes the following steps A11-A12:

[0092] Step A11: Determine the affinity domain. The affinity domain is used to indicate the scope of the affinity between the antibody and the antigen.

[0093] In the embodiments of the present invention, when calculating the affinity between an antigen and an antibody, the scope of the affinity, i.e., the affinity domain, is first determined. Affinity refers to the binding strength between an antigen-binding site on an antibody and the corresponding antigenic determinant. The relationship function between the antigen and the antibody, i.e., the affinity calculation criterion, can be represented by f. Accordingly, the affinity domain is the scope of the relationship function f for the antibody and antigen. For example, using Z to represent the scope, the affinity domain can be expressed as:

[0094] Z=(P Z , Z H ) (1)

[0095] Among them, P Z The form of affinity scope includes both spatial and temporal characteristics, which is used to represent basic features; Z H Indicates the regional features corresponding to the affinity scope, such as the edge coordinates of the affinity scope.

[0096] In the embodiment of the present invention, the interaction range can be the contact area between the antibody and the antigen, or the interaction template between the antibody and the antigen. In the process of calculating affinity, considering the inherent characteristics of the affinity scope, the relationship function can be defined based on its size, shape, etc. Specifically, the basic characteristics of the affinity function scope P ZThe affinity scope may have various shapes (e.g., scope size, shape, etc.). The size of the affinity scope of the same shape determines its scope of application. The larger the affinity scope, the wider its scope of application, the more information it acquires, and the more computational effort it also requires. The shape of the affinity scope determines the area of ​​information that the affinity scope focuses on. Optionally, the above step A11 "determining the affinity scope" may specifically include:

[0097] Step A111: selecting an affinity scope of a corresponding shape according to the splicing features of the first image and the second image. The shapes of the affinity scope include: rectangle, cross, line, ellipse, and circle.

[0098] The embodiment of the present invention provides affinity scopes of various shapes, which may specifically be rectangular, cross, linear, elliptical, or circular. Figure 2 The following diagrams illustrate affinity scopes in rectangular, cross, linear, and elliptical shapes. For example, if the affinity scope is a line passing through a point, the affinity scope only focuses on the information on that line and is indifferent to information on other lines passing through the point. For another example, if the affinity scope is a circle centered at the point, the affinity scope is equally interested in all points on the lines passing through the point. Figure 2 The characteristics of the four affinity scopes shown are as follows:

[0099] (1) Rectangular scope: Rectangular scope refers to the affinity scope with a rectangular shape. The height and width of the rectangular scope can be set to different values; Figure 2 As shown in (a), the half-height h and half-width w of the rectangular scope can be different. While rectangular scopes can capture a large amount of information, they can also easily lead to redundancy. When different regions of the image within the rectangular scope contain many similar points, antibody diversity can be hindered, causing the clone selection algorithm to mature prematurely.

[0100] (2) Cross scope: A cross scope refers to an affinity scope that is cross-shaped. It places more emphasis on the information of feature points in two directions (for example, horizontal and vertical directions). Compared with a rectangular scope, a cross scope ignores the information of the four corners of the rectangle. Under the same conditions, a cross scope saves more time than a rectangular scope. Figure 2 The cross-shaped scope template shown in (b) has a half-height and half-width of L2 and L1, respectively. The half-height and half-width of the protruding part are W1 and W2, respectively. When L1 = W2 and L2 = W1, the cross-shaped scope becomes a rectangular scope. In practice, the cross-shaped scope should satisfy W1 < L2 and W2 < L1.

[0101] (3) Linear scope: Linear scope refers to the affinity scope with a linear shape. Under the same conditions, the linear scope obtains the least amount of information compared with the other three scopes, but it can obtain information on any angle straight line with the feature point as the midpoint, which is the most flexible. Figure 2 As shown in (c), the half-length of the linear scope is L and the angle is α.

[0102] (4) Elliptical scope: Elliptical scope refers to the affinity scope with an elliptical shape. Its information content and redundancy are between the cross scope and the rectangular scope. Since the calculation process uses pixels as the unit, Figure 2 The elliptical scope shown in (d) should satisfy a, b ≥ 4, where a and b represent the major axis and minor axis of the elliptical scope respectively. When a = b, the elliptical scope is a circular scope.

[0103] For different images to be stitched, the features that can reflect their characteristics may be different; for example, assuming that T = T (D, E) is the image to be stitched (for example, the first image and the second image), D represents the neighborhood that can properly reflect the characteristics of the feature points, that is, the area D can represent the stitching feature of the image to be stitched, and the stitching feature can be the feature of the overlapping part of the two images to be stitched; E is the other image area except area D. Among them, the area D can be expressed as D = (P D , D H ), similar to the above formula (1), P D 、D H Respectively represent the basic characteristics and regional characteristics of the region D; in order to achieve the best effect, it is expected to meet the following requirements:

[0104]

[0105] Therefore, by selecting an affinity scope that matches the splicing characteristics, the above formula (2) can be better satisfied.

[0106] The shape of the affinity function scope is different, and the amount of information obtained varies, which affects the computational efficiency of the algorithm. In addition, the shape of the affinity function scope also affects the diversity of antibodies in the clone selection algorithm. Figure 3A shows a first image and a second image, Figure 3BA schematic diagram of the region division of the first and second images is shown, dividing the first and second images to be stitched into nine regions. As can be seen from the figure, regions 1, 3, 6, 7, and 9 in the first image are identical to the corresponding regions in the second image. If a rectangular scope is used to enclose all nine regions, the presence of regions 1, 3, 6, 7, and 9 results in a higher affinity when matching (a) and (b). In other words, the identical nature of regions 1, 3, 6, 7, and 9 increases the overall affinity between the first and second images, thereby reducing the impact of the differences between the first and second images on the affinity. This will hinder the variation of the antibody represented by the second image due to its higher affinity, thereby hindering antibody diversity and making the clonal selection algorithm more likely to fall into a local optimum.

[0107] In this embodiment of the present invention, if the affinity scope is applied only to regions 2, 4, 5, 6, and 8, the first image and the second image can be quickly distinguished. Therefore, a cross-shaped affinity scope can be selected, that is, the affinity between the first image and the second image can be calculated based on regions 2, 4, 5, 6, and 8. This improves computational efficiency and can increase antibody diversity.

[0108] Step A12: Determine the characteristic difference between the current round antigen and the current round antibody, and determine the similarity between the current round antigen and the current round antibody within the affinity scope, and determine the affinity between the current round antigen and the current round antibody based on the characteristic difference and the similarity; there is a negative correlation between affinity and characteristic difference, and a positive correlation between affinity and similarity.

[0109] In the embodiments of the present invention, the difference between the antigen and antibody, i.e., the characteristic difference, is determined based on their own characteristics. The similarity between the antigen and antibody within the affinity domain is then determined. The affinity between the antigen and antibody is then determined based on the characteristic difference and the similarity. A greater characteristic difference indicates a greater difference between the antigen and antibody, a lower affinity, and a negative correlation between the two. Correspondingly, a greater similarity indicates a greater similarity between the antigen and antibody within the affinity domain, a higher affinity, and a positive correlation between the two.

[0110] In the process of antibody recognition and response to antigens, the affinity between antibodies and antigens plays a crucial role. The embodiments of the present invention comprehensively determine the affinity between the two by combining the characteristic differences between the antibodies and antigens and the similarities within the affinity domain. This can maintain good antibody diversity, ensure good recognition and response to various antigens, and enhance robustness.

[0111] Optionally, the process of "determining the characteristic difference between the current round antigen and the current round antibody" in the above step A12 specifically includes the following steps A121:

[0112] Step A121: Determine the eigenvalue A(i) of the first feature point corresponding to the current round antigen i, determine the eigenvalue B(j) of the second feature point corresponding to the current round antibody j, and use the absolute value of the difference between the eigenvalue A(i) and the eigenvalue B(j) as the feature difference between the current round antigen i and the current round antibody j.

[0113] In an embodiment of the present invention, during the current round of clonal selection, it is necessary to determine the affinity between the current round antigen i and multiple current round antibodies respectively. For the current round antibody j, the characteristic difference between the two (i.e., the current round antigen i and the current round antibody j) is determined by the characteristic values ​​of the corresponding characteristic points. Specifically, if the current round antigen i corresponds to the first characteristic point A in the first image, and the characteristic value of the first characteristic point A is A(i), and the current round antibody j corresponds to the second characteristic point B in the second image, and the characteristic value of the second characteristic point B is B(j), then the absolute value of the difference between the two characteristic values ​​is used as the characteristic difference between the current round antigen i and the current round antibody j, that is, the characteristic difference can be expressed as |A(i)-B(j)|.

[0114] Furthermore, in this embodiment, the similarity between the antibody and the antigen is determined using the pixel values ​​(e.g., grayscale values) of multiple pixels within the affinity scope. Specifically, the process of "determining the similarity between the current round antigen and the current round antibody within the affinity scope" in step A12 above may include the following steps A122-A123:

[0115] Step A122: Determine the pixel values ​​of multiple pixels within a first window in the first image, and determine the pixel values ​​of multiple pixels within a second window in the second image; the first window is a window based on a first feature point corresponding to the current round antigen i, and the second window is a window based on a second feature point corresponding to the current round antibody j, and both the first window and the second window match the affinity scope.

[0116] In an embodiment of the present invention, a first window and a second window are determined that match the selected affinity scope; for example, the first window and the second window have the same size and shape as the affinity scope. Furthermore, if the current round antigen i corresponds to the first feature point A, then the first feature point A is located within the first window, and the first window is a window based on the first feature point A, for example, the first feature point A is located at the center of the first window. Similarly, if the current round antibody j corresponds to the second feature point B, then the second feature point B is located within the second window, and the second window is a window based on the second feature point B, for example, the second feature point B is located at the center of the second window.

[0117] The first window is applied to the first image to determine a portion of the first image, and the second window is applied to the second image to determine a portion of the second image. Specifically, multiple pixels within the first window in the first image can be determined, and their pixel values ​​can be determined. Similarly, multiple pixels within the second window in the second image can be determined, and their pixel values ​​can be determined.

[0118] Step A123: Determine the similarity between the current round antigen i and the current round antibody j according to the pixel values ​​of the multiple pixel points in the first window and the pixel values ​​of the multiple pixel points in the second window.

[0119] In the embodiment of the present invention, the similarity between the current round antigen i and the current round antibody j is determined by combining the pixel values ​​of multiple pixels in the first window and the pixel values ​​of multiple pixels in the second window. Specifically, since the first window and the second window have the same size and shape, the positions in the two windows can be represented by the same coordinates; for example, for the position (x k ,y k ), the pixel value of the pixel in the first window is W i (x k ,y k ), the pixel value of the pixel in the second window is W j (x k ,y k ), different (x k ,y k ) represents different positions in the window. This embodiment is based on the pixel values ​​W at multiple positions. i (x k ,y k ), W j (x k ,y k ) to determine the similarity between the current round antigen i and the current round antibody j.

[0120] Optionally, NCC(i, j) represents the similarity between the current round antigen i and the current round antibody j, and the similarity NCC(i, j) satisfies the following formula:

[0121]

[0122] Among them, W i (x k ,y k ) indicates that the first window is located at position (x k ,y k ) pixel value at the pixel point, W j (x k ,y k ) indicates that the second window is located at position (xk ,y k ) pixel value at the pixel point, u i Represents the mean value of the pixel values ​​of multiple pixels in the first window, u j Represents the mean of the pixel values ​​of multiple pixels in the second window.

[0123] In the embodiment of the present invention, the mean value u of the pixel values ​​of multiple pixels (eg, all pixels) in the first window can be determined. i , and the mean value u of the pixel values ​​of multiple pixels (for example, all pixels) in the second window j By measuring the degree to which pixel values ​​deviate from their mean and standardizing the degree of deviation using the standard deviation, the similarity between the pixels in two windows can be described more accurately.

[0124] Optionally, the affinity between the current round antigen and the current round antibody satisfies:

[0125]

[0126] Among them, f(i, j) represents the affinity between the current round antigen i and the current round antibody j, NCC(i, j) represents the similarity between the current round antigen i and the current round antibody j, b is an adjustment coefficient not less than 1, and a1 and a2 both represent weighting coefficients.

[0127] In an embodiment of the present invention, the feature difference |A(i)-R(j)| and the similarity NCC(i, j) are processed in a weighted manner to obtain the affinity f(i, j) between the current round antigen i and the current round antibody j. a1 and a2 represent the weighting coefficients of the feature difference and similarity, respectively. When a1 is large, the influence of the feature values ​​of the feature points in the two images to be stitched on the affinity becomes greater; when a2 is large, the influence of the pixel value relationship between the windows centered on the feature points on the affinity becomes greater. Generally, a1, a2∈[0, 1] and a1+a2=1. The adjustment coefficient b is a preset coefficient that is not less than 1, for example, b=2, or b=10, etc.

[0128] Based on any of the above embodiments, after step 104, the method may further include step C1:

[0129] Step C1: stitching the registered first image and the second image to form a fused image.

[0130] In an embodiment of the present invention, after registering the first image and the second image, the stitching orientation of the two images can be known, and the geometric coordinates of the stitching, such as the coordinates of the registration point, can be determined; then, the coordinate transformation relationship between the two can be determined, and by performing a coordinate transformation on one of the images, image stitching and fusion can be achieved to form a fused image determined after stitching the first image and the second image.

[0131] Optionally, in order to accurately determine the coordinate transformation relationship between the first image and the second image, the number of registration point pairs determined in step 104 is at least three. In addition, step C1 "joining the first image and the second image after registration" includes steps C11-C12:

[0132] Step C11: Based on at least three registration point pairs, a clonal selection algorithm is used to perform an optimization solution to determine coordinate transformation parameters between the first image and the second image.

[0133] Step C12: performing coordinate transformation on one of the first image and the second image based on the coordinate transformation parameters, and then stitching the other of the first image and the second image.

[0134] There are many types of image coordinate transformation relationships, such as translation, rotation, scaling, etc. Usually, these transformation relationships can be implemented through a rotation matrix. For example, the application of the rotation matrix can be seen in the following formula:

[0135]

[0136] Among them, (x, y) represents the original coordinates before transformation, (x′, y′) represents the coordinates after transformation; a, b, c, d, T x 、T y Represents the six unknown parameters in the rotation matrix, which require at least three registration point pairs to solve. When the number of registration point pairs is greater than 3, the parameters can be solved through optimization.

[0137] Commonly used parameter solving methods include least squares and RANSAC (Random Sample Consensus). The least squares algorithm is sensitive to errors and is susceptible to bad pixel images. RANSAC can only estimate one model from a specific data set. If there are two (or more) models, RANSAC cannot find another model. When the image feature information is unclear and the number of bad pixels is large or even exceeds 50%, the RANSAC calculation error increases. If the SURF operator is used to calculate the feature points of the image to be registered, and a rough match is performed based on the feature information, due to the similar feature information, the registration point pairs will deviate. At this time, when the rotation matrix is ​​calculated by RANSAC, the calculation result will have errors, making it difficult to achieve a seamless result.

[0138] In the embodiment of the present invention, when solving the parameters of the above formula (5), it can be converted into:

[0139] min(z x )=min∑(ax i +by i +T x -x′ i ) 2 (6)

[0140] min(z y )=min∑(cx i +d yi +T y -y′ i ) 2 (7)

[0141] That is, the parameter solution in equation (5) is converted into two single-objective optimization problems, solving the objective function z x 、z y Minimum value. In the embodiment of the present invention, the clonal selection algorithm is used for optimization and solution, which can more accurately determine the coordinate transformation parameters between the first image and the second image (for example, unknown parameters in the rotation matrix). Coordinate transformation based on the coordinate transformation parameters can achieve seamless splicing and effectively avoid errors.

[0142] Furthermore, after stitching the images, they must be fused to generate a fused image. Because the two images were captured at different times, the signal strength of their pixels may differ, resulting in a difference in brightness (grayscale) in the overlapping area. Furthermore, considering that the image may be deformed to a certain extent due to issues like magnetic field inhomogeneity, corrections must also be made to the deformed boundary areas. This grayscale difference and deformation correction are handled through image fusion.

[0143] The image processing method provided by the embodiment of the present invention is described in detail below through an embodiment.

[0144] In the embodiment of the present invention, the two images to be spliced ​​can be seen in Figure 4 As shown, the white boxes in the figure represent the similar areas in the two images.

[0145] right Figure 4 The two images shown are stitched together, and it can be seen that Figure 4 There are cases where parts of the two images are exactly the same, but there are obvious differences in the four directions of the upper, lower, left, and right of the large area where the same local areas are located. Therefore, the affinity scopes of rectangle, cross, and ellipse can be used. Figure 2 The relevant parameters of the four domains of rectangle, cross, line and ellipse shown are: w=L1=a=25 (unit is pixel), h=L2=L=b=30, W1=W2=5, α=90°.

[0146] The number of pixels contained in the four scopes can be seen in Table 1:

[0147] Table 1

[0148] rectangle Cross Linear oval Number of pixels 3111 1111 61 2347

[0149] At this time, the linear scope, cross scope and elliptical scope are contained in the rectangular scope. Since the image to be spliced ​​( Figure 4 ) focuses more on the horizontal and vertical information of feature points, so the impact of the points discarded by the cross scope and elliptical scope on the search for registration points is within a tolerable range. From Table 1, it can be seen that the linear scope contains very few pixels, but the amount of information contained is low; the rectangular scope and elliptical scope have more pixels, but the amount of calculation is large; the cross template contains far fewer feature points than the rectangular template and contains more sufficient information. When performing affinity calculation based on the above formula (4), the use of the cross scope can not only better complete the matching search, but also improve the calculation efficiency.

[0150] Moreover, although the rectangular scope can obtain relatively complete information, the information obtained may contain some unnecessary factors or even hinder clonal mutation, while the cross scope can weaken this problem. Figure 4 The two images shown are matched using rectangular and cross scopes respectively. The affinity trend graphs obtained are shown in Figure 5A 、 5B shown; among them, Figure 5A is the affinity trend diagram of the rectangular scope, Figure 5B This is the affinity trend graph of the cross scope.

[0151] Depend on Figure 5A 、 5B As can be seen, the rectangular scope exhibited superior affinity to the cross scope in the early stages of clonal selection, quickly reaching equilibrium and preventing antibody mutation, resulting in the optimal antibody. Although the cross scope exhibited lower affinity than the rectangular scope in the early stages of clonal selection, its antibodies exhibited excellent diversity and, through antibody mutation, yielded superior antibodies, ultimately exceeding the affinity of the rectangular scope. For images with textures that differ significantly in the horizontal and vertical directions but are essentially identical in other regions, the cross scope maintains greater antibody diversity than the rectangular scope, allowing for better identification of optimal antibodies and, consequently, more accurate determination of registration point pairs.

[0152] When determining the coordinate transformation relationship, due to the similarity of feature information, the registration point pairs are prone to deviation. When calculating the rotation matrix through RANSAC, the error is large. The schematic diagram after splicing can be seen in Figure 6 As shown. Figure 6 As shown in Figure 2, when the alignment point pairs have large deviations, the rotation matrix splicing result calculated by RANSAC has obvious deviations. In the embodiment of the present invention, the coordinate transformation parameters are determined by the clone selection algorithm. Figure 4 The result after stitching and fusion of the images to be stitched can be found in Figure 7 As shown, Figure 7 The fused image of the two images is shown, with good stitching effect and no obvious stitching marks.

[0153] The above describes in detail the image processing method provided by the embodiment of the present invention. The method can also be implemented by a corresponding device. The following describes in detail the image processing device provided by the embodiment of the present invention.

[0154] Figure 7 FIG. 1 shows a schematic diagram of the structure of an image processing device provided by an embodiment of the present invention. Figure 7 As shown, the image processing device includes:

[0155] A preprocessing module 71 is configured to obtain a first image and a second image to be stitched, and extract first feature points in the first image and second feature points in the second image;

[0156] a generating module 72, configured to generate an antigen based on the first feature point, and generate an antibody based on the second feature point;

[0157] Iteration module 73, configured to cyclically execute multiple rounds of clone selection operations until the iteration ends and determine the global optimal antibody;

[0158] a registration module 74 for registering the first image and the second image by using a first feature point corresponding to the antigen matching the global optimal antibody and a second feature point corresponding to the global optimal antibody as a registration point pair;

[0159] The iteration module 73 includes:

[0160] The clonal selection unit 731 is used to determine the local optimal antibody that matches the current round of antigens based on the clonal selection algorithm and memorize the local optimal antibody; the global optimal antibody is the optimal antibody determined based on multiple local optimal antibodies;

[0161] The antigen mutation unit 732 is used to mutate the antigen of the current round when further iteration is required to generate an antigen for performing the next round of clonal selection operation.

[0162] In a possible implementation, the clone selection unit 731 determines the local optimal antibody that matches the current round antigen, including:

[0163] The affinity between the current round antigen and the current round antibody is determined, and the current round antibody with the highest affinity is used as the local optimal antibody matching the current round antigen.

[0164] In a possible implementation, the clone selection unit 731 includes:

[0165] A first determination subunit is used to determine an affinity domain, where the affinity domain represents a range of action when determining the affinity between an antibody and an antigen;

[0166] The second determination subunit is used to determine the characteristic difference between the current round antigen and the current round antibody, and determine the similarity between the current round antigen and the current round antibody within the affinity scope, and determine the affinity between the current round antigen and the current round antibody based on the characteristic difference and the similarity; the affinity is negatively correlated with the characteristic difference, and the affinity is positively correlated with the similarity.

[0167] In a possible implementation, the first determining subunit determines the affinity scope, including:

[0168] An affinity scope of a corresponding shape is selected according to the splicing features of the first image and the second image, and the shapes of the affinity scope include: rectangle, cross, line, ellipse, and circle.

[0169] In a possible implementation, the second determining subunit determines the characteristic difference between the current round antigen and the current round antibody, including:

[0170] Determine the characteristic value A(i) of the first characteristic point corresponding to the current round antigen i, determine the characteristic value B(j) of the second characteristic point corresponding to the current round antibody j, and use the absolute value of the difference between the characteristic value A(i) and the characteristic value B(j) as the characteristic difference between the current round antigen i and the current round antibody j;

[0171] Furthermore, the second determining subunit determines the similarity between the current round antigen and the current round antibody within the affinity scope, including:

[0172] Determining pixel values ​​of a plurality of pixels within a first window in the first image, and determining pixel values ​​of a plurality of pixels within a second window in the second image; the first window is a window based on a first feature point corresponding to the current round antigen i, and the second window is a window based on a second feature point corresponding to the current round antibody j, and both the first window and the second window match the affinity scope; and

[0173] The similarity between the current round antigen i and the current round antibody j is determined according to the pixel values ​​of the multiple pixel points in the first window and the pixel values ​​of the multiple pixel points in the second window.

[0174] In one possible implementation, the affinity between the current round antigen and the current round antibody satisfies:

[0175]

[0176] Among them, f(i, j) represents the affinity between the current round antigen i and the current round antibody j, NCC(i, j) represents the similarity between the current round antigen i and the current round antibody j, b is an adjustment coefficient not less than 1, and a1 and a2 both represent weighting coefficients.

[0177] In one possible implementation, the similarity NCC(i, j) between the current round antigen i and the current round antibody j satisfies:

[0178]

[0179] Among them, W i (x k ,y k ) indicates that the first window is located at position (x k ,y k ) pixel value at the pixel point, W j (x k ,y k ) indicates that the second window is located at position (x k ,y k ) pixel value at the pixel point, u i represents the mean value of the pixel values ​​of multiple pixels in the first window, u j Represents the average pixel value of multiple pixels in the second window.

[0180] In a possible implementation, the device further includes:

[0181] A fusion module is used to stitch the first image and the second image after registration to form a fused image.

[0182] In a possible implementation, the number of the registration point pairs is at least three;

[0183] The fusion module stitches the registered first image and the second image, including:

[0184] Based on at least three of the registration point pairs, a clonal selection algorithm is used to perform an optimization solution to determine coordinate transformation parameters between the first image and the second image;

[0185] Coordinate transformation is performed on one of the first image and the second image based on the coordinate transformation parameters, and then the other of the first image and the second image is stitched together.

[0186] In addition, an embodiment of the present invention also provides an electronic device, including a bus, a transceiver, a memory, a processor, and a computer program stored in the memory and runable on the processor. The transceiver, the memory, and the processor are respectively connected via a bus. When the computer program is executed by the processor, each process of the above-mentioned image processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0187] For details, see Figure 8 As shown, an embodiment of the present invention further provides an electronic device, which includes a bus 1110 , a processor 1120 , a transceiver 1130 , a bus interface 1140 , a memory 1150 and a user interface 1160 .

[0188] In an embodiment of the present invention, the electronic device further includes: a computer program stored in the memory 1150 and executable on the processor 1120 , and when the computer program is executed by the processor 1120 , each process of the above-mentioned image processing method embodiment is implemented.

[0189] The transceiver 1130 is configured to receive and send data under the control of the processor 1120 .

[0190] In an embodiment of the present invention, a bus architecture (represented by bus 1110) may include any number of interconnected buses and bridges, and bus 1110 connects various circuits including one or more processors represented by processor 1120 and a memory represented by memory 1150.

[0191] Bus 1110 represents one or more of any of several types of bus structures, including a memory bus and memory controller, a peripheral bus, an Accelerated Graphical Port (AGP), a processor, or a local bus using any of a variety of bus architectures. By way of example and not limitation, such architectures include an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA), and a Peripheral Component Interconnect (PCI) bus.

[0192] The processor 1120 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiment can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above-mentioned processor includes: a general-purpose processor, a central processing unit (CPU), a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a complex programmable logic device (CPLD), a programmable logic array (PLA), a microcontroller unit (MCU) or other programmable logic devices, discrete gates, transistor logic devices, discrete hardware components. The various methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. For example, the processor can be a single-core processor or a multi-core processor, and the processor can be integrated into a single chip or located on multiple different chips.

[0193] The processor 1120 can be a microprocessor or any conventional processor. The method steps disclosed in conjunction with the embodiments of the present invention can be directly executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a readable storage medium known in the art, such as a random access memory (RAM), a flash memory (Flash Memory), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), or a register. The readable storage medium is located in a memory, and the processor reads the information in the memory and performs the steps of the above method in conjunction with its hardware.

[0194] The bus 1110 may also connect various other circuits, such as peripheral devices, voltage regulators, or power management circuits. The bus interface 1140 provides an interface between the bus 1110 and the transceiver 1130. These are all well known in the art and are therefore not further described in this embodiment of the present invention.

[0195] The transceiver 1130 can be a single component or multiple components, such as multiple receivers and transmitters, providing a means for communicating with various other devices over a transmission medium. For example, the transceiver 1130 receives external data from other devices and transmits data processed by the processor 1120 to other devices. Depending on the nature of the computer system, a user interface 1160 may also be provided, such as a touch screen, physical keyboard, display, mouse, speaker, microphone, trackball, joystick, or stylus.

[0196] It should be understood that in an embodiment of the present invention, the memory 1150 may further include a memory remotely located relative to the processor 1120, and these remotely located memories may be connected to a server via a network. One or more parts of the aforementioned network may be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless local area network (WLAN), a wide area network (WAN), a wireless wide area network (WWAN), a metropolitan area network (MAN), the Internet, a public switched telephone network (PSTN), a plain old telephone service network (POTS), a cellular telephone network, a wireless network, a wireless fidelity (Wi-Fi) network, or a combination of two or more of the aforementioned networks. For example, the cellular telephone network and the wireless network can be a Global System for Mobile Communications (GSM) system, a Code Division Multiple Access (CDMA) system, a Worldwide Interoperability for Microwave Access (WiMAX) system, a General Packet Radio Service (GPRS) system, a Wideband Code Division Multiple Access (WCDMA) system, a Long Term Evolution (LTE) system, an LTE Frequency Division Duplex (FDD) system, an LTE Time Division Duplex (TDD) system, an Advanced Long Term Evolution (LTE-A) system, a Universal Mobile Telecommunications (UMTS) system, an Enhanced Mobile Broadband (eMBB) system, a Massive Machine Type of Communication (mMTC) system, an Ultra Reliable Low Latency Communications (uRLLC) system, and the like.

[0197] It should be understood that the memory 1150 in the embodiment of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Non-volatile memories include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory.

[0198] Volatile memory includes random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The memory 1150 of the electronic device described in the embodiments of the present invention includes, but is not limited to, the above and any other suitable types of memory.

[0199] In the embodiment of the present invention, the memory 1150 stores the following elements of the operating system 1151 and the application 1152: executable modules, data structures, or subsets thereof, or extended sets thereof.

[0200] Specifically, the operating system 1151 includes various system programs, such as a framework layer, a core library layer, and a driver layer, which are used to implement various basic services and process hardware-based tasks. The application 1152 includes various application programs, such as a media player and a browser, which are used to implement various application services. The program that implements the method of the embodiment of the present invention may be included in the application 1152. The application 1152 includes applets, objects, components, logic, data structures, and other computer system executable instructions that perform specific tasks or implement specific abstract data types.

[0201] In addition, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned image processing method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0202] Computer-readable storage media include: permanent and non-permanent, removable and non-removable media, which are tangible devices that can retain and store instructions for use by instruction execution devices. Computer-readable storage media include: electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, and any suitable combination of the above. Computer-readable storage media include: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette storage, magnetic disk storage or other magnetic storage devices, memory sticks, mechanical encoding devices (such as punched cards or raised structures with grooves in which instructions are recorded), or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined in the embodiments of the present invention, computer-readable storage media does not include temporary signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (such as light pulses passing through fiber optic cables), or electrical signals transmitted through wires.

[0203] In the several embodiments provided in this application, it should be understood that the disclosed devices, electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or can be an electrical, mechanical or other form of connection.

[0204] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, and may be located in a single location or distributed across multiple network units. Some or all of these units may be selected based on actual needs to address the issues addressed by the embodiments of the present invention.

[0205] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0206] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (including: a personal computer, a server, a data center or other network device) to perform all or part of the steps of the method described in each embodiment of the present invention. The above-mentioned storage medium includes the various media that can store program codes as listed above.

[0207] In describing the embodiments of the present invention, those skilled in the art should understand that the embodiments of the present invention can be implemented as methods, apparatuses, electronic devices, and computer-readable storage media. Therefore, the embodiments of the present invention can be specifically implemented in the following forms: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software. In addition, in some embodiments, the embodiments of the present invention can also be implemented in the form of a computer program product in one or more computer-readable storage media, wherein the computer-readable storage medium contains computer program code.

[0208] The above-mentioned computer-readable storage medium may adopt any combination of one or more computer-readable storage media. Computer-readable storage media include: electrical, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or components, or any combination thereof. More specific examples of computer-readable storage media include: portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM), flash memories (Flash Memory), optical fibers, compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices or any combination thereof. In an embodiment of the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or component.

[0209] The computer program code contained in the computer-readable storage medium may be transmitted using any appropriate medium, including wireless, wire, optical cable, radio frequency (RF), or any suitable combination thereof.

[0210] The computer program code for performing the operations of the embodiments of the present invention can be written in assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, or in one or more programming languages ​​or a combination thereof, wherein the programming language includes an object-oriented programming language, such as Java, Smalltalk, C++, and also includes a conventional procedural programming language, such as C language or a similar programming language. The computer program code can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, and entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer or to an external computer via any type of network, including a local area network (LAN) or a wide area network (WAN).

[0211] The embodiments of the present invention describe the provided methods, devices, and electronic devices through flowcharts and / or block diagrams.

[0212] It should be understood that each block in the flowchart and / or block diagram, as well as combinations of blocks in the flowchart and / or block diagram, can be implemented by computer-readable program instructions. These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine. These computer-readable program instructions are executed by the computer or other programmable data processing device to produce a device that implements the functions / operations specified in the blocks in the flowchart and / or block diagram.

[0213] These computer-readable program instructions may also be stored in a computer-readable storage medium that enables a computer or other programmable data processing device to operate in a specific manner. In this way, the instructions stored in the computer-readable storage medium produce an instruction device product that implements the functions / operations specified in the blocks in the flowchart and / or block diagram.

[0214] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby enabling the instructions executed on the computer or other programmable data processing apparatus to provide a process that implements the functions / operations specified by the blocks in the flowchart and / or block diagram.

[0215] The above description is merely a specific implementation of the embodiments of the present invention, but the scope of protection of the embodiments of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the embodiments of the present invention should be included in the scope of protection of the embodiments of the present invention. Therefore, the scope of protection of the embodiments of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for image processing, characterized in that: include: Acquire a first image and a second image to be stitched, and extract first feature points in the first image and second feature points in the second image; generating an antigen based on the first feature point, and generating an antibody based on the second feature point; Repeat multiple rounds of clone selection until the iteration ends and the global optimal antibody is determined; registering the first image and the second image by using a first feature point corresponding to the antigen matching the global optimal antibody and a second feature point corresponding to the global optimal antibody as a registration point pair; The clone selection operation includes: Determine the local optimal antibody that matches the current round antigen based on the clonal selection algorithm, and memorize the local optimal antibody; the global optimal antibody is the optimal antibody determined based on multiple local optimal antibodies; When further iteration is required, mutating the current round antigen to generate an antigen for performing the next round of clonal selection operation; The determining of the local optimal antibody matching the current round antigen comprises: Determining the affinity between the current round antigen and the current round antibody, and taking the current round antibody with the highest affinity as the local optimal antibody matching the current round antigen; Determining the affinity between the current round antigen and the current round antibody comprises: Determining an affinity domain, wherein the affinity domain is used to represent the scope of action when determining the affinity between the antibody and the antigen; Determine the characteristic difference between the current round antigen and the current round antibody, and determine the similarity between the current round antigen and the current round antibody within the affinity scope, and determine the affinity between the current round antigen and the current round antibody based on the characteristic difference and the similarity; the affinity is negatively correlated with the characteristic difference, and the affinity is positively correlated with the similarity.

2. The method according to claim 1, characterized in that Determining the affinity scope includes: An affinity scope of a corresponding shape is selected according to the splicing features of the first image and the second image, and the shapes of the affinity scope include: rectangle, cross, line, ellipse, and circle.

3. The method according to claim 1, characterized in that Determining the characteristic difference between the current round antigen and the current round antibody comprises: Determine the characteristic value A(i) of the first characteristic point corresponding to the current round antigen i, determine the characteristic value B(j) of the second characteristic point corresponding to the current round antibody j, and use the absolute value of the difference between the characteristic value A(i) and the characteristic value B(j) as the characteristic difference between the current round antigen i and the current round antibody j; Determining the similarity between the current round antigen and the current round antibody within the affinity domain comprises: Determining pixel values ​​of a plurality of pixels within a first window in the first image, and determining pixel values ​​of a plurality of pixels within a second window in the second image; the first window is a window based on a first feature point corresponding to the current round antigen i, and the second window is a window based on a second feature point corresponding to the current round antibody j, and both the first window and the second window match the affinity scope; and The similarity between the current round antigen i and the current round antibody j is determined according to the pixel values ​​of the multiple pixel points in the first window and the pixel values ​​of the multiple pixel points in the second window.

4. The method according to claim 3, characterized in that The affinity between the current round antigen and the current round antibody satisfies: ； in, represents the affinity between the current round antigen i and the current round antibody j, represents the similarity between the current round antigen i and the current round antibody j, b is an adjustment coefficient not less than 1, 、 Both represent weighting coefficients.

5. The method according to claim 4, characterized in that The similarity between the current round antigen i and the current round antibody j satisfy: ； in, Indicates that the first window is located at position The pixel value of the pixel at Indicates that the second window is located at position The pixel value of the pixel at represents the average value of the pixel values ​​of multiple pixels in the first window, Represents the average pixel value of multiple pixels in the second window.

6. The method according to any one of claims 1 to 5, characterized in that Also includes: The registered first image and the registered second image are stitched together to form a fused image.

7. The method according to claim 6, characterized in that The number of the registration point pairs is at least three; The first image and the second image after stitching and registration include: Based on at least three of the registration point pairs, a clonal selection algorithm is used to perform an optimization solution to determine coordinate transformation parameters between the first image and the second image; Coordinate transformation is performed on one of the first image and the second image based on the coordinate transformation parameters, and then the other of the first image and the second image is stitched together.

8. An image processing device, characterized in that: include: a preprocessing module, configured to obtain a first image and a second image to be stitched, and extract first feature points in the first image and second feature points in the second image; a generating module, configured to generate an antigen based on the first feature point, and generate an antibody based on the second feature point; The iteration module is used to cyclically execute multiple rounds of clone selection operations until the iteration ends and the global optimal antibody is determined; a registration module, configured to register the first image and the second image by using a first feature point corresponding to the antigen matching the global optimal antibody and a second feature point corresponding to the global optimal antibody as a registration point pair; Wherein, the iteration module includes: a clonal selection unit, configured to determine a local optimal antibody that matches the current round of antigens based on a clonal selection algorithm, and memorize the local optimal antibody; the global optimal antibody is the optimal antibody determined based on multiple local optimal antibodies; an antigen mutation unit, configured to mutate the antigens of the current round, if further iteration is required, to generate antigens for performing the next round of clonal selection operations; The clonal selection unit determines the local optimal antibody that matches the current round antigen, comprising: Determining the affinity between the current round antigen and the current round antibody, and taking the current round antibody with the highest affinity as the local optimal antibody matching the current round antigen; The clone selection unit comprises: A first determination subunit is used to determine an affinity domain, where the affinity domain represents a range of action when determining the affinity between an antibody and an antigen; The second determination subunit is used to determine the characteristic difference between the current round antigen and the current round antibody, and determine the similarity between the current round antigen and the current round antibody within the affinity scope, and determine the affinity between the current round antigen and the current round antibody based on the characteristic difference and the similarity; the affinity is negatively correlated with the characteristic difference, and the affinity is positively correlated with the similarity.

9. An electronic device comprising a bus, a transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the transceiver, the memory, and the processor are connected via the bus, wherein: When the computer program is executed by the processor, the steps of the image processing method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 7 are implemented.

Images