Image processing method and device and electronic equipment

Abstract

The invention discloses an image processing method and device and electronic equipment, and the method comprises the steps: determining at least one first region of interest in a first sub-image and a second sub-image, and determining the image features of at least two first regions of interest; the first subimage and the second subimage represent mutually overlapped areas in the first image and the second image respectively; and splicing the first image and the second image into a target image based on the image feature matching relationship of the at least two first regions of interest in the first sub-image and the second sub-image.

Description

Technical Field

[0001] The present disclosure relates to, but is not limited to, the field of computer technology, and particularly relates to an image processing method, apparatus, and electronic device. Background Art

[0002] Image stitching refers to an image processing technology that stitches at least two images into one image. With the progress of image sensor technology, for example, the ultra-high-resolution industrial cameras that have entered the commercial field in recent years, the image resolution has gradually increased, which has led to a substantial increase in the computing resources consumed by image stitching. If image stitching is completed by reducing the image resolution, it is easy to cause local stitching errors. Therefore, how to improve the quality of image stitching and reduce the computing resources consumed by image stitching has become an urgent problem to be solved. Summary of the Invention

[0003] In view of this, the present disclosure provides at least an image processing method, apparatus, and electronic device.

[0004] The technical solution of the present disclosure is implemented as follows:

[0005] On the one hand, the present disclosure provides an image processing method, which includes:

[0006] Determine at least one first interest region in each of a first sub-image and a second sub-image, and determine the image features of at least two first interest regions; the first sub-image and the second sub-image respectively represent the overlapping regions of a first image and a second image with each other;

[0007] Based on the image feature matching relationship of at least two first interest regions in the first sub-image and the second sub-image, stitch the first image and the second image into a target image.

[0008] On the other hand, the present disclosure also provides an image processing apparatus, including:

[0009] A determination module, which determines at least one first interest region in each of a first sub-image and a second sub-image, and determines the image features of at least two first interest regions; the first sub-image and the second sub-image respectively represent the overlapping regions of a first image and a second image with each other;

[0010] A stitching module, which stitches the first image and the second image into a target image based on the image feature matching relationship of at least two first interest regions in the first sub-image and the second sub-image.

[0011] On the other hand, the present disclosure also provides an electronic device, which includes a processing unit and a communication unit; wherein,

[0012] The communication unit is configured to obtain a first image and a second image from an image acquisition device;

[0013] A processing unit is configured to respectively determine at least one first region of interest in a first sub-image and a second sub-image, and determine image features of at least two first regions of interest; the first sub-image and the second sub-image respectively represent overlapping regions of a first image and a second image with each other.

[0014] Based on the image feature matching relationship of at least two first regions of interest in the first sub-image and the second sub-image, the first image and the second image are stitched into a target image.

[0015] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the technical solutions of the present disclosure. Description of the Drawings

[0016] The drawings herein are incorporated into the specification and constitute a part of this specification. These drawings show embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure.

[0017] Figure 1A It is a schematic diagram of a workpiece image captured by a line-scan camera in an industrial assembly line;

[0018] Figure 1B It is a schematic diagram of a workpiece image captured by a line-scan camera in an industrial assembly line;

[0019] Figure 2 It is a schematic diagram of a process of performing image stitching based on an image stitching method in the related art;

[0020] Figure 3 It is a schematic diagram of an implementation process of an image processing method provided by the present disclosure;

[0021] Figure 4 It is a schematic diagram of a process of determining a second region of interest based on the image processing method provided by the present disclosure;

[0022] Figure 5 It is a schematic diagram of a process of performing image stitching based on the image processing method provided by the present disclosure;

[0023] Figure 6 It is a schematic diagram of an implementation process of an embodiment of the image processing method provided by the present disclosure;

[0024] Figure 7 It is a schematic diagram of a composition structure of an image processing apparatus provided by the present disclosure;

[0025] Figure 8 It is a schematic diagram of a hardware entity of an electronic device provided by the present disclosure. Detailed Embodiments

[0026] To make the objectives, technical solutions, and advantages of the present disclosure clearer, the technical solutions of the present disclosure will be further elaborated in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be construed as limitations on the present disclosure. All other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present disclosure.

[0027] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments and can be combined with each other without conflict.

[0028] The terms "first / second / third" involved are only used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first / second / third" can be interchanged with a specific order or sequence when permitted, so that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein.

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this disclosure belongs. The terms used herein are only for the purpose of describing this disclosure and are not intended to limit this disclosure.

[0030] In the scenario of appearance defect detection in an industrial assembly line, a line scan camera is usually used to take appearance pictures of workpieces on the assembly line, and appearance defects of the workpieces are identified based on the taken images. In order to reduce the time cost of taking workpiece images, there is often a need to splice images of workpieces taken under the occlusion of different conveyor belts in this scenario of appearance defect detection in the industrial assembly line.

[0031] Figure 1A and Figure 1B are respectively images obtained by the line scan camera taking pictures of the workpiece 110 placed on the assembly line 120. From Figure 1A and Figure 1B it can be seen that due to the occlusion of the conveyor belt 130, conveyor belt 140, and conveyor belt 150, there is a certain degree of overlap in the effective areas of the workpiece 110 captured by the line scan camera, such as Figure 1A the left part of the conveyor belt 130 and Figure 1B the right part of the conveyor belt 140 in. Based on this overlapping part, the images shown in Figure 1A and Figure 1B can be spliced to obtain a complete image of the workpiece 110 for facilitating appearance defect detection.

[0032] As described above, currently, industrial cameras with ultra-high resolution are widely used, so some workpiece images have a relatively high resolution. To reduce the computing resources consumed by stitching high-resolution images, the following solutions have been proposed in the related art.

[0033] First, manually determine the overlapping area in two images to be stitched. As Figure 2 shown, the shaded area in the image to be stitched 21 and the image 22 is the overlapping area of the two images to be stitched;

[0034] Then, perform a reduction process on the original image according to a preset reduction ratio to obtain a reduced image. As Figure 2 shown, the images 23 and 24 are the reduced images corresponding to the images 21 and 22 respectively;

[0035] After that, calculate the image features of the overlapping area based on the reduced image, and perform a matching calculation on the overlapping area based on the image features to obtain a matching image and an affine transformation matrix. As Figure 2 shown, perform a matching calculation based on the image features of the overlapping area of the images 23 and 24 to obtain a matching image 25;

[0036] After that, perform an enlargement process on the matching image 25 according to the enlargement ratio corresponding to the above reduction multiple to obtain an enlarged image, such as Figure 2 the image 26 shown;

[0037] Finally, perform an affine transformation on the basis of the enlarged image based on the affine transformation matrix to obtain a stitched image, such as Figure 2 the image 27 shown.

[0038] In the above technical solution, there are slight deviations when restoring the image features calculated based on the reduced image to the original image coordinates. In the high-resolution image stitching scenario, this slight deviation will directly affect the image stitching quality after affine transformation. Therefore, for the high-resolution image stitching scenario, this solution still has problems with stitching quality.

[0039] Based on this, the present disclosure provides an image processing method, which can be executed by an electronic device. The electronic device can be various types of terminals such as a laptop computer, a tablet computer, a desktop computer, a set-top box, a mobile device (e.g., a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, a portable gaming device), etc., or can be implemented as a server. The server can be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0040] Next, in combination with the accompanying drawings of the present disclosure, the technical solutions in the present disclosure will be described clearly and completely.

[0041] Figure 3 It is a schematic implementation flowchart of an image processing method provided by the present disclosure. As Figure 3 shown, the method includes the following steps S301 to step S302:

[0042] Step S301, respectively determine at least one first region of interest in the first sub-image and the second sub-image, and determine the image features of at least two first regions of interest; the first sub-image and the second sub-image respectively represent the overlapping regions of the first image and the second image with each other.

[0043] Here, the first image and the second image are images to be stitched. In some embodiments, the first image and / or the second image can be an image obtained by preprocessing the original image. For example, it can be an image obtained by filtering, enhancing, or sharpening the original image, etc. In some embodiments, the first image and the second image can be images in any format, such as JPG format, GIF format (Graphics Interchange Format), TIFF format (Tagged Image File Format), RAW format, and so on.

[0044] The first sub-image and the second sub-image respectively represent the overlapping regions of the first image and the second image, that is, the first sub-image and the second sub-image are the common and mutually overlapping parts of the first image and the second image. In some embodiments, the overlapping regions of the first image and the second image can be determined manually and demarcated as the first sub-image and the second sub-image. In some embodiments, the overlapping regions of the first image and the second image can be determined by performing a preliminary detection (such as texture detection, etc.) on the first image and the second image that consumes less computing resources.

[0045] The first region of interest refers to any type of region determined respectively from the first sub-image and the second sub-image. In some embodiments, the first region of interest can be a region of any shape. For example, the shape of the first region of interest can be a rectangle, a circle, an ellipse, or an irregular polygon, etc. In some embodiments, different first regions of interest in the first sub-image and the second sub-image can have different shapes. For example, in the case where there are two first regions of interest in the first sub-image or the second sub-image, the two first regions of interest can be a circle and a rectangle respectively.

[0046] In some embodiments, at least one first region of interest can be determined respectively from the first sub-image and the second sub-image by manual demarcation, that is, at least one first region of interest is determined by the staff based on work experience.

[0047] In some embodiments, a specified image detection algorithm can be used to perform image detection on the first sub-image and the second sub-image respectively to determine at least one first region of interest. In implementation, the image detection algorithm can include but is not limited to a sliding window algorithm, a saliency detection algorithm, a selective search algorithm, a deep learning-based object detection algorithm, etc.

[0048] The image features of the first region of interest refer to any information that can be used to describe, distinguish, or identify the image content of the first region of interest. In some embodiments, the image features can be any type of features. In some embodiments, the image features can be color features, texture features, shape features, spatial relationship features, pixel statistical features, or transform domain features, etc. corresponding to the first region of interest.

[0049] In some embodiments, the image features of the first region of interest can be determined by any suitable image feature detection algorithm. In one embodiment, based on the type of the image features, the detection method of the image features is determined. For example, when the image features are color features, algorithms such as color histogram, color moment or color correlation vector can be used to determine the image features of the first region of interest; for another example, when the image features are texture features of the image, algorithms such as gray level co-occurrence matrix algorithm, fractal model or mathematical morphology algorithm can be used to determine the image features of the first region of interest; and so on.

[0050] Step S302: Based on the image feature matching relationship of at least two first regions of interest in the first sub-image and the second sub-image, splice the first image and the second image into a target image.

[0051] Here, after determining the image features of at least two first regions of interest in the first sub-image and the second sub-image, image feature matching is performed based on the image features of the at least two first regions of interest to determine the image feature matching relationship.

[0052] In some embodiments, the image feature matching relationship can be the image feature matching relationship between multiple pixel points in the first region of interest of the first sub-image and multiple pixel points in the first region of interest of the second sub-image established based on the image features. In this way, the matching relationship between the first image and the second image can be established based on the image feature matching relationship between the multiple pixel points, and the first image and the second image are spliced based on this matching relationship to obtain the target image.

[0053] In some embodiments, the image feature matching relationship can be the image feature matching relationship between multiple sub-regions in the first region of interest of the first sub-image and multiple sub-regions in the first region of interest of the second sub-image established based on the image features. In this way, the matching relationship between the first image and the second image can be established based on the image feature matching relationship between the multiple sub-regions, and the first image and the second image are spliced based on this matching relationship to obtain the target image.

[0054] In the image processing method provided by the present disclosure, first, at least one first region of interest in the first sub-image and the second sub-image is respectively determined, and the image features of at least two first regions of interest are determined, where the first sub-image and the second sub-image respectively represent the overlapping regions in the first image and the second image; then, based on the image feature matching relationship of at least two first regions of interest in the first sub-image and the second sub-image, the first image and the second image are stitched into a target image. In this way, the image features of the first region of interest are determined based on the original resolution of the images to be stitched, that is, when calculating the image features, the first image and the second image to be stitched are not subjected to a reduction process. Therefore, compared with the problem that there will be a deviation when restoring the image features calculated based on the reduced image to the original image resolution in the related art, the present disclosure can retain the original image features of the images to be stitched to the greatest extent, and thus can obtain a better image stitching effect when performing image stitching based on the image features, improving the robustness of the image stitching method.

[0055] In some embodiments, respectively determining at least one first region of interest in the first sub-image and the second sub-image in step S301 above can be implemented as the following steps S3011 to S3013:

[0056] Step S3011, respectively perform a reduction process on the first sub-image and the second sub-image to obtain corresponding first reduced images and second reduced images.

[0057] Here, when performing the reduction process on the first sub-image and the second sub-image, the reduction process can be performed based on any suitable reduction ratio. In some embodiments, the reduction ratio can be determined based on the resolutions of the first sub-image and the second sub-image and a preset target resolution; where the resolution of the first sub-image and the resolution of the second sub-image can be the same or different; the preset target resolution refers to the resolutions of the first reduced image and the second reduced image, and this target resolution is less than the resolutions of the first sub-image and the second sub-image.

[0058] In some embodiments, the target resolution can be determined based on any suitable standard. In some example embodiments, since different image detection purposes have different resolution requirements for the detected images, therefore, the target resolution can be determined based on the resolution requirements of the image features to be determined (for example, color features, texture features, shape features, spatial relationship features, pixel statistical features or transform domain features) for the images to be detected. In some embodiments, the target resolution can be determined based on the available computing resources. For example, when the amount of available computing resources is large, in order to improve the processing accuracy of the first reduced image and the second reduced image, a higher target resolution can be set; on the contrary, a lower target resolution is set.

[0059] In some embodiments, any suitable method may be adopted to perform downsampling processing on the first sub-image and the second sub-image respectively.

[0060] Step S3012: respectively determine at least one second region of interest in the first downsampled image and the second downsampled image.

[0061] Here, the second region of interest refers to any type of region respectively determined from the first downsampled image and the second downsampled image, and is used to assist in determining the first region of interest. In some embodiments, the second region of interest may be a region of any shape. For example, the second region of interest may be a region in the shape of a rectangle, a circle, an ellipse, or an irregular polygon. In some embodiments, different second regions of interest in the first downsampled image and the second downsampled image may have different shapes.

[0062] In some embodiments, at least one second region of interest may be respectively determined from the first downsampled image and the second downsampled image by manual delineation, that is, the second region of interest is determined by the staff based on work experience.

[0063] In some embodiments, a specified image detection algorithm may be used to perform image detection on the first downsampled image and the second downsampled image respectively to determine at least one second region of interest. In implementation, the image detection algorithm may be any suitable image detection algorithm such as a sliding window algorithm, a saliency detection algorithm, a selective search algorithm, or an object detection algorithm based on deep learning.

[0064] Step S3013: based on the second region of interest, respectively determine at least one first region of interest in the first sub-image and the second sub-image.

[0065] Since the first downsampled image and the second downsampled image are respectively images obtained after performing downsampling processing on the first sub-image and the second sub-image, the second regions of interest in the first downsampled image and the second downsampled image may be enlarged according to the enlargement ratio corresponding to the downsampling ratio to obtain the first regions of interest corresponding to each second region of interest in the first sub-image and the second sub-image.

[0066] In the embodiments of the present disclosure, determining the second region of interest based on the downsampled image and further determining the first region of interest based on the second region of interest can reduce the computational resource cost and time cost required for determining the region of interest.

[0067] In some embodiments, the step of respectively determining at least one second region of interest in the first downsampled image and the second downsampled image, that is, the above step S3012, may be implemented as the following steps S30121 to S30122:

[0068] Step S30121: Perform saliency detection on the first reduced image and the second reduced image to obtain the saliency features in the first reduced image and the second reduced image.

[0069] Here, the saliency feature refers to a feature that is more prominent than the surrounding environment in the image and has a high degree of distinctiveness. Based on the saliency feature, the corresponding image can be quickly distinguished and recognized. In some embodiments, the saliency feature may include color features, brightness features, texture features, edge features, or orientation features exhibited by pixel points or regions in the image.

[0070] The saliency feature can be obtained by performing saliency detection on the first reduced image and the second reduced image. In some embodiments, any suitable saliency detection algorithm can be used to perform saliency detection. In implementation, the saliency detection algorithm may include, but is not limited to, the Itti visual saliency detection algorithm, the residual spectrum algorithm, the frequency-tuned (FT) saliency detection algorithm, and the attention prediction algorithm based on deep learning.

[0071] Step S30122: Based on the saliency features, respectively determine at least one second region of interest in the first reduced image and the second reduced image.

[0072] Here, after determining the saliency features in the first reduced image and the second reduced image, the second region of interest is determined based on the saliency feature.

[0073] In some embodiments, the region containing the saliency feature can be used as the second region of interest. In some embodiments, any suitable shape can be used to distinguish each determined saliency feature from the surrounding environment, and the region inside the shape is used as the second region of interest. For example, circles, ellipses, rectangles, polygons, etc. can be used to distinguish each saliency feature. In some embodiments, the irregular polygon region formed by the boundary lines of each saliency feature can be used as the second region of interest.

[0074] In some embodiments, when the saliency feature region is large, a region containing a part of the saliency feature can be used as the second region of interest. In some embodiments, a region corresponding to any shape can be determined in the region where each saliency feature is located as the second region of interest.

[0075] In an embodiment of the present disclosure, the second region of interest determined by saliency detection includes saliency features. Therefore, the first region of interest determined based on the second region of interest is also the region including saliency features in the first sub-image and the second sub-image. In this way, the image features corresponding to the first region of interest can more accurately represent the image features of the corresponding sub-image.

[0076] In some embodiments, based on the saliency features, at least one second region of interest in the first reduced image and the second reduced image is respectively determined. That is, step S30122 described above can be implemented as the following steps S30123 to step S30124:

[0077] Step S30123: Using a dilation algorithm, aggregate the saliency features in the first reduced image and the second reduced image to obtain the aggregated saliency features.

[0078] Here, the dilation algorithm refers to making an object larger by expanding the foreground object in the image, thereby filling small holes or broken parts in the foreground object.

[0079] In this embodiment, for the first reduced image and the second reduced image, the saliency features therein are used as the foreground objects, and the aggregation is performed using the dilation algorithm to obtain the aggregated saliency features.

[0080] Step S30124: Based on the aggregated saliency features, respectively determine at least one second region of interest in the first reduced image and the second reduced image.

[0081] Here, after obtaining the aggregated saliency features, the second regions of interest are respectively determined based on the aggregated saliency features in the first reduced image and the second reduced image. In some embodiments, the region containing the aggregated saliency features can be used as the second region of interest. In some embodiments, any suitable shape can be used to distinguish each determined aggregated saliency feature from the surrounding environment, and the region inside the shape can be used as the second region of interest. In some embodiments, the irregular polygon region formed by the boundary lines of each aggregated saliency feature can be used as the second region of interest.

[0082] Next, in combination with Figure 4 , the process of determining the second region of interest using the image processing method provided by the present disclosure will be described. Among them, the first reduced image 41 and the second reduced image 42 respectively include saliency features represented by circles, semi-circular rings or rectangles, etc.

[0083] First, perform saliency binarization extraction on the first reduced image 41 and the second reduced image 42 respectively to obtain mask images of the saliency feature regions, that is, the mask image 43 corresponding to the first reduced image 41 and the mask image 44 corresponding to the second reduced image 42;

[0084] Then, use the dilation algorithm to aggregate the white regions in the image 43 and the image 44 respectively to obtain the corresponding aggregated saliency feature regions, that is, the regions marked with dot-shaped shadows in the images 45 and 46;

[0085] Finally, extract the bounding boxes of the regions marked with dot-shaped shadows in the images 45 and 46 to obtain a plurality of second interest regions, that is, the regions shown by the plurality of solid rectangle frames in the images 47 and 48; among them, the images 47 and 48 correspond to the first reduced image 41 and the second reduced image 42 respectively.

[0086] In the embodiment provided by the present disclosure, the dilation algorithm is used to aggregate the saliency features in the reduced image, so that the range of the second interest region determined based on the aggregated saliency features is larger, and further the range of the first interest region determined based on the second interest region is larger. In this way, more refined and rich image features can be extracted based on the first interest region with a larger range, improving the image stitching quality.

[0087] In some embodiments, the image processing method provided by the present disclosure further includes the following step S303:

[0088] Step S303, based on the first interest region, determine at least one third interest region in the first sub-image and the second sub-image respectively.

[0089] Here, after determining the first interest regions in the first sub-image and the second sub-image, further determine the third interest regions based on the first interest regions.

[0090] In some embodiments, the third interest region may be a supplementary interest region corresponding to the first interest region to expand the range of the interest regions on the first sub-image and the second sub-image. In implementation, for the first sub-image and the second sub-image, a preset number of supplementary interest regions may be added between adjacent first interest regions as the third interest regions.

[0091] In some embodiments, the third interest region is an extended interest region corresponding to the first interest region to expand the range of the interest regions on the first sub-image and the second sub-image. In implementation, for the first sub-image and the second sub-image, taking the boundary of each first interest region as the starting line, expand a specified distance outward to obtain the corresponding third interest region.

[0092] At least one third interest region in the first sub-image and the second sub-image is respectively determined, and image features are extracted for each third interest region. In implementation, the image features of each third interest region can be extracted by using the same or different image feature extraction methods as those for the first interest region.

[0093] In this way, based on the image feature matching relationship of at least two first interest regions in the first sub-image and the second sub-image, the first image and the second image are stitched into a target image. That is, the above step S302 can be implemented as the following step S3021:

[0094] Step S3021: Based on the image feature matching relationship of at least two first interest regions and at least two third interest regions in the first sub-image and the second sub-image, the first image and the second image are stitched into a target image.

[0095] Here, after determining at least two first interest regions and their corresponding image features in the first sub-image and the second sub-image, and the image features corresponding to at least two third interest region clusters, image feature matching is performed on at least one first interest region and at least one third interest region in the first sub-image and at least one first interest region and at least one third interest region in the second sub-image, and the image feature matching relationship is determined.

[0096] In some embodiments, the image feature matching relationship can be the image feature matching relationship between multiple pixel points in the first interest region and the third interest region in the first sub-image and the first interest region and the third interest region in the second sub-image based on the image features. In this way, the matching relationship between the first image and the second image can be established based on the image feature matching relationship between multiple pixel points, and the first image and the second image are stitched based on this matching relationship to obtain the target image.

[0097] In some embodiments, the image feature matching relationship can be the image feature matching relationship between multiple sub-regions in the first interest region and the third interest region in the first sub-image and the first interest region and the third interest region in the second sub-image based on the image features. In this way, the matching relationship between the first image and the second image can be established based on the image feature matching relationship between multiple sub-regions, and the first image and the second image are stitched based on this matching relationship to obtain the target image.

[0098] In an embodiment of the present disclosure, by further determining a third region of interest in the first sub-image and the second sub-image, and performing image stitching on the first image and the second image based on the image feature matching relationship between the first region of interest and the third region of interest, the range of feature extraction for the first sub-image and the second sub-image is expanded. Furthermore, a more accurate and rich image feature matching relationship can be established, improving the quality of image stitching.

[0099] In some embodiments, each of the first sub-image and the second sub-image includes a first region of interest; the third region of interest includes a fourth region of interest.

[0100] Here, when there is only one first region of interest in both the first sub-image and the second sub-image, the area of the non-region of interest in the first sub-image and the second sub-image is much larger than the area of the region of interest. If the image feature matching relationship is determined only based on these two first regions of interest, it will result in the calculated image feature matching relationship being concentrated in a partial area of the first sub-image and the second sub-image, thereby leading to the problem of low image stitching accuracy.

[0101] Based on this, in an embodiment of the present disclosure, the step of respectively determining at least one third region of interest in the first sub-image and the second sub-image based on the first region of interest, that is, the above step S303, can be implemented as the following step S3031:

[0102] Step S3031: Based on a preset first spacing, respectively determine at least one fourth region of interest in the first sub-image and the second sub-image; wherein, the shape and / or area of the fourth region of interest is determined based on the corresponding first region of interest.

[0103] Here, the fourth region of interest is a supplementary region of interest for the first region of interest. The shape and / or area of the fourth region of interest is determined based on the corresponding first region of interest, that is, the fourth region of interest has the same or similar shape and / or area as the corresponding first region of interest, so as to quickly determine the fourth region of interest based on the first region of interest.

[0104] In some embodiments, for the first sub-image and the second sub-image, the first region of interest and at least one fourth region of interest are both arranged along the length direction or the width direction of the image.

[0105] The first spacing can be any suitable spacing. In some embodiments, in order to obtain more image feature matching relationships, a smaller first spacing can be set. In some embodiments, in order to improve the calculation speed of image features, a larger first spacing can be set.

[0106] In an embodiment of the present disclosure, by respectively determining at least one fourth region of interest in a first sub-image and a second sub-image each including only one first region of interest, the area of the regions of interest in the first sub-image and the second sub-image can be enlarged, so that more image feature matching relationships can be obtained, and the image stitching quality can be improved.

[0107] In some embodiments, each of the first sub-image and the second sub-image includes at least two first regions of interest; the third region of interest includes a fifth region of interest.

[0108] Here, each of the first sub-image and the second sub-image includes at least two first regions of interest. To obtain more image feature matching relationships, at least one third region of interest, that is, at least one fifth region of interest, may be additionally determined in the first sub-image and the second sub-image.

[0109] In this way, the step of respectively determining at least one third region of interest in the first sub-image and the second sub-image based on the first region of interest, that is, step S303 above, may be implemented as the following step S3032:

[0110] Step S3032: Based on a second spacing, respectively determine at least one fifth region of interest in sparse regions of the first sub-image and the second sub-image; where

[0111] the second spacing represents an average value of distances between multiple adjacent first regions of interest;

[0112] the sparse region represents a region between adjacent first regions of interest with a distance greater than the average value, or a region between an image boundary and the closest first region of interest with a distance greater than the average value;

[0113] the shape and / or area of the fifth region of interest is determined based on corresponding at least two first regions of interest.

[0114] Here, for each sub-image (i.e., the first sub-image or the second sub-image), when determining at least one fifth region of interest:

[0115] First, determine distances between multiple adjacent first regions of interest in the sub-image, and calculate an average value of the determined multiple distances;

[0116] Then, determine whether the distance between each adjacent first region of interest is greater than the average value. If it is greater, determine the region between the adjacent first regions of interest as a sparse region; at the same time, determine whether the distance between the image boundary line and the closest first region of interest is greater than the average value. If it is greater, determine the region between the image boundary line and the closest first region of interest as a sparse region;

[0117] After that, based on the shapes and / or areas of at least two first regions of interest in the sub-image, determine the shape and / or area of a fifth region of interest in the sub-image;

[0118] In some embodiments, the average value of the areas of at least two first regions of interest may be used as the area of the fifth region of interest, and the shape of any one of the first regions of interest may be used as the shape of the fifth region of interest. In some embodiments, the area and shape of the first region of interest with the largest or smallest area among at least two first regions of interest may be used as the area and shape of the fifth region of interest. Here, the manner of determining the area and / or shape of the corresponding fifth region of interest based on at least two first regions of interest is not limited.

[0119] Finally, based on the determined area and shape of the fifth region of interest, determine at least one fifth region of interest on the sub-image at a second interval.

[0120] In some embodiments, at least one fifth region of interest and at least two first regions of interest may be determined along the length direction or the width direction of the sub-image.

[0121] In the embodiments of the present disclosure, by respectively determining at least one fifth region of interest in the first sub-image and the second sub-image, the area of the region of interest in the sub-image can be enlarged, and further, more reminder feature matching relationships can be obtained, improving the quality of image stitching.

[0122] In some embodiments, based on the image feature matching relationships of at least two first regions of interest and at least two third regions of interest in the first sub-image and the second sub-image, stitching the first image and the second image into a target image, that is, the above step S3021 can be implemented as the following steps S30211 to S30213, including:

[0123] Step S30211, based on the image features of the first region of interest and the third region of interest, determine the coordinate information of at least three pixel points having an image feature matching relationship in the first sub-image and the second sub-image.

[0124] Here, after determining the image features, any suitable feature matching algorithm can be used to determine at least three pixel points with an image feature matching relationship in the first sub-image and the second sub-image and their coordinate information. In one embodiment, algorithms such as the Random Sample Consensus (RANSAC), Brute Force Matching, K-Nearest Neighbor Matching algorithm, or the Fast Library for Approximate Nearest Neighbors (FLANN) algorithm can be used to determine at least three pixel points and their coordinate information.

[0125] For example, when using the brute force matching algorithm to determine the image feature matching relationship, calculate the distance (e.g., Euclidean distance or Hamming distance) between the descriptors of each pixel point in the region of interest of the first sub-image (i.e., at least one first region of interest and at least one third region of interest) and the descriptors of each pixel point in the region of interest of the first sub-image (i.e., at least one first region of interest and at least one third region of interest), and determine the pixel point with the smallest distance as the pixel point with an image feature matching relationship.

[0126] Step S30212, based on the coordinate information, establish an affine transformation matrix.

[0127] Here, the affine transformation matrix is a matrix used to assist in performing affine transformations (e.g., linear transformations (e.g., rotation, scaling, shearing, etc.) and translation, etc.) on elements (e.g., pixels) in an image. When solving the affine transformation matrix, at least 3 non-collinear control point information is required to determine the unknown parameters in the affine transformation matrix. In implementation, the coordinate information of the above at least three pixel points is used as 3 control point information to establish the affine transformation matrix.

[0128] In implementation, the number of pixel points used to establish the affine transformation matrix will directly affect the solution accuracy of the affine transformation matrix. Therefore, in the above embodiments of the present disclosure, by respectively determining the first region of interest and the supplementary region of interest (i.e., the third region of interest) of the first region of interest on the first sub-image and the second sub-image, the area of the region of interest is expanded, so that more pixel points with an image feature matching relationship can be obtained, thereby improving the accuracy of the established affine transformation matrix.

[0129] In implementation, the more uniformly the pixel points for establishing the affine transformation matrix are distributed on the image, the higher the accuracy of the established affine transformation matrix. Therefore, in the above implementation manner of the present disclosure, by determining at least one third region of interest in the sparse regions of the first sub-image and the second sub-image, the coverage range of the region of interest for each sub-image is expanded, so that pixel points with an image feature matching relationship that are uniformly distributed on each sub-image can be obtained, thereby improving the accuracy of the established affine transformation matrix.

[0130] Step S30213, based on the affine transformation matrix, perform stitching processing on the first image and the second image to obtain the target image.

[0131] Here, based on the affine transformation matrix, perform an affine transformation (i.e., perform a linear transformation (such as rotation, scaling, shearing, etc.) or a translation operation, etc.) on the pixel points in the first image and the second image to obtain the target image. In some implementation manners, use the affine transformation matrix to transform the coordinates of the pixel points in the first image into the coordinate system of the pixel points in the second image, or transform the coordinates of the pixel points in the second image into the coordinate system of the pixel points in the first image, thereby obtaining the target image.

[0132] Next, in combination with Figure 5 , the process of image stitching for two images taken in the above industrial pipeline scenario and having a conveyor belt occlusion problem will be described.

[0133] First, manually determine the overlapping regions in the images 51 and 52 to be stitched, that is, the shaded regions in the images 51 and 52, to obtain the sub-images 53 and 54;

[0134] Then, perform a reduction process on the sub-images 53 and 54 respectively to obtain the reduced images 55 and 56;

[0135] After that, perform a saliency detection on the reduced images 55 and 56 respectively, and determine a plurality of second regions of interest on the reduced images based on the detected saliency features, such as the regions shown in the solid rectangular frames in the images 57 and 58;

[0136] After that again, perform an enlargement process on the images 57 and 58 containing a plurality of second regions of interest respectively to obtain the images 59 and 510; wherein, the images 59 and 510 respectively contain a plurality of first regions of interest corresponding to the plurality of second regions of interest in the images 57 and 58;

[0137] After that again, for the sparse regions in the images 59 and 510, determine the supplementary regions of interest of the plurality of first regions of interest respectively, that is, a plurality of third regions of interest, such as the regions shown in the dashed rectangular frames in the images 511 and 512;

[0138] After that, for the multiple first interest regions shown by the solid rectangular frames and the multiple third interest regions shown by the dashed rectangular frames in Image 511 and Image 512, calculate the image features respectively and perform image feature matching to determine multiple pixel points having an image feature matching relationship. As shown in Image 513, connect the pixel points having an image feature matching relationship with straight lines;

[0139] Finally, based on the coordinate information of the multiple pixel points having an image feature matching relationship, establish an affine transformation matrix, and use the affine transformation matrix to transform the pixel point coordinates of Image 51 to the pixel point coordinate system of Image 52 to complete image stitching and obtain the target image 514.

[0140] Next, in conjunction with Figure 6 , the implementation process of an embodiment of the image processing method according to the present disclosure will be described. As Figure 6 described, this embodiment includes the following steps S601 to step S610

[0141] Step S601, obtain a third image and a fourth image to be stitched; after that, execute step S602;

[0142] Step S602, extract the overlapping regions of the third image and the fourth image to obtain a third sub-image and a fourth sub-image; after that, execute step S603;

[0143] Step S603, perform a shrinking process on the third sub-image and the fourth sub-image respectively to obtain a third shrunk image and a fourth shrunk image; after that, execute step S604;

[0144] Step S604, perform a saliency detection on the third shrunk image and the fourth shrunk image respectively to obtain saliency features, and determine the interest regions in the third shrunk image and the fourth shrunk image respectively based on the saliency features; after that, execute step S605;

[0145] Step S605, determine the interest regions in the third sub-image and the fourth sub-image respectively based on the interest regions in the third shrunk image and the fourth shrunk image; after that, execute step S606

[0146] Step S606, generate supplementary interest regions for the third sub-image and the fourth sub-image respectively based on the distribution of the interest regions in the third sub-image and the fourth sub-image; after that, execute step S607;

[0147] Step S607, calculate the image features of the interest regions and the supplementary interest regions in the third sub-image and the fourth sub-image; after that, execute step S608;

[0148] Step S608: Perform image feature matching on the regions of interest and supplementary regions of interest in the third sub-image and the fourth sub-image to obtain the coordinate information of multiple pixel points with image feature matching relationships; then, perform Step S609;

[0149] Step S609: Based on the coordinate information of multiple pixel points with image feature matching relationships, establish an affine transformation matrix; then, perform Step S610;

[0150] Step S610: Based on the affine transformation matrix, transform the pixel point coordinates of the third image into the pixel point coordinate system of the fourth image to obtain the spliced target image.

[0151] As can be seen from the above embodiments of the present disclosure, for the image processing method provided by the present disclosure, on the one hand, compared with directly performing feature calculation on the original image in the related art, the present disclosure does not need to perform feature calculation on each pixel point in the original image, but performs image feature calculation on the regions of interest with significant features, so the amount of image feature calculation is greatly reduced; on the other hand, compared with performing image feature calculation based on the reduced image in the related art, the present disclosure calculates the image features of partial images within the regions of interest on the basis of the original resolution of the images to be spliced, which can retain the original features of the images to be spliced to the greatest extent, thereby improving the image splicing quality; on the further hand, after determining the regions of interest based on the significant features, the present disclosure also generates supplementary regions of interest in the sparse regions, so as to ensure that the image features used for image feature matching calculation can be evenly distributed on the overlapping regions, which is beneficial to establishing a more accurate affine transformation matrix and increasing the robustness of the algorithm for generating higher-quality frequency images in the subsequent process.

[0152] Based on the foregoing embodiments, the present disclosure provides an image processing apparatus. The apparatus includes each unit included and each module included in each unit, and can be implemented by a processor in an electronic device; of course, it can also be implemented by a specific logic circuit; during the implementation process, the processor can be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0153] Figure 7 FIG. is a schematic structural diagram of a composition of an image processing apparatus provided by the present disclosure. As Figure 7 shown, the image processing apparatus 700 includes: a first determination module 710 and a splicing module 720, wherein:

[0154] A first determination module 710 determines at least one first region of interest in the first sub-image and the second sub-image respectively, and determines the image features of at least two first regions of interest; the first sub-image and the second sub-image respectively represent the overlapping regions of the first image and the second image with each other.

[0155] A stitching module 720 stitches the first image and the second image into a target image based on the image feature matching relationship of at least two first regions of interest in the first sub-image and the second sub-image.

[0156] In some embodiments, the first determination module 710 is configured to:

[0157] Perform a reduction process on the first sub-image and the second sub-image respectively to obtain corresponding first reduced images and second reduced images;

[0158] Determine at least one second region of interest in the first reduced image and the second reduced image respectively;

[0159] Based on the second region of interest, determine at least one first region of interest in the first sub-image and the second sub-image respectively.

[0160] In some embodiments, the first determination module 710 is configured to:

[0161] Perform a saliency detection on the first reduced image and the second reduced image to obtain the saliency features in the first reduced image and the second reduced image;

[0162] Based on the saliency features, determine at least one second region of interest in the first reduced image and the second reduced image respectively.

[0163] In some embodiments, the first determination module 710 is configured to:

[0164] Use a dilation algorithm to perform aggregation on the saliency features in the first reduced image and the second reduced image to obtain the aggregated saliency features;

[0165] Based on the aggregated saliency features, determine at least one second region of interest in the first reduced image and the second reduced image respectively.

[0166] In some embodiments, the apparatus 700 further includes a second determination module; the second determination module is configured to:

[0167] Based on the first region of interest, determine at least one third region of interest in the first sub-image and the second sub-image respectively;

[0168] The splicing module 720 is configured to splice the first image and the second image into a target image based on the image feature matching relationship between at least two first interest regions and at least two third interest regions in the first sub-image and the second sub-image.

[0169] In some embodiments, each of the first sub-image and the second sub-image includes one first interest region; the third interest region includes a fourth interest region;

[0170] The second determination module is configured to determine at least one fourth interest region in each of the first sub-image and the second sub-image based on a preset first spacing; wherein, the shape and / or area of the fourth interest region is determined based on the corresponding first interest region.

[0171] In some embodiments, each of the first sub-image and the second sub-image includes at least two first interest regions; the third interest region includes a fifth interest region;

[0172] The second determination module is configured to determine at least one fifth interest region in the sparse regions of each of the first sub-image and the second sub-image based on a second spacing; wherein,

[0173] The second spacing represents an average value of distances between multiple adjacent first interest regions;

[0174] The sparse region represents a region between adjacent first interest regions with a distance greater than the average value, or a region between the image boundary with a distance greater than the average value and the closest first interest region;

[0175] The shape and / or area of the fifth interest region is determined based on the corresponding at least two first interest regions.

[0176] In some embodiments, the splicing module 720 is configured to:

[0177] Determine coordinate information of at least three pixel points having an image feature matching relationship in the first sub-image and the second sub-image based on the image features of the first interest region and the third interest region;

[0178] Establish an affine transformation matrix based on the coordinate information;

[0179] Perform splicing processing on the first image and the second image based on the affine transformation matrix to obtain the target image.

[0180] The description of the above device embodiments is similar to that of the above method embodiments and has similar beneficial effects to those of the method embodiments. In some embodiments, the functions or modules included in the device provided by the present disclosure can be used to execute the methods described in the above method embodiments. For technical details not disclosed in the device embodiments of the present disclosure, please refer to the description of the method embodiments of the present disclosure for understanding.

[0181] Based on the foregoing embodiments, the present disclosure also provides an electronic device. As Figure 8 shown, the electronic device 800 includes a processing unit 810 and a communication unit 820; wherein,

[0182] The communication unit 820 is configured to obtain a first image and a second image from an image acquisition device;

[0183] The processing unit 810 is configured to respectively determine at least one first region of interest in the first sub-image and the second sub-image, and determine the image features of at least two first regions of interest; the first sub-image and the second sub-image respectively represent the overlapping regions of the first image and the second image with each other;

[0184] Based on the image feature matching relationship of at least two first regions of interest in the first sub-image and the second sub-image, the first image and the second image are stitched into a target image.

[0185] The description of the above device embodiments is similar to that of the above method embodiments and has similar beneficial effects to those of the method embodiments. In some embodiments, the functions or units included in the device provided by the present disclosure can be used to execute the methods described in the above method embodiments. For technical details not disclosed in the device embodiments of the present disclosure, please refer to the description of the method embodiments of the present disclosure for understanding.

[0186] It should be noted that in the embodiments of the present disclosure, if the above image processing method is implemented in the form of software function modules and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present disclosure essentially or the part that contributes to the related technology can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the various embodiments of the present disclosure. The foregoing storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disc that can store program codes. In this way, the embodiments of the present disclosure are not limited to any specific hardware, software, or firmware, or any combination of hardware, software, and firmware.

[0187] An embodiment of the present disclosure provides a computer device, including a memory and a processor. The memory stores a computer program that can run on the processor, and when the processor executes the program, some or all of the steps in the above method are implemented.

[0188] An embodiment of the present disclosure provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, some or all of the steps in the above method are implemented. The computer-readable storage medium can be transient or non-transient.

[0189] An embodiment of the present disclosure provides a computer program, including computer-readable code. When the computer-readable code runs in a computer device, the processor in the computer device executes to implement some or all of the steps in the above method.

[0190] An embodiment of the present disclosure provides a computer program product. The computer program product includes a non-transient computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, some or all of the steps in the above method are implemented. The computer program product can be specifically implemented by means of hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium. In other embodiments, the computer program product is specifically embodied as a software product, such as a Software Development Kit (SDK), etc.

[0191] It should be noted here that the descriptions of the above embodiments tend to emphasize the differences between the embodiments, and their similarities can be referred to each other. The descriptions of the above device, storage medium, computer program, and computer program product embodiments are similar to the descriptions of the above method embodiments and have similar beneficial effects to the method embodiments. For the technical details not disclosed in the embodiments of the device, storage medium, computer program, and computer program product of the present disclosure, please refer to the descriptions of the method embodiments of the present disclosure for understanding.

[0192] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present disclosure. Therefore, the appearances of "in one embodiment" or "in an embodiment" throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present disclosure, the magnitudes of the serial numbers of the above steps / processes do not mean the sequence of execution. The execution sequence of each step / process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present disclosure. The serial numbers of the embodiments of the present disclosure above are only for description and do not represent the advantages or disadvantages of the embodiments.

[0193] It should be noted that in this article, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article or device including the element.

[0194] In several embodiments provided by the present disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the couplings, direct couplings, or communication connections between the components shown or discussed with each other can be through some interfaces. The indirect couplings or communication connections of devices or units can be electrical, mechanical or other forms.

[0195] The units described as separate components above may or may not be physically separated, and the components shown as units may or may not be physical units; they can be located in one place or distributed to multiple network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0196] In addition, each functional unit in the embodiments of the present disclosure can be all integrated in a processing unit, or each unit can be separately taken as a unit, or two or more units can be integrated in one unit; the above integrated unit can be implemented in the form of hardware, or in the form of a hardware plus software functional unit.

[0197] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps including those of the above method embodiments; and the foregoing storage medium includes: various media such as removable storage devices, read-only memory (ROM), magnetic disks, or optical discs that can store program codes.

[0198] Alternatively, if the above-mentioned integrated unit is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present disclosure, in essence or the part that contributes to the related art, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the various embodiments of the present disclosure. And the foregoing storage medium includes: various media such as removable storage devices, ROM, magnetic disks, or optical discs that can store program codes.

[0199] As described above, only the embodiments of the present disclosure are provided, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present disclosure can easily think of changes or substitutions, which should all be covered within the protection scope of the present disclosure.

Claims

1. An image processing method, comprising: Determine at least one first region of interest in the first sub-image and the second sub-image respectively, and determine image features of at least two first regions of interest; The first sub-image and the second sub-image represent overlapping areas of the first image and the second image respectively; Based on the image feature matching relationship between at least two first regions of interest in the first sub-image and the second sub-image, the first image and the second image are spliced ​​into a target image.

2. The method according to claim 1, wherein the step of respectively determining at least one first region of interest in the first sub-image and the second sub-image comprises: Reducing the first sub-image and the second sub-image respectively to obtain a corresponding first reduced image and a second reduced image; determining at least one second region of interest in the first reduced image and the second reduced image respectively; Based on the second region of interest, at least one first region of interest in the first sub-image and the second sub-image is determined respectively.

3. The method according to claim 2, wherein the determining of at least one second region of interest in the first reduced image and the second reduced image comprises: Performing saliency detection on the first reduced image and the second reduced image to obtain saliency features in the first reduced image and the second reduced image; At least one second region of interest in the first reduced image and the second reduced image is determined respectively based on the salient features.

4. The method according to claim 3, wherein: The determining, based on the salient features, at least one second region of interest in the first reduced image and the second reduced image, respectively, comprises: Aggregating the salient features in the first reduced image and the second reduced image using a dilation algorithm to obtain aggregated salient features; At least one second region of interest in the first reduced image and the second reduced image is determined based on the aggregated salient features.

5. The method according to any one of claims 1 to 4, further comprising: Based on the first region of interest, determining at least one third region of interest in the first sub-image and the second sub-image respectively; The step of stitching the first image and the second image into a target image based on the image feature matching relationship between at least two first interest regions in the first sub-image and the second sub-image includes: Based on the image feature matching relationship between at least two first interest regions and at least two third interest regions in the first sub-image and the second sub-image, the first image and the second image are spliced ​​into a target image.

6. The method according to claim 5, wherein: The first sub-image and the second sub-image each include a first region of interest; the third region of interest includes a fourth region of interest; The determining, based on the first region of interest, at least one third region of interest in the first sub-image and the second sub-image, respectively, comprises: Based on a preset first distance, at least one fourth region of interest is determined in the first sub-image and the second sub-image respectively; wherein, The shape and / or area of ​​the fourth region of interest is determined based on the corresponding first region of interest.

7. The method according to claim 5, wherein: The first sub-image and the second sub-image respectively include at least two first regions of interest; the third region of interest includes a fifth region of interest; The determining, based on the first region of interest, at least one third region of interest in the first sub-image and the second sub-image, respectively, comprises: Based on the second spacing, at least one fifth region of interest is determined in the sparse regions in the first sub-image and the second sub-image respectively; wherein, The second spacing represents an average value of distances between a plurality of adjacent first regions of interest; The sparse region represents a region between adjacent first regions of interest whose distance is greater than the average value, or a region between an image boundary and the closest first region of interest whose distance is greater than the average value; The shape and / or area of ​​the fifth region of interest is determined based on the corresponding at least two first regions of interest.

8. The method according to claim 5, wherein: The step of stitching the first image and the second image into a target image based on the image feature matching relationship between at least two first interest regions and at least two third interest regions in the first sub-image and the second sub-image includes: Based on the image features of the first region of interest and the third region of interest, determining coordinate information of at least three pixel points having an image feature matching relationship in the first sub-image and the second sub-image; Based on the coordinate information, establishing an affine transformation matrix; Based on the affine transformation matrix, a stitching process is performed on the first image and the second image to obtain the target image.

9. An image processing device, comprising: a determination module, for determining at least one first region of interest in the first sub-image and the second sub-image respectively, and determining image features of at least two first regions of interest; The first sub-image and the second sub-image represent overlapping areas of the first image and the second image respectively; A stitching module stitches the first image and the second image into a target image based on a matching relationship between image features of at least two first interest regions in the first sub-image and the second sub-image.

10. An electronic device, comprising a processing unit and a communication unit; wherein: The communication unit is used to acquire the first image and the second image from the image acquisition device; The processing unit is used to determine at least one first region of interest in the first sub-image and the second sub-image, respectively, and to determine image features of at least two first regions of interest; the first sub-image and the second sub-image respectively represent regions in the first image and the second image that overlap with each other; Based on the image feature matching relationship between at least two first regions of interest in the first sub-image and the second sub-image, the first image and the second image are spliced ​​into a target image.

Images