Image splicing method and device, electronic equipment and storage medium

Abstract

The invention relates to an image splicing method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining the position information of a preset region and a current spliced image in a process of carrying out image collection based on a preset collection sequence; determining a current local area based on the collected area; acquiring a current area image based on an image acquisition device; determining current area position information and historical area position information from the preset area position information; determining a target fusion sub-region based on the current region position information and the historical region position information; based on the current spliced image, performing image fusion processing on the target fusion sub-region in the current region image to obtain a fused current region image; performing splicing processing on the fused current region image and the current spliced image to obtain a target spliced image; and under the condition that the current local area and the acquired area are a plurality of preset local areas, taking the target spliced image as a target object global image.

Description

Technical Field

[0001] The present disclosure relates to the technical field of image stitching, and in particular to an image stitching method, device, electronic device and storage medium. Background Art

[0002] A microscope is an instrument that consists of one lens or a combination of several lenses, and is mainly used to magnify tiny objects so that they can be seen by the naked eye. However, the field of view of a microscope shrinks as the magnification increases, and it is impossible to observe the entire sample at once. Under a low-power microscope, although the field of view is larger, the clarity is poor; under a high-power microscope, although high-resolution local sample information can be obtained, it is difficult to obtain overall sample information. Therefore, by using image stitching technology, by stitching multiple high-definition images, a high-resolution image of the entire sample can be obtained.

[0003] In practical applications, due to the installation deviation of the camera, it is difficult to keep the xy direction seen by the camera completely consistent with the xy direction of the motion platform, which results in that the image captured by the camera cannot be directly stitched according to the running direction of the motion platform. A conventional stitching method is to first stitch all the horizontal single images together to form a stepped image, then take out the maximum inscribed rectangle from the stepped image as the long strip image required for subsequent stitching, and finally stitch all the long strip images together along the vertical direction, and select the maximum inscribed rectangle from the vertical stepped image as the final result. However, the above method has the following problems: it is impossible to shoot and stitch each single image at the same time, because when forming a long strip image, it is necessary to obtain a whole stepped image before knowing how to obtain the maximum inscribed rectangle, and when obtaining the final result, it is also necessary to obtain all the long strip images to intercept the maximum inscribed rectangle from the vertical stepped image. The maximum inscribed rectangle needs to be selected every time, and the final result loses a lot of edge information. When a large number of image stitching tasks need to be processed, the cropped part of the stepped image exceeds the repetition ratio between the individual images, and there is no overlap between the long strip images, so it is impossible to perform vertical stitching. Therefore, the stitching efficiency of the above method is low and the stitching effect is poor. Summary of the invention

[0004] In view of the above-mentioned technical problems, the present disclosure proposes an image stitching method, device, electronic device and storage medium.

[0005] According to one aspect of an embodiment of the present disclosure, there is provided an image stitching method, the method comprising:

[0006] In the process of performing image acquisition for multiple preset local areas corresponding to the object to be acquired based on a preset acquisition order, the preset area position information of each of the multiple preset local areas and the current stitched image are obtained; the current stitched image is obtained by stitching the regional images corresponding to the acquired areas; the preset area position information of any preset local area represents the position of any preset local area in the global image of the target object corresponding to the object to be acquired; the preset area position information of any preset local area is determined in advance based on the acquisition offset information of the image acquisition device;

[0007] Based on the captured area, determine a current local area from the multiple preset local areas; the current local area is the local area currently captured by the image capture device in the object to be captured;

[0008] Based on the image acquisition device, obtaining a current area image corresponding to the current local area;

[0009] Determine, from the preset area location information of each of the plurality of preset local areas, the current area location information corresponding to the current local area and the historical area location information corresponding to the collected area;

[0010] Based on the current region position information and the historical region position information, determining a target fusion sub-region corresponding to the current region image; the target fusion sub-region is an overlapping region between the current stitched image and the current region image;

[0011] Based on the current stitched image, performing image fusion processing on the target fusion sub-region in the current region image to obtain a fused current region image;

[0012] Performing stitching processing on the fused current region image and the current stitched image to obtain a target stitched image corresponding to the current local region;

[0013] In a case where the current local area and the acquired area are the multiple preset local areas, the target stitched image is used as the global image of the target object.

[0014] Optionally, the method further includes:

[0015] In a case where the current local area and the acquired area are partial areas of the plurality of preset local areas, updating the current stitched image based on the target stitched image;

[0016] Based on the current local area, updating the acquired area;

[0017] Based on the updated current stitched image and the updated acquired area, return to the step of determining the current local area from the multiple preset local areas based on the acquired area.

[0018] Optionally, performing image fusion processing on the target fusion sub-region in the current region image based on the current stitched image to obtain a fused current region image includes:

[0019] Determine at least one associated region image from the current stitched image; any associated region image is a region image that horizontally overlaps with the current region image or a region image that vertically overlaps with the current region image among the multiple region images corresponding to the current stitched image;

[0020] Based on the preset area position information corresponding to each associated area image and the current area position information, determining the associated fusion sub-area corresponding to each associated area image; the associated fusion sub-area corresponding to each associated area image represents the overlapping area between each associated area image and the current area image;

[0021] Based on each associated region image, image fusion processing is performed on the associated fusion sub-regions in the current region image to obtain the fused current region image.

[0022] Optionally, performing image fusion processing on the associated fusion sub-regions in the current region image based on each associated region image to obtain the fused current region image includes:

[0023] Determine a target associated region image from the at least one associated region image; the target associated region image is any associated region image among the at least one associated region image that has not been subjected to image fusion processing;

[0024] Based on the target associated region image, performing image fusion processing on the target sub-region in the current region image to obtain a first fused region image; the target sub-region is an associated fused sub-region corresponding to the target associated region image;

[0025] Based on the first fused region image, the current region image is updated, and based on the updated current region image, the steps of determining a target associated region image from the at least one associated region image and updating the current region image based on the first fused region image are repeated until the fused region image is the at least one associated region image; the fused region image is the currently fused associated region image among the at least one associated region image;

[0026] The updated current region image obtained when the fused region image is the at least one associated region image is used as the fused current region image.

[0027] Optionally, performing image fusion processing on the target sub-region in the current region image based on the target associated region image to obtain a first fused region image includes:

[0028] Based on the preset area position information corresponding to the target associated area image and the current area position information, determining a target gradient fusion direction corresponding to the target associated area image from a plurality of preset gradient fusion directions;

[0029] Based on the target gradient fusion direction and the target associated region image, a linear gradient fusion process is performed on the target sub-region in the current region image to obtain the first fusion region image.

[0030] Optionally, the preset acquisition order is an order of sequentially acquiring local area images of preset local areas in each column; the method further includes:

[0031] Acquire preset area location information of a next local area corresponding to the current local area;

[0032] In the case where the current local area is located at a non-first column position among the multiple preset local areas, the stitching process is performed on the fused current area image and the current stitched image to obtain a target stitched image corresponding to the current local area, including:

[0033] Based on the preset area position information of the next local area and the current area position information, determining a first sub-area corresponding to the current local area; the first sub-area is an overlapping area between the current local area and the next local area;

[0034] Based on the first sub-region, determining a second sub-region corresponding to the current local region; the second sub-region is a region of the current local region excluding the first sub-region;

[0035] Based on the current region position information, the current stitched image and the sub-region image corresponding to the second sub-region in the fused current region image are stitched together to obtain the target stitched image.

[0036] Optionally, the method further includes:

[0037] Acquire a previous region image; the previous region image is a region image of a previous local region corresponding to the current local region;

[0038] In the case where the at least one associated region image is two associated region images and the preset local regions corresponding to the two associated region images include the previous local region, performing image fusion processing on the associated fusion sub-regions in the current region image based on each associated region image to obtain the fused current region image includes:

[0039] Based on the previous regional image, performing image fusion processing on the associated fusion sub-region corresponding to the previous regional image in the current regional image to obtain a second fusion regional image;

[0040] Based on the associated area image corresponding to the other associated area in the two associated area images, image fusion processing is performed on the associated fusion sub-area corresponding to the other associated area in the second fusion area image to obtain the fused current area image; the other associated area is the area other than the previous local area in the preset local areas corresponding to each of the two associated area images.

[0041] Optionally, the performing stitching processing on the fused current region image and the current stitched image to obtain a target stitched image corresponding to the current local region includes:

[0042] Based on the current region position information, the fused current region image is pasted into a preset canvas to obtain the target stitched image, and the preset canvas is a canvas on which the current stitched image has been pasted.

[0043] Optionally, when the current local area and the acquired area are the multiple preset local areas, after taking the target stitched image as the global image of the target object, the method further includes:

[0044] Obtaining an edge processing dimension corresponding to the global image of the target object;

[0045] Based on the edge processing dimension, determining the cropping range position information corresponding to the global image of the target object;

[0046] An image corresponding to the cropping range position information is cropped from the preset canvas to obtain a cropped global image corresponding to the object to be collected.

[0047] According to another aspect of an embodiment of the present disclosure, there is provided an image stitching device, the device comprising:

[0048] A data acquisition module, used for acquiring preset region position information of each of the plurality of preset local regions and a currently stitched image during image acquisition for the plurality of preset local regions corresponding to the object to be acquired based on a preset acquisition sequence; the currently stitched image is obtained by stitching together the regional images corresponding to the acquired regions; the preset region position information of any preset local region represents the position of the any preset local region in the global image of the target object corresponding to the object to be acquired;

[0049] A current region determination module, configured to determine a current local region from among the plurality of preset local regions based on the captured region; the current local region is a local region of the object to be captured that is currently captured by the image capture device;

[0050] A current image acquisition module, used for acquiring a current region image corresponding to the current local region based on the image acquisition device;

[0051] A location determination module, configured to determine, from the preset area location information of each of the plurality of preset local areas, the current area location information corresponding to the current local area and the historical area location information corresponding to the collected area;

[0052] A sub-region determination module, used to determine a target fusion sub-region corresponding to the current region image based on the current region position information and the historical region position information; the target fusion sub-region is an overlapping region between the current stitched image and the current region image;

[0053] An image fusion module, configured to perform image fusion processing on the target fusion sub-region in the current region image based on the current stitched image, to obtain a fused current region image;

[0054] A stitching processing module, used for performing stitching processing on the fused current region image and the current stitched image to obtain a target stitching image corresponding to the current local region;

[0055] A global image generation module is used to use the target stitching image as the target object global image when the current local area and the acquired area are the multiple preset local areas.

[0056] According to another aspect of an embodiment of the present disclosure, an electronic device is provided, including: a processor; and a memory for storing a computer program; wherein the processor is configured to execute the computer program to implement the above-mentioned image stitching method.

[0057] According to another aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When a computer program in the storage medium is executed by a processor of an electronic device, the electronic device is enabled to perform the above-mentioned image stitching method.

[0058] According to another aspect of an embodiment of the present disclosure, a computer program product including a computer program is provided. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned image stitching method.

[0059] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0060] By acquiring the preset area position information of each of the multiple preset local areas and the current stitched image during the image acquisition process for the multiple preset local areas corresponding to the object to be acquired based on the preset acquisition order, wherein the current stitched image is obtained by stitching the regional images corresponding to the acquired areas, the preset area position information of any preset local area represents the position of any preset local area in the global image of the target object corresponding to the object to be acquired, and the preset area position information of any preset local area is determined in advance based on the acquisition offset information of the image acquisition device, so that the acquisition of the preset area position information of the multiple preset local areas determined in advance and the acquisition of the current stitched image can be realized, and then the current local area is determined from the multiple preset local areas in combination with the acquired areas, so that the preset local area currently acquired in the acquisition process can be quickly determined, and then, in combination with the image acquisition device, the current area image corresponding to the current local area is acquired, so that the acquisition of the regional image of the current local area can be realized, and then, from the preset area position information of each of the multiple preset local areas, the current area position information corresponding to the current local area and the historical area position information corresponding to the acquired area can be determined, so that The rapid positioning of the current local area and the acquired area is achieved. Then, the target fusion sub-area corresponding to the current area image is determined by combining the current area position information and the historical area position information. The target fusion sub-area is the overlapping area between the current stitched image and the current area image. The rapid determination of the area to be fused in the current area image can be achieved, thereby improving the image stitching efficiency. Then, based on the current stitched image, the target fusion sub-area in the current area image is subjected to image fusion processing to obtain the fused current area image. The fusion of the overlapping areas in the current area image can be achieved, thereby eliminating the influence of uneven illumination, eliminating image stitching traces, and improving the image stitching effect. Then, the fused current area image and the current stitched image are stitched to obtain the target stitching image corresponding to the current local area. The stitching of the current area image can be achieved, thereby achieving the effect of stitching while shooting, and further improving the image stitching efficiency. When the current local area and the acquired area are multiple preset local areas, the target stitching image is used as the global image of the target object, which can greatly improve the image stitching efficiency, eliminate image stitching traces, and improve the image stitching effect.

[0061] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0063] Figure 1 is a structural block diagram of an image generating device according to an exemplary embodiment;

[0064] Figure 2 is a flow chart of an image stitching method according to an exemplary embodiment;

[0065] Figure 3 is a schematic diagram showing a preset canvas and a plurality of preset local areas according to an exemplary embodiment;

[0066] Fig. 4(a)(b) is a schematic diagram showing vertical overlap between regional images according to an exemplary embodiment;

[0067] Fig. 5(a)(b) is a schematic diagram showing horizontal overlap between regional images according to an exemplary embodiment;

[0068] Figure 6 is a schematic diagram showing a cropping effect corresponding to a first edge processing dimension according to an exemplary embodiment;

[0069] Figure 7 is a schematic diagram showing a cropping effect corresponding to a second edge processing dimension according to an exemplary embodiment;

[0070] Figure 8 is a schematic diagram of a regional image fusion and stitching process corresponding to multiple preset local areas according to an exemplary embodiment;

[0071] Fig. 9 is a schematic diagram of another process of regional image fusion and stitching corresponding to multiple preset local areas according to an exemplary embodiment;

[0072] Fig.10 is a schematic diagram of another process of regional image fusion and stitching corresponding to multiple preset local areas according to an exemplary embodiment;

[0073] Fig.11 is a schematic diagram of another process of regional image fusion and stitching corresponding to multiple preset local areas according to an exemplary embodiment;

[0074] Fig.12 is a block diagram of an image stitching device according to an exemplary embodiment;

[0075] Fig.13 is a block diagram of an electronic device for stitching to obtain a global image of an object to be captured according to an exemplary embodiment;

[0076] Fig.14It is a block diagram of another electronic device for stitching to obtain a global image of an object to be captured according to an exemplary embodiment. DETAILED DESCRIPTION

[0077] Various exemplary embodiments, features and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0078] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0079] In addition, in order to better illustrate the present application, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present application can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present application.

[0080] See also Figure 1 , Figure 1 is a structural block diagram of an image generating device according to an exemplary embodiment. Specifically, Figure 1 As shown, the image generation device may include a processor, an image acquisition module and a driving module. The processor may be used to implement the above-mentioned image stitching method. The image acquisition module may be used to perform regional image acquisition on any preset local area. The driving module may be used to control the movement of the image acquisition module so that the image acquisition module moves to the acquisition position corresponding to each preset local area in sequence based on the preset acquisition sequence.

[0081] Specifically, Figure 2 FIG. 1 is a flow chart of an image stitching method according to an exemplary embodiment. Figure 2 As shown, the image stitching method can be used in electronic devices such as terminals or servers. In this embodiment, taking the application of the image stitching method to the above-mentioned image generating device as an example, the following steps may be specifically included:

[0082] S201: in a process of performing image acquisition for a plurality of preset local areas corresponding to an object to be acquired based on a preset acquisition sequence, obtaining preset area position information of each of the plurality of preset local areas and a currently stitched image.

[0083] In a specific embodiment, the object to be collected may refer to an object that currently needs to be spliced ​​to obtain a global image. The object to be collected may include a sample to be observed, etc.

[0084] In a specific embodiment, the multiple preset local areas may refer to the areas corresponding to the multiple local images required to acquire the global image of the object to be acquired. The range of any preset local area may correspond to the shooting range of the image acquisition module. It is understandable that when the image acquisition module is aligned with any preset local area, the image acquisition module can acquire the regional image of any of the above preset local areas; the global image of the object to be acquired can be obtained by splicing the regional images corresponding to the above multiple preset local areas.

[0085] In a specific embodiment, the preset acquisition sequence can be used to instruct the driving module to control the moving trajectory of the image acquisition module. The preset acquisition sequence can include a column-by-column acquisition sequence or a row-by-row acquisition sequence. Among them, the column-by-column acquisition sequence can be used to instruct the image acquisition module to perform local area image acquisition on the preset local areas in each column in sequence according to the order of columns; the row-by-row acquisition sequence can be used to instruct the image acquisition module to perform local area image acquisition on the preset local areas in each row in sequence according to the order of rows. Exemplarily, taking the case where the object to be acquired contains 3*3 preset local areas, and the preset acquisition sequence is a column-by-column acquisition sequence, the acquisition can be performed in the order of the first column, the second column, and the third column. Further, when acquiring each column, it can be performed in the order of the preset local areas in the column, that is, from the first preset local area in each column to the last preset local area in the column.

[0086] In a specific embodiment, the preset area position information of any preset local area can represent the position of any preset local area in the global image of the target object corresponding to the object to be collected. The preset area position information of any preset local area may include the position information of the key point in any of the above preset local areas in the preset canvas. Among them, the preset canvas can be used for pasting and splicing to form a global image. Specifically, in the case where the area shape of the preset local area is a rectangle, at least one vertex in the preset local area can be used as the key point of any of the above preset local areas; exemplarily, the coordinates of the upper left corner vertex in each preset local area in the preset canvas can be used as the position information of each of the above preset local areas.

[0087] In a specific embodiment, the preset area position information of any preset local area may be determined in advance based on the acquisition offset information of the image acquisition device. The acquisition offset information of the image acquisition device may represent the offset of the actual moving direction of the image acquisition device relative to the preset moving direction during the acquisition process based on the preset acquisition sequence. The above-mentioned acquisition offset information may include the offset in the horizontal axis direction or the offset in the vertical axis direction.

[0088] In a specific embodiment, Figure 3FIG. 1 is a schematic diagram showing a preset canvas and multiple preset local areas according to an exemplary embodiment. Specifically, Figure 3 As shown, taking the case where the object to be captured contains 7*7 preset local areas as an example, the preset area position information corresponding to the first preset local area can be determined in the above preset canvas; the preset area position information corresponding to each other preset local area can be determined in combination with the capture offset information of the image capture device; based on the preset area position information corresponding to each preset local area, the regional images corresponding to each preset local area can be fused and pasted to the corresponding position to obtain the global image of the object to be captured. Among them, the width and height of the above preset canvas can be calculated by the following formula:

[0089] H=(row-1)×(1-olr y )×h+h+2×(row bias_max ×col)

[0090] W = (col-1) × (1-olr x )×w+w+2×(col bias_max ×row)

[0091] Wherein, H is the height of the preset canvas; W is the width of the preset canvas; row is the number of preset local areas contained in the above-mentioned multiple preset local areas in the row direction; col is the number of preset local areas contained in the above-mentioned multiple preset local areas in the column direction; h is the height of the regional image collected by the image acquisition module; w is the width of the regional image collected by the image acquisition module; olr y is the overlap between adjacent preset local areas in the row direction; olr x is the overlap between adjacent preset local areas in the column direction; row bias_max col is the deviation in the row direction when the overlap direction of adjacent preset local areas is in the column direction; bias_max It is the deviation in the column direction when the overlap direction of adjacent preset local areas is in the row direction.

[0092] Correspondingly, the first preset local area may be a preset local area in the first row and first column of the plurality of preset local areas; the preset area position information corresponding to the first preset local area may be determined as (start x , start y ), where start x =col bias_max *row;start y =row bias_max *col.

[0093] In a specific embodiment, the image acquisition module can be controlled in advance by the driving module to move in accordance with a preset acquisition sequence, and the offset of the image acquisition device relative to the preset moving direction can be detected during the movement; accordingly, the acquisition offset information of the image acquisition device can be determined based on the offset of the image acquisition device relative to the preset moving direction; the preset area position information of any preset local area can be determined based on the acquisition offset information of the image acquisition device and the preset acquisition sequence. Wherein, the preset moving direction can include a horizontal axis moving direction or a vertical axis moving direction. Specifically, the image acquisition module can be controlled by the driving module to move in accordance with the vertical axis moving direction, and the moving trajectory of the image acquisition module can be collected; the actual moving direction of the image acquisition module can be determined based on the collected moving trajectory; based on the comparison of the actual moving direction and the preset moving direction, the offset of the image acquisition device in the vertical axis moving direction can be determined; based on the above method, the offset of the image acquisition device in the horizontal axis moving direction can also be determined; accordingly, the offset of the image acquisition device in the vertical axis moving direction and the offset of the image acquisition device in the horizontal axis moving direction can be used as the acquisition offset information of the image acquisition device.

[0094] In a specific embodiment, the current stitched image may refer to an image that has been stitched during the acquisition process. The current stitched image may be stitched based on the regional image corresponding to the acquired area. The acquired area may refer to a local area in which image acquisition has been currently performed among multiple preset local areas. The acquired area may include at least one preset local area. Further, the current stitched image may be stitched based on all or part of the images in the regional image corresponding to the acquired area. It is understandable that, in the case where a partial image of any regional image in the acquired area can be covered by the next regional image, the partial image except the area that can be covered can be pasted into the preset canvas.

[0095] S203: Based on the acquired area, determine a current local area from a plurality of preset local areas.

[0096] In a specific embodiment, the current local area may refer to a local area in the object to be captured that is currently captured by the image capture device. The current local area may be one of the above-mentioned multiple preset local areas.

[0097] In a specific embodiment, at least one uncollected area can be determined from the above-mentioned multiple preset local areas based on the collected area; and a preset collected area can be selected from the above-mentioned at least one uncollected area as the current local area. The above-mentioned at least one uncollected area can refer to a local area of ​​the object to be collected that has not been image collected before the current moment. Specifically, a preset collected area can be randomly selected from the above-mentioned at least one uncollected area as the current local area.

[0098] In a specific embodiment, the current local area can be determined from multiple preset local areas based on the collected areas and the preset collection order. Specifically, the previous collection area can be determined based on the collected areas; the next preset collection area of ​​the previous collection area can be determined based on the preset collection order; accordingly, the next preset collection area of ​​the previous collection area can be used as the current local area. The previous collection area can refer to the preset local area that was collected most recently before the current moment.

[0099] S205: Based on the image acquisition device, a current area image corresponding to the current local area is obtained.

[0100] In a specific embodiment, the current area image may refer to an image of the current local area acquired by an image acquisition device.

[0101] In a specific embodiment, the driving module can control the image acquisition device to move to the acquisition position corresponding to the current local area; control the image acquisition device to perform image acquisition operations to obtain the current area image. The acquisition position corresponding to the current local area can refer to the position where the image acquisition device can acquire the current area image.

[0102] S207: Determine the current region position information corresponding to the current local region and the historical region position information corresponding to the collected region from the respective preset region position information of the plurality of preset local regions.

[0103] In a specific embodiment, the current area location information may refer to the preset area location information corresponding to the current local area.

[0104] In a specific embodiment, the current region position information may be determined from the preset region position information of each of the plurality of preset local regions based on the current region identification information corresponding to the current local region. The current region identification information may refer to the region identification information of the current local region. The region identification information may be used to identify each preset local region.

[0105] In a specific embodiment, the historical area location information may refer to the preset area location information corresponding to the previous local area of ​​the current local area.

[0106] In a specific embodiment, the historical area location information may be determined from the preset area location information of each of a plurality of preset local areas based on the area identification information corresponding to the previous local area.

[0107] S209: Determine the target fusion sub-region corresponding to the current region image based on the current region position information and the historical region position information.

[0108] In a specific embodiment, the target fusion sub-region may refer to a region in the current region image that needs to be fused. The target fusion sub-region may be an overlapping region between the current stitched image and the current region image.

[0109] In a specific embodiment, the target fusion sub-region may be subjected to overlapping position analysis based on the current region position information and the historical region position information to obtain overlapping region position information; accordingly, the target fusion sub-region may be determined based on the overlapping region position information.

[0110] In a specific embodiment, when the area shape of the preset local area is a rectangle, the maximum value of the horizontal coordinates in the current area location information and the historical area location information can be used as the horizontal coordinate of the upper left corner vertex of the target fused sub-area; the minimum value of the horizontal coordinates in the current area location information and the historical area location information can be used as the horizontal coordinate of the lower right corner vertex of the target fused sub-area; the maximum value of the vertical coordinates in the current area location information and the historical area location information can be used as the vertical coordinate of the upper left corner vertex of the target fused sub-area; the minimum value of the vertical coordinates in the current area location information and the historical area location information can be used as the vertical coordinate of the lower right corner vertex of the target fused sub-area. Exemplarily, assuming that the current area location information includes the upper left corner vertex coordinates (x1, y1) and the lower right corner vertex coordinates (x2, y2), and the historical area location information includes the upper left corner coordinates (x3, y3) and the lower right corner vertex coordinates (x4, y4), it can be determined that the upper left corner coordinates of the target fused sub-area are (max(x1, x3), max(y1, y3)), and the lower right corner coordinates of the target fused sub-area are (min(x2, x4), min(y2, y4)).

[0111] In a specific embodiment, based on the preset area position information of multiple preset local areas, the overlapping area between each two preset local areas can be analyzed for overlapping positions, and the overlapping area position information corresponding to each of the above two preset local areas can be obtained, and the overlapping area position information corresponding to each of the above two preset local areas can be stored in a preset memory; accordingly, during the image acquisition process, the overlapping area position information corresponding to the current area position information and the historical area position information can be searched from the above preset memory. It can be understood that the overlapping position analysis can be performed during the image acquisition process or before the image acquisition; performing it before the image acquisition can further shorten the image stitching time and further improve the image stitching efficiency.

[0112] In the above embodiment, based on the current region position information and the historical region position information, the target fusion sub-region corresponding to the current region image is determined, which can achieve rapid and accurate determination of the region to be fused in the current region image, thereby improving the image stitching efficiency.

[0113] S211: Based on the current stitched image, image fusion processing is performed on the target fusion sub-region in the current region image to obtain a fused current region image.

[0114] In a specific embodiment, the fused current region image may refer to the current region image obtained after the target fused sub-region is subjected to image fusion processing.

[0115] In a specific embodiment, performing image fusion processing on the target fusion sub-region in the current region image based on the current stitched image to obtain the fused current region image may include:

[0116] Determine at least one associated region image from the currently stitched images;

[0117] Determine the associated fusion sub-region corresponding to each associated region image based on the preset region position information and the current region position information corresponding to each associated region image;

[0118] Based on each associated region image, image fusion processing is performed on the associated fusion sub-regions in the current region image to obtain a fused current region image.

[0119] In a specific embodiment, any associated regional image may be a regional image that has horizontal overlap or vertical overlap with the current regional image among the multiple regional images corresponding to the current stitched image. The horizontally overlapping regional images may refer to a greater overlap ratio in the horizontal direction than in the vertical direction, and the overlap ratio difference is greater than a preset ratio difference; the vertically overlapping regional images may refer to a greater overlap ratio in the vertical direction than in the horizontal direction, and the overlap ratio difference is greater than a preset ratio difference.

[0120] In a specific embodiment, FIG. 4(a)(b) is a schematic diagram showing a vertical overlap between regional images according to an exemplary embodiment; FIG. 5(a)(b) is a schematic diagram showing a horizontal overlap between regional images according to an exemplary embodiment. Specifically, as shown in FIG. 4(a) and FIG. 4(b), when the associated regional image is I i，j , and the current region image is I i+1，j In the case of , there can be two different vertical overlap situations; as shown in Figure 5 (a) and Figure 5 (b), when the associated area image is I i，j , and the current region image is I i+1，j In this case, there can be two different cases of horizontal overlap.

[0121] In a specific embodiment, a region image in the current stitched image that horizontally overlaps with the current region image may be used as an associated region image, and a region image in the current stitched image that vertically overlaps with the current region image may be used as an associated region image.

[0122] In a specific embodiment, the associated fusion sub-region corresponding to each associated region image may represent an overlapping region between each associated region image and the current region image.

[0123] In a specific embodiment, based on the preset region position information and the current region position information corresponding to each associated region image, an overlapping position analysis can be performed on each associated region image to obtain the overlapping region position information of the associated fusion sub-region corresponding to each associated region image. Specifically, the overlapping position analysis process can refer to the overlapping position analysis process for the target fusion sub-region, which is not described in detail in this disclosure.

[0124] In a specific embodiment, the image fusion processing is performed on the associated fusion sub-regions in the current region image based on each associated region image to obtain the fused current region image, which may include:

[0125] Determining a target associated area image from at least one associated area image;

[0126] Based on the target associated region image, performing image fusion processing on the target sub-region in the current region image to obtain a first fused region image;

[0127] Based on the first fused region image, the current region image is updated, and based on the updated current region image, a target associated region image is repeatedly determined from at least one associated region image, until the current region image is updated based on the first fused region image, until the fused region image is at least one associated region image;

[0128] The updated current region image obtained when the fused region image is at least one associated region image is used as the fused current region image.

[0129] In a specific embodiment, the target associated region image may be any associated region image among the at least one associated region image that has not been subjected to image fusion processing.

[0130] In a specific embodiment, any associated region image that has not been subjected to image fusion processing among the at least one associated region image may be used as a target associated region image.

[0131] In a specific embodiment, the target sub-region may be an associated fusion sub-region corresponding to the target associated region image.

[0132] In a specific embodiment, the first fused region image may refer to a current region image obtained after image fusion processing is performed based on the target associated region image.

[0133] In a specific embodiment, the image fusion processing of the target sub-region in the current region image based on the target associated region image to obtain the first fused region image may include:

[0134] Based on the preset area position information and the current area position information corresponding to the target associated area image, determining the target gradient fusion direction corresponding to the target associated area image from a plurality of preset gradient fusion directions;

[0135] Based on the target gradient fusion direction and the target associated region image, a linear gradient fusion process is performed on the target sub-region in the current region image to obtain a first fused region image.

[0136] In a specific embodiment, the preset gradient fusion direction may refer to the direction in which the colors gradually blend and transition. The preset gradient fusion direction may refer to the direction of the linear gradient during the linear gradient fusion process. The preset gradient fusion direction may include a longitudinal gradient fusion direction and a transverse gradient fusion direction. The longitudinal gradient fusion direction may include a gradient direction from top to bottom and a gradient direction from bottom to top. The transverse gradient fusion direction may include a gradient direction from left to right and a gradient direction from right to left.

[0137] In a specific embodiment, the target gradient blending direction may refer to the gradient blending direction used in the linear gradient blending process for the target associated region image.

[0138] In a specific embodiment, the target position relationship corresponding to the target associated area image can be determined based on the preset area position information and the current area position information corresponding to the target associated area image; the target gradient fusion direction can be determined from a plurality of preset gradient fusion directions based on the target position relationship. Among them, the target position relationship corresponding to the target associated area image can refer to the position relationship between the target associated area image and the current area image. The target position relationship can include a horizontal arrangement position relationship or a vertical arrangement position relationship. Specifically, in the case where the target position relationship is a horizontal arrangement position relationship, the target gradient fusion direction can be determined to be a horizontal gradient fusion direction; in the case where the target position relationship is a vertical arrangement position relationship, the target gradient fusion direction can be determined to be a vertical gradient fusion direction.

[0139] In a specific embodiment, based on the target gradient fusion direction, a first fusion matrix corresponding to the target associated area image and a second fusion matrix corresponding to the current area image can be determined from multiple preset fusion matrices; the image pixel data corresponding to the target sub-area in the target associated area image can be multiplied by the first fusion matrix to obtain first image pixel data, and the image pixel data corresponding to the target sub-area in the current area image can be multiplied by the second fusion matrix to obtain second image pixel data; the first image pixel data and the second image pixel data can be superimposed to obtain third image pixel data; the image pixel data of the target sub-area in the current area image can be replaced by the third image pixel data to obtain the first fused area image.

[0140] In a specific embodiment, the number of rows of the first fusion matrix is ​​the same as the number of rows of the image pixel data corresponding to the target sub-region; the number of columns of the first fusion matrix is ​​the same as the number of columns of the image pixel data corresponding to the target sub-region; the number of rows of the second fusion matrix is ​​the same as the number of rows of the image pixel data corresponding to the target sub-region; the number of columns of the second fusion matrix is ​​the same as the number of columns of the image pixel data corresponding to the target sub-region. Exemplarily, taking the target position relationship as a vertical arrangement position relationship as an example, the preset fusion matrix used by the first fusion matrix can be shown in Table 1 below, and the preset fusion matrix used by the second fusion matrix can be shown in Table 2 below, where row' is the number of rows of the image pixel data corresponding to the target sub-region. Correspondingly, in the case where the target position relationship is a horizontal arrangement position relationship, the preset fusion matrix used by the corresponding first fusion matrix can be obtained by transposing the preset fusion matrix used by the first fusion matrix corresponding to the vertical arrangement position relationship; the preset fusion matrix used by the corresponding second fusion matrix can be obtained by transposing the preset fusion matrix used by the second fusion matrix corresponding to the vertical arrangement position relationship.

[0141] Table 1 The first fusion matrix

[0142]

[0143] Table 2 Second fusion matrix

[0144]

[0145] In a specific embodiment, the first fused region image may be used as the updated current region image to implement the update of the current region image.

[0146] In a specific embodiment, the fused region image may be a currently fused associated region image in at least one associated region image. Specifically, the fused region image may include one or more associated region images. It is understandable that after the target sub-region in any associated region image and the target sub-region in the current region image are subjected to image fusion processing, any of the above associated region images may be used as the fused region image; in the case where the fused region image is at least one of the above associated region images, it may be determined that all associated region images corresponding to the above current region image have been subjected to image fusion processing.

[0147] In a specific embodiment, when the fused regional image is at least one associated regional image, the current regional image obtained by current updating may be used as the fused current regional image.

[0148] In a specific embodiment, the above method may further include:

[0149] Get the previous area image;

[0150] Correspondingly, in the case where at least one associated region image is two associated region images and the preset local regions corresponding to the two associated region images include the previous local region, the above-mentioned performing image fusion processing on the associated fusion sub-regions in the current region image based on each associated region image to obtain the fused current region image may include:

[0151] Based on the previous region image, performing image fusion processing on the associated fusion sub-region corresponding to the previous region image in the current region image to obtain a second fusion region image;

[0152] Based on the associated region image corresponding to the other associated region of the two associated region images, image fusion processing is performed on the associated fusion sub-region corresponding to the other associated region in the second fusion region image to obtain a fused current region image.

[0153] In a specific embodiment, the previous region image may refer to the region image of the previous local region corresponding to the current local region. The above-mentioned previous region image may be an image containing all pixel point data in the above-mentioned previous local region. It is understandable that there may be at least one local region image in the current stitched image that is data of partial pixel point data in any preset local region, that is, the stitched image may be a partial region image; and the above-mentioned previous region image is an image containing all pixel point data in the above-mentioned previous local region, that is, the above-mentioned previous region image is a complete local region image.

[0154] In a specific embodiment, each time a regional image of any preset local area is acquired, the regional image of any preset local area and the regional identification information of any preset local area can be stored in a preset memory, so that the complete regional image can be read out in a subsequent fusion process. Correspondingly, when it is necessary to obtain a previous regional image, the previous regional image can be read out from the preset memory based on the regional identification information corresponding to the previous regional image.

[0155] In a specific embodiment, the second fused region image may refer to a current region image obtained after image fusion processing is performed on an associated fused sub-region corresponding to a previous region image in the current region image.

[0156] In a specific embodiment, the positional relationship between the preset area position information corresponding to the previous area image and the current area position information can be determined based on the preset area position information corresponding to the previous area image and the current area position information; accordingly, the gradient fusion direction corresponding to the previous area image can be determined from a plurality of preset gradient fusion directions based on the above positional relationship; the associated fusion sub-area corresponding to the previous area image in the current area image can be subjected to linear gradient fusion processing based on the gradient fusion direction corresponding to the above previous area image and the previous area image to obtain a second fusion area image. Specifically, for the specific linear gradient fusion processing process of the associated fusion sub-area corresponding to the previous area image in the current area image, the linear gradient fusion processing process of the target sub-area in the above current area image can be referred to, and the present disclosure will not repeat it.

[0157] In a specific embodiment, another associated region may be a region other than the previous local region in the preset local regions corresponding to the two associated region images. Exemplarily, assuming that the two associated region images corresponding to the current region image are image 1 and image 2, where image 1 is the previous region image corresponding to the current region image, then the above-mentioned another associated region is the preset local region corresponding to the above-mentioned image 2.

[0158] In a specific embodiment, the positional relationship between the preset area position information and the current area position information corresponding to another associated area can be determined based on the preset area position information and the current area position information corresponding to another associated area; accordingly, the gradient fusion direction corresponding to another associated area can be determined from a plurality of preset gradient fusion directions based on the above-mentioned positional relationship; the associated fusion sub-area corresponding to another associated area in the current area image can be linearly gradient fused based on the gradient fusion direction corresponding to the above-mentioned other associated area and the current area image to obtain the above-mentioned fused current area image. Specifically, for the specific linear gradient fusion processing of the associated fusion sub-area corresponding to another associated area in the current area image, the linear gradient fusion processing of the target sub-area in the above-mentioned current area image can be referred to, and the present disclosure will not elaborate on it.

[0159] In the above embodiment, by combining with the previous area image, image fusion processing is performed on the associated fusion sub-area corresponding to the previous area image in the current area image to obtain a second fusion area image, and then based on the associated area image corresponding to the other associated area in the two associated area images, image fusion processing is performed on the associated fusion sub-area corresponding to the other associated area in the second fusion area image to obtain a fused current area image. By performing image fusion on each complete overlapping area, a larger area can be used for gradual fusion, which will appear smoother, thereby further improving the image fusion effect and then improving the image stitching effect.

[0160] S213: performing stitching processing on the fused current region image and the current stitched image to obtain a target stitched image corresponding to the current local region.

[0161] In a specific embodiment, the target stitched image may refer to an image obtained by stitching together the fused current region image and the current stitched image.

[0162] In a specific embodiment, the above-mentioned stitching process of the fused current region image and the current stitched image to obtain the target stitched image corresponding to the current local region may include:

[0163] Based on the current area position information, the fused current area image is pasted into the preset canvas to obtain the target stitching image.

[0164] In a specific embodiment, the preset canvas may be a canvas on which the currently stitched image has been pasted. The preset canvas may be a canvas whose corresponding size is larger than the size of the global image corresponding to the object to be collected.

[0165] In a specific embodiment, the fused current region image may be pasted to a position indicated by the current region position information in a preset canvas to obtain a target stitched image.

[0166] In a specific embodiment, when the preset acquisition order is an order of sequentially acquiring local area images of preset local areas in each column, the above method may further include:

[0167] Obtaining preset area location information of the next local area corresponding to the current local area;

[0168] Correspondingly, when the current local area is located in a non-first column position among the multiple preset local areas, the above-mentioned stitching process of the fused current area image and the current stitched image to obtain the target stitched image corresponding to the current local area may include:

[0169] Determine a first sub-region corresponding to the current local region based on the preset region position information of the next local region and the current region position information;

[0170] Based on the first sub-region, determining a second sub-region corresponding to the current local region;

[0171] Based on the current region position information, the current stitched image and the sub-region image corresponding to the second sub-region in the fused current region image are stitched together to obtain a target stitched image.

[0172] In a specific embodiment, the first sub-region may be an overlapping region between the current local region and the next local region.

[0173] In a specific embodiment, based on the preset region location information and the current region location information of the next local region, the specific process of determining the first sub-region can refer to the specific process of determining the target fusion sub-region mentioned above, which will not be described in detail in this disclosure.

[0174] In a specific embodiment, the second sub-region may be a region other than the first sub-region in the current local region.

[0175] In a specific embodiment, the area except the first sub-area in the current local area may be used as the second sub-area.

[0176] In a specific embodiment, the sub-region image corresponding to the second sub-region in the fused current region image may be pasted to the position indicated by the current region position information in the preset canvas to obtain a target stitched image.

[0177] In the above embodiment, based on the preset area position information of the next local area and the current area position information, the first sub-area corresponding to the current local area is determined, based on the first sub-area, the second sub-area corresponding to the current local area is determined, and based on the current area position information, the sub-area image corresponding to the second sub-area in the current area image after fusion is stitched together to obtain a target stitched image. This can avoid occlusion of the image of the first sub-area portion of the current stitched image during the stitching of the current area image, resulting in the next area image being unable to be fused with the image of the occluded portion during the subsequent image fusion process, thereby improving the image fusion effect and further improving the image stitching effect.

[0178] S215: When the current local area and the captured area are multiple preset local areas, the target stitching image is used as the global image of the target object.

[0179] In a specific embodiment, the target object global image may refer to a global image of the object to be captured obtained by stitching.

[0180] In a specific embodiment, when the current local area and the captured area are multiple preset local areas, it can be determined that multiple preset local areas of the object to be captured have all been captured and stitched together, and accordingly, the current target stitched image can be used as the global image of the target object.

[0181] In a specific embodiment, the above method may further include:

[0182] When the current local area and the acquired area are partial areas of a plurality of preset local areas, updating the current stitched image based on the target stitched image;

[0183] Based on the current local area, update the collected area;

[0184] Based on the updated current stitched image and the updated acquired area, return to the step of determining the current local area from a plurality of preset local areas based on the acquired area.

[0185] In a specific embodiment, when the current local area and the acquired area are partial areas of multiple preset local areas, it can be determined that there is at least one preset local area in the object to be acquired that has not been acquired, then the process returns to the step of determining the current local area from multiple preset local areas based on the acquired area, and image acquisition and stitching can be performed on the remaining unacquired areas.

[0186] In a specific embodiment, when the current local area and the acquired area are partial areas of multiple preset local areas, the target stitched image can be used as the updated current stitched image, and the current local area can be used as the updated acquired area; accordingly, based on the updated current stitched image and the updated acquired area, the process can return to the step of determining the current local area from multiple preset local areas based on the acquired area to realize image acquisition and stitching of the next local area until the acquisition and stitching of all preset local areas are completed.

[0187] In a specific embodiment, when the current local area and the acquired area are multiple preset local areas, after the target stitching image is used as the global image of the target object, the above method may further include:

[0188] Obtain the edge processing dimension corresponding to the global image of the target object;

[0189] Based on the edge processing dimension, determine the cropping range position information corresponding to the global image of the target object;

[0190] An image corresponding to the cropping range position information is cropped from the preset canvas to obtain a cropped global image corresponding to the object to be collected.

[0191] In a specific embodiment, the edge processing dimension may include a first edge processing dimension and a second edge processing dimension. The first edge processing dimension may be used to indicate cropping with the minimum circumscribed rectangle of the global image of the target object. The second edge processing dimension may be used to indicate cropping with the maximum inscribed rectangle of the global image of the target object. It can be understood that by determining the cropping range position information of the first edge processing dimension and performing cropping, all edge information of the object to be collected can be retained on the basis of minimal black edges; by determining the cropping range position information of the second edge processing dimension and performing cropping, the black edges around the image can be eliminated on the basis of minimal loss of edge information.

[0192] In a specific embodiment, an edge dimension setting operation may be performed on the image generating device so that the image generating device acquires the edge processing dimension corresponding to the object to be captured.

[0193] In a specific embodiment, the cropping range position information corresponding to the target object global image may be used to indicate the cropping range for a preset canvas.

[0194] In a specific embodiment, contour extraction processing can be performed on the global image of the target object in the preset canvas to obtain image contour information corresponding to the global image of the target object; when the edge processing dimension is the first edge processing dimension, the minimum circumscribed rectangle analysis can be performed on the global image of the target object based on the image contour information corresponding to the global image of the target object, and the contour information of the minimum circumscribed rectangle corresponding to the global image of the target object can be obtained, and the contour information of the minimum circumscribed rectangle corresponding to the global image of the target object can be used as the cropping range position information corresponding to the global image of the target object; when the edge processing dimension is the second edge processing dimension, the maximum inscribed rectangle analysis can be performed on the global image of the target object based on the image contour information corresponding to the global image of the target object, and the contour information of the maximum inscribed rectangle corresponding to the global image of the target object can be obtained, and the contour information of the maximum inscribed rectangle corresponding to the global image of the target object can be used as the cropping range position information corresponding to the global image of the target object.

[0195] In a specific embodiment, the cropped global image may refer to a global image of the object to be captured obtained by cropping.

[0196] In a specific embodiment, the preset canvas may be cropped based on the above cropping range position information to obtain a cropped global image corresponding to the object to be captured.

[0197] In a specific embodiment, Figure 6 is a schematic diagram showing a cropping effect corresponding to a first edge processing dimension according to an exemplary embodiment; Figure 7 FIG. 1 is a schematic diagram showing a cropping effect corresponding to a second edge processing dimension according to an exemplary embodiment. Specifically, Figure 6 and Figure 7 As shown, when the edge processing dimension is the first edge processing dimension, the image corresponding to the cropping range position information corresponding to the first edge processing dimension is cropped from the preset canvas, and the cropped global image corresponding to the object to be collected can be obtained (i.e., as shown in Figure 6 As shown); when the edge processing dimension is the second edge processing dimension, the image corresponding to the cropping range position information corresponding to the second edge processing dimension is cropped from the preset canvas to obtain a cropped global image corresponding to the object to be collected (i.e., as shown in Figure 7shown).

[0198] In the above embodiment, by obtaining the edge processing dimension corresponding to the global image of the target object, the cropping range position information corresponding to the global image of the target object is determined based on the edge processing dimension, and the image corresponding to the cropping range position information is cropped from the preset canvas to obtain the cropped global image corresponding to the object to be collected. It is possible to flexibly choose to retain all edge information or eliminate the black edges around the image with minimal loss of edge information as needed, thereby further improving the image stitching effect.

[0199] In a specific embodiment, Figure 8 FIG. 1 is a schematic diagram showing a process of fusing and stitching regional images corresponding to multiple preset local regions according to an exemplary embodiment. Figure 8 As an example of a preset position relationship between local regions, the current region image is I i，j In the case of I, the current region image can be pasted to the preset canvas based on the corresponding current region position information. i+1，j In the case of, the target fusion sub-region (i.e., Figure 8 Based on the current stitched image, the current area image I i+1，j The target fusion sub-region is subjected to image fusion processing to obtain the fused current region image I i+1，j ; The current region image I after the above fusion i+1，j Paste it to the preset canvas according to the current area location information. i，j+1 In the case of, the target fusion sub-region (i.e., Figure 8 Based on the current stitched image, the current area image I i，j+1 The target fusion sub-region is subjected to image fusion processing to obtain the fused current region image I i，j+1 ; The fused current region image I i，j+1 The second sub-region (i.e. Figure 8 I shown in i，j+1 The sub-region images of the regions other than R9, R10, R12, R14, and R15 are pasted to the preset canvas. i+1，j+1 In the case of i，j+1 , for the associated fusion sub-region corresponding to the previous region image in the current region image (that is, Figure 8The rectangular area composed of R9, R10, R12 and R15 in the image is fused to obtain a second fused area image; based on the current stitched image, the fused sub-area in the second fused area image (i.e. Figure 8 The rectangular area composed of R5, R6, R7, R9, R10 and R11 in the figure is subjected to image fusion processing to obtain a fused current area image; in the current area image I i+1，j+1 When the last preset local area in the corresponding current column is the current area image I i+1，j+1 Paste into the preset canvas.

[0200] In a specific embodiment, Fig. 9 FIG. 1 is a schematic diagram of another process of regional image fusion and splicing corresponding to multiple preset local areas according to an exemplary embodiment. Fig. 9 As an example of another preset position relationship between local regions, in the current region image I i，j In the case of I, the current region image can be pasted to the preset canvas based on the corresponding current region position information. i+1，j In the case of, the target fusion sub-region (i.e., Fig. 9 Based on the current stitched image, the current area image I i+1，j The target fusion sub-region is subjected to image fusion processing to obtain the fused current region image I i+1，j ; The current region image I after the above fusion i+1，j Paste it to the preset canvas according to the current area location information. i，j+1 In the case of, the target fusion sub-region (i.e., Fig. 9 Based on the current stitched image, the current area image I i，j+1 The target fusion sub-region is subjected to image fusion processing to obtain the fused current region image I i，j+1 ; The fused current region image I i，j+1 The second sub-region (i.e. Fig. 9 I shown in i，j+1 The sub-region image of the area other than R10, R11, R13, and R15 in the current region is pasted to the preset canvas. i+1，j+1 In the case of i，j+1 , for the associated fusion sub-region corresponding to the previous region image in the current region image (that is, Fig. 9The rectangular area composed of R10, R11, R13, and R15 in the second fusion area image is fused to obtain a second fusion area image; based on the current stitched image, the fusion sub-area in the second fusion area image (i.e. Fig. 9 The rectangular area composed of R5, R6, R7, R10, and R11 in the figure is subjected to image fusion processing to obtain a fused current area image; in the current area image I i+1，j+1 When the last preset local area in the corresponding current column is the current area image I i+1，j+1 Paste into the preset canvas.

[0201] In a specific embodiment, Fig.10 FIG. 1 is a schematic diagram of another process of regional image fusion and splicing corresponding to multiple preset local areas according to an exemplary embodiment. Fig.10 As an example of another preset position relationship between local regions, in the current region image I i，j In the case of I, the current region image can be pasted to the preset canvas based on the corresponding current region position information. i+1，j In the case of, the target fusion sub-region (i.e., Fig.10 Based on the current stitched image, the current area image I i+1，j The target fusion sub-region is subjected to image fusion processing to obtain the fused current region image I i+1，j ; The current region image I after the above fusion i+1，j Paste it to the preset canvas according to the current area location information. i，j+1 In the case of, the target fusion sub-region (i.e., Fig.10 Based on the current stitched image, the current area image I i，j+1 The target fusion sub-region is subjected to image fusion processing to obtain the fused current region image I i，j+1 ; The fused current region image I i，j+1 The second sub-region (i.e. Fig.10 I shown in i，j+1 The sub-region images of the regions other than R5, R6, R9, R10, R12, R13, R14, and R15 are pasted to the preset canvas. i+1，j+1 In the case of i，j+1 , for the associated fusion sub-region corresponding to the previous region image in the current region image (that is, Fig.10The rectangular area composed of R9, R10, R12, R13, and R15 in the image is fused to obtain a second fused area image; based on the current stitched image, the fused sub-area in the second fused area image (i.e. Fig.10 The rectangular area composed of R7, R9, R10, R11, R12, and R13 in the image is fused to obtain a fused current area image; in the current area image I i+1，j+1 When the last preset local area in the corresponding current column is the current area image I i+1，j+1 Paste into the preset canvas.

[0202] In a specific embodiment, Fig.11 FIG. 1 is a schematic diagram of another process of regional image fusion and splicing corresponding to multiple preset local areas according to an exemplary embodiment. Fig.11 As an example of another preset position relationship between local regions, in the current region image I i，j In the case of I, the current region image can be pasted to the preset canvas based on the corresponding current region position information. i+1，j In the case of, the target fusion sub-region (i.e., Fig.11 Based on the current stitched image, the current area image I i+1，j The target fusion sub-region is subjected to image fusion processing to obtain the fused current region image I i+1，j ; The current region image I after the above fusion i+1，j Paste it to the preset canvas according to the current area location information. i，j+1 In the case of, the target fusion sub-region (i.e., Fig.11 Based on the current stitched image, the current area image I i，j+1 The target fusion sub-region is subjected to image fusion processing to obtain the fused current region image I i，j+1 ; The fused current region image I i，j+1 The second sub-region (i.e. Fig.11 I shown in i，j+1 The sub-region images of the regions other than R5, R6, R10, R11, R12, R13, and R15 are pasted to the preset canvas. i+1，j+1 In the case of i，j+1 , for the associated fusion sub-region corresponding to the previous region image in the current region image (that is, Fig.11The rectangular area composed of R10, R11, R12, R13, and R15 in the image is fused to obtain a second fused area image; based on the current stitched image, the fused sub-area in the second fused area image (i.e. Fig.11 The rectangular area composed of R7, R10, R11, R12, and R13 in the image is subjected to image fusion processing to obtain a fused current area image; in the current area image I i+1，j+1 When the last preset local area in the corresponding current column is the current area image I i+1，j+1 Paste into the preset canvas.

[0203] In the above embodiment, during the image acquisition process for multiple preset local areas corresponding to the object to be acquired based on a preset acquisition order, the preset area position information of each of the multiple preset local areas and the current stitched image are obtained, wherein the current stitched image is obtained by stitching the regional images corresponding to the acquired areas, the preset area position information of any preset local area represents the position of any preset local area in the global image of the target object corresponding to the object to be acquired, and the preset area position information of any preset local area is determined in advance based on the acquisition offset information of the image acquisition device, so that the acquisition of the preset area position information of the multiple preset local areas determined in advance and the acquisition of the current stitched image can be realized, and then the current local area is determined from the multiple preset local areas in combination with the acquired areas, so that the preset local area currently acquired in the acquisition process can be quickly determined, and then, in combination with the image acquisition device, the current regional image corresponding to the current local area is acquired, so that the acquisition of the regional image of the current local area can be realized, and then, from the preset area position information of each of the multiple preset local areas, the current regional position information corresponding to the current local area and the historical regional position information corresponding to the acquired area are determined. Information can be used to quickly locate the current local area and the acquired area. Then, the target fusion sub-area corresponding to the current area image is determined by combining the current area location information and the historical area location information. The target fusion sub-area is the overlapping area between the current stitched image and the current area image. The area to be fused in the current area image can be quickly determined, thereby improving the image stitching efficiency. Then, based on the current stitched image, the target fusion sub-area in the current area image is subjected to image fusion processing to obtain the fused current area image. The fusion of the overlapping areas in the current area image can be achieved, thereby eliminating the influence of uneven illumination, eliminating image stitching traces, and improving the image stitching effect. Then, the fused current area image and the current stitched image are stitched to obtain the target stitching image corresponding to the current local area. The stitching of the current area image can be achieved, thereby achieving the effect of stitching while shooting, and further improving the image stitching efficiency. When the current local area and the acquired area are multiple preset local areas, the target stitching image is used as the global image of the target object, which can greatly improve the image stitching efficiency, eliminate image stitching traces, and improve the image stitching effect.

[0204] Fig.12 FIG. 1 is a block diagram of an image stitching device according to an exemplary embodiment. Fig.12 As shown, the device may include:

[0205] The data acquisition module 1210 can be used to acquire the preset region position information of each of the plurality of preset local regions and the current stitched image during the image acquisition process for the plurality of preset local regions corresponding to the object to be acquired based on the preset acquisition order; the current stitched image is obtained by stitching the regional images corresponding to the acquired regions; the preset region position information of any preset local region represents the position of any preset local region in the global image of the target object corresponding to the object to be acquired;

[0206] The current region determination module 1220 may be used to determine a current local region from a plurality of preset local regions based on the captured region; the current local region is a local region currently captured by the image capture device in the object to be captured;

[0207] The current image acquisition module 1230 may be used to acquire a current region image corresponding to the current local region based on an image acquisition device;

[0208] The location determination module 1240 may be used to determine the current area location information corresponding to the current local area and the historical area location information corresponding to the collected area from the respective preset area location information of the plurality of preset local areas;

[0209] The sub-region determination module 1250 may be used to determine a target fusion sub-region corresponding to the current region image based on the current region position information and the historical region position information; the target fusion sub-region is an overlapping region between the current stitched image and the current region image;

[0210] The image fusion module 1260 may be used to perform image fusion processing on a target fusion sub-region in the current region image based on the current stitched image to obtain a fused current region image;

[0211] The stitching processing module 1270 can be used to stitch the fused current region image and the current stitched image to obtain a target stitched image corresponding to the current local region;

[0212] The global image generation module 1280 may be used to use the target stitching image as the global image of the target object when the current local area and the acquired area are multiple preset local areas.

[0213] In a specific embodiment, the above device may further include:

[0214] An image updating module may be used to update the current stitched image based on the target stitched image when the current local area and the acquired area are partial areas of a plurality of preset local areas;

[0215] The area update module can be used to update the collected area based on the current local area;

[0216] The first execution module may be configured to return to the step of determining the current local area from a plurality of preset local areas based on the acquired area based on the updated current stitched image and the updated acquired area.

[0217] In a specific embodiment, the image fusion module 1260 may include:

[0218] The associated image determination module can be used to determine at least one associated regional image from the current stitched image; any associated regional image is a regional image that has horizontal overlap or vertical overlap with the current regional image among the multiple regional images corresponding to the current stitched image;

[0219] The associated sub-region determination module can be used to determine the associated fusion sub-region corresponding to each associated region image based on the preset region position information and the current region position information corresponding to each associated region image; the associated fusion sub-region corresponding to each associated region image represents the overlapping region between each associated region image and the current region image;

[0220] The associated fusion module can be used to perform image fusion processing on the associated fusion sub-regions in the current region image based on each associated region image to obtain a fused current region image.

[0221] In a specific embodiment, the above-mentioned association fusion module may include:

[0222] The target image determination module can be used to determine a target associated region image from at least one associated region image; the target associated region image is any associated region image in the at least one associated region image that has not been subjected to image fusion processing;

[0223] The first fusion module can be used to perform image fusion processing on the target sub-region in the current region image based on the target associated region image to obtain a first fused region image; the target sub-region is the associated fusion sub-region corresponding to the target associated region image;

[0224] The second execution module may be used to update the current region image based on the first fused region image, and to repeatedly determine the target associated region image from at least one associated region image based on the updated current region image, until the current region image is updated based on the first fused region image, until the fused region image is at least one associated region image; the fused region image is the currently fused associated region image in the at least one associated region image;

[0225] The third execution module may be configured to use the updated current region image obtained when the fused region image is at least one associated region image as the fused current region image.

[0226] In a specific embodiment, the first fusion module may include:

[0227] A fusion direction determination module may be used to determine a target gradient fusion direction corresponding to the target association region image from a plurality of preset gradient fusion directions based on preset region position information corresponding to the target association region image and current region position information;

[0228] The gradient fusion module can be used to perform linear gradient fusion processing on the target sub-region in the current region image based on the target gradient fusion direction and the target associated region image to obtain a first fused region image.

[0229] In a specific embodiment, the above device may further include:

[0230] An information acquisition module can be used to obtain preset area location information of the next local area corresponding to the current local area;

[0231] Accordingly, when the current local area is located in a non-first column position among the plurality of preset local areas, the splicing processing module 1270 may include:

[0232] The first sub-region determining module may be used to determine the first sub-region corresponding to the current local region based on the preset region position information of the next local region and the current region position information; the first sub-region is the overlapping region between the current local region and the next local region;

[0233] The second sub-region determining module may be used to determine the second sub-region corresponding to the current local region based on the first sub-region; the second sub-region is the region other than the first sub-region in the current local region;

[0234] The sub-image stitching module can be used to stitch the current stitched image with the sub-region image corresponding to the second sub-region in the fused current region image based on the current region position information to obtain a target stitched image.

[0235] In a specific embodiment, the above device may further include:

[0236] The previous image acquisition module can be used to acquire the previous region image; the previous region image is the region image of the previous local region corresponding to the current local region;

[0237] Correspondingly, in the case where at least one associated region image is two associated region images and the preset local regions corresponding to the two associated region images include the previous local region, the associated fusion module may include:

[0238] The second fusion module may be used to perform image fusion processing on the associated fusion sub-region corresponding to the previous regional image in the current regional image based on the previous regional image to obtain a second fusion regional image;

[0239] The third fusion module can be used to perform image fusion processing on the associated fusion sub-region corresponding to another associated region in the second fusion region image based on the associated region image corresponding to the other associated region in the two associated region images, so as to obtain a fused current region image; the other associated region is the region other than the previous local region in the preset local regions corresponding to each of the two associated region images.

[0240] In a specific embodiment, the splicing processing module 1270 may include:

[0241] The image pasting module can be used to paste the fused current area image into a preset canvas based on the current area position information to obtain a target stitched image. The preset canvas is a canvas on which the current stitched image has been pasted.

[0242] In a specific embodiment, the above device may further include:

[0243] A dimension acquisition module can be used to obtain the edge processing dimension corresponding to the global image of the target object;

[0244] A cropping range determination module can be used to determine the cropping range position information corresponding to the global image of the target object based on the edge processing dimension;

[0245] The cropped image acquisition module can be used to crop an image corresponding to the cropping range position information from a preset canvas to obtain a cropped global image corresponding to the object to be collected.

[0246] Regarding the device in the above embodiment, the specific manner in which each module and unit performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0247] Fig.13 is a block diagram of an electronic device for stitching together a global image of an object to be captured according to an exemplary embodiment. The electronic device may be a server, and its internal structure diagram may be as shown in FIG. Fig.13As shown. The electronic device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an image stitching method is implemented.

[0248] Fig.14 is a block diagram of another electronic device for stitching to obtain a global image of an object to be collected according to an exemplary embodiment. The electronic device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig.14 As shown. The electronic device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an image stitching method is implemented. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a key, trackball or touchpad set on the housing of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0249] Those skilled in the art will understand that Fig.13 or Fig.14 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present disclosure, and does not constitute a limitation on the electronic device to which the scheme of the present disclosure is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0250] In an exemplary embodiment, an electronic device is also provided, including: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the image stitching method in the embodiment of the present disclosure.

[0251] In an exemplary embodiment, a computer-readable storage medium is further provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the image stitching method in the embodiment of the present disclosure.

[0252] In an exemplary embodiment, a computer program product including instructions is also provided. When the computer program product is run on a computer, the computer is enabled to perform the image stitching method in the embodiment of the present disclosure.

[0253] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0254] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0255] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image stitching method, characterized in that: The method comprises: In the process of performing image acquisition for multiple preset local areas corresponding to the object to be acquired based on a preset acquisition order, the preset area position information of each of the multiple preset local areas and the current stitched image are obtained; the current stitched image is obtained by stitching the regional images corresponding to the acquired areas; the preset area position information of any preset local area represents the position of any preset local area in the global image of the target object corresponding to the object to be acquired; the preset area position information of any preset local area is determined in advance based on the acquisition offset information of the image acquisition device; Based on the captured area, determine a current local area from the multiple preset local areas; the current local area is the local area currently captured by the image capture device in the object to be captured; Based on the image acquisition device, obtaining a current area image corresponding to the current local area; Determine, from the preset area location information of each of the plurality of preset local areas, the current area location information corresponding to the current local area and the historical area location information corresponding to the collected area; Based on the current region position information and the historical region position information, determining a target fusion sub-region corresponding to the current region image; the target fusion sub-region is an overlapping region between the current stitched image and the current region image; Based on the current stitched image, performing image fusion processing on the target fusion sub-region in the current region image to obtain a fused current region image; Performing stitching processing on the fused current region image and the current stitched image to obtain a target stitched image corresponding to the current local region; In a case where the current local area and the acquired area are the multiple preset local areas, the target stitched image is used as the global image of the target object.

2. The method according to claim 1, characterized in that The method further comprises: In a case where the current local area and the acquired area are partial areas of the plurality of preset local areas, updating the current stitched image based on the target stitched image; Based on the current local area, updating the acquired area; Based on the updated current stitched image and the updated acquired area, return to the step of determining the current local area from the multiple preset local areas based on the acquired area.

3. The method according to claim 1, characterized in that The step of performing image fusion processing on the target fusion sub-region in the current region image based on the current stitched image to obtain a fused current region image includes: Determine at least one associated region image from the current stitched image; any associated region image is a region image that horizontally overlaps with the current region image or a region image that vertically overlaps with the current region image among the multiple region images corresponding to the current stitched image; Based on the preset area position information corresponding to each associated area image and the current area position information, determining the associated fusion sub-area corresponding to each associated area image; the associated fusion sub-area corresponding to each associated area image represents the overlapping area between each associated area image and the current area image; Based on each associated region image, image fusion processing is performed on the associated fusion sub-regions in the current region image to obtain the fused current region image.

4. The method according to claim 3, characterized in that The step of performing image fusion processing on the associated fusion sub-regions in the current region image based on each associated region image to obtain the fused current region image includes: Determine a target associated region image from the at least one associated region image; the target associated region image is any associated region image among the at least one associated region image that has not been subjected to image fusion processing; Based on the target associated region image, performing image fusion processing on the target sub-region in the current region image to obtain a first fused region image; the target sub-region is an associated fused sub-region corresponding to the target associated region image; Based on the first fused region image, the current region image is updated, and based on the updated current region image, the steps of determining a target associated region image from the at least one associated region image and updating the current region image based on the first fused region image are repeated until the fused region image is the at least one associated region image; the fused region image is the currently fused associated region image among the at least one associated region image; The updated current region image obtained when the fused region image is the at least one associated region image is used as the fused current region image.

5. The method according to claim 4, characterized in that The step of performing image fusion processing on the target sub-region in the current region image based on the target associated region image to obtain a first fused region image includes: Based on the preset area position information corresponding to the target associated area image and the current area position information, determining a target gradient fusion direction corresponding to the target associated area image from a plurality of preset gradient fusion directions; Based on the target gradient fusion direction and the target associated region image, a linear gradient fusion process is performed on the target sub-region in the current region image to obtain the first fusion region image.

6. The method according to claim 3, characterized in that The preset acquisition sequence is the sequence of sequentially acquiring local area images of the preset local areas in each column; the method further includes: Acquire preset area location information of a next local area corresponding to the current local area; In the case where the current local area is located at a non-first column position among the multiple preset local areas, the stitching process is performed on the fused current area image and the current stitched image to obtain a target stitched image corresponding to the current local area, including: Based on the preset area position information of the next local area and the current area position information, determining a first sub-area corresponding to the current local area; the first sub-area is an overlapping area between the current local area and the next local area; Based on the first sub-region, determining a second sub-region corresponding to the current local region; the second sub-region is a region of the current local region excluding the first sub-region; Based on the current region position information, the current stitched image and the sub-region image corresponding to the second sub-region in the fused current region image are stitched together to obtain the target stitched image.

7. The method according to claim 6, characterized in that The method further comprises: Acquire a previous region image; the previous region image is a region image of a previous local region corresponding to the current local region; In the case where the at least one associated region image is two associated region images and the preset local regions corresponding to the two associated region images include the previous local region, performing image fusion processing on the associated fusion sub-regions in the current region image based on each associated region image to obtain the fused current region image includes: Based on the previous regional image, performing image fusion processing on the associated fusion sub-region corresponding to the previous regional image in the current regional image to obtain a second fusion regional image; Based on the associated area image corresponding to the other associated area in the two associated area images, image fusion processing is performed on the associated fusion sub-area corresponding to the other associated area in the second fusion area image to obtain the fused current area image; the other associated area is the area other than the previous local area in the preset local areas corresponding to each of the two associated area images.

8. The method according to claim 1, characterized in that The step of performing stitching processing on the fused current region image and the current stitched image to obtain a target stitched image corresponding to the current local region includes: Based on the current region position information, the fused current region image is pasted into a preset canvas to obtain the target stitched image, and the preset canvas is a canvas on which the current stitched image has been pasted.

9. The method according to claim 8, characterized in that In the case where the current local area and the acquired area are the multiple preset local areas, after the target stitching image is used as the global image of the target object, the method further includes: Obtaining an edge processing dimension corresponding to the global image of the target object; Based on the edge processing dimension, determining the cropping range position information corresponding to the global image of the target object; An image corresponding to the cropping range position information is cropped from the preset canvas to obtain a cropped global image corresponding to the object to be collected.

10. An image stitching device, characterized in that: The device comprises: A data acquisition module, used for acquiring preset region position information of each of the plurality of preset local regions and a currently stitched image during image acquisition for the plurality of preset local regions corresponding to the object to be acquired based on a preset acquisition sequence; the currently stitched image is obtained by stitching together the regional images corresponding to the acquired regions; the preset region position information of any preset local region represents the position of the any preset local region in the global image of the target object corresponding to the object to be acquired; A current region determination module, configured to determine a current local region from among the plurality of preset local regions based on the captured region; the current local region is a local region of the object to be captured that is currently captured by the image capture device; A current image acquisition module, used for acquiring a current region image corresponding to the current local region based on the image acquisition device; A location determination module, configured to determine, from the preset area location information of each of the plurality of preset local areas, the current area location information corresponding to the current local area and the historical area location information corresponding to the collected area; A sub-region determination module, used to determine a target fusion sub-region corresponding to the current region image based on the current region position information and the historical region position information; the target fusion sub-region is an overlapping region between the current stitched image and the current region image; An image fusion module, configured to perform image fusion processing on the target fusion sub-region in the current region image based on the current stitched image, to obtain a fused current region image; A stitching processing module, used for performing stitching processing on the fused current region image and the current stitched image to obtain a target stitching image corresponding to the current local region; A global image generation module is used to use the target stitching image as the target object global image when the current local area and the acquired area are the multiple preset local areas.

11. An electronic device, characterized in that: include: processor; memory for storing computer programs; Wherein, the processor is configured to execute the computer program to implement the image stitching method described in any one of claims 1 to 9.

12. A non-volatile computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image stitching method according to any one of claims 1 to 9 is implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by an electronic device, the image stitching method according to any one of claims 1 to 9 is implemented.

Images