Video stitching method and device and electronic equipment

Abstract

The invention provides a video stitching method, a video stitching device and electronic equipment. Under the condition that K current to-be-spliced images at different visual angles are obtained, for each current to-be-spliced image, the historical color compensation coefficient is determined as the target color compensation coefficient, or the historical color compensation coefficient is determined as the target color compensation coefficient. And determining the target color compensation coefficient based on the current color compensation coefficient of the current to-be-spliced image and the historical color compensation coefficient, so that the influence of the color information of the historical to-be-spliced image on the color compensation coefficient of the current to-be-spliced image is fully considered. And further performing color correction on the current to-be-spliced image according to the target color compensation coefficient of the current to-be-spliced image to obtain a target color image corresponding to the current to-be-spliced image, and splicing the target color images corresponding to the K current to-be-spliced images to obtain a target spliced image for forming a target video. And color balance in the video stitching process is realized.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a video stitching method, device and electronic device. Background Art

[0002] In actual scenes, multiple frames of images captured by multiple scattered video acquisition devices are usually stitched together to provide a larger, wider and more complete perspective. In the process of video stitching, in order to avoid misalignment at the seams, the current images to be stitched are usually aligned. However, in the process of video stitching, due to changes in scene lighting, differences between photosensitive devices, etc., even if the images are aligned, there will still be problems such as uneven colors and unnatural image transitions in the stitching. Summary of the invention

[0003] In view of this, the present application provides a video stitching method, device and electronic device to achieve color balance in the video stitching process.

[0004] The technical solutions provided by this application are as follows: According to an embodiment of the first aspect of the present application, a video stitching method is provided, the method comprising: Obtain K current images to be stitched from different perspectives, where K is not less than 2; For each current image to be stitched, a target color compensation coefficient of the current image to be stitched is determined based on a historical color compensation coefficient of a historical image to be stitched corresponding to the current image to be stitched; wherein the historical image to be stitched and the current image to be stitched are images at the same viewing angle in the same video; wherein the historical color compensation coefficient is determined as the target color compensation coefficient, or the target color compensation coefficient is determined based on the current color compensation coefficient of the current image to be stitched and the historical color compensation coefficient; Performing color correction on the current image to be stitched according to the target color compensation coefficient of the current image to be stitched, to obtain a target color image corresponding to the current image to be stitched; The target color images corresponding to the K current images to be stitched are stitched to obtain a target stitched image; the target stitched image is used to form a target video.

[0005] Optionally, determining the historical color compensation coefficient as the target color compensation coefficient includes: If the difference between the current color distribution of each current image to be stitched and the historical color distribution of the historical image to be stitched is within a preset range, the historical color compensation coefficient of the historical image to be stitched is used as the target color compensation coefficient of the current image to be stitched; wherein the historical image to be stitched is the previous frame of the historical image to be stitched before the current image to be stitched.

[0006] Optionally, determining the target color compensation coefficient based on the current color compensation coefficient of the current image to be stitched and the historical color compensation coefficient includes: If the difference between the current color distribution of any current image to be stitched and the historical color distribution of the historical image to be stitched is not within a preset range, a weighted operation is performed on the current color compensation coefficient of the current image to be stitched and the historical color compensation coefficient of the historical image to be stitched to obtain the target color compensation coefficient; wherein the historical image to be stitched is the previous M frames of the historical image to be stitched before the current image to be stitched, and M is greater than or equal to 1.

[0007] Optionally, the K current images to be stitched include a reference image and at least one non-reference image, and the reference image is any current image to be stitched selected from the K current images to be stitched; and the current color compensation coefficient of the current image to be stitched is determined by the following method: For the reference image, obtaining a configured default color compensation coefficient, and determining the default color compensation coefficient as a current color compensation coefficient of the reference image; For each non-reference image, a current color compensation coefficient of the non-reference image relative to the reference image is determined based on the current color distribution of the non-reference image and the current color distribution of the reference image.

[0008] Optionally, the current color distribution of the non-reference image and the current color distribution of the reference image include a first channel color distribution, a second channel color distribution, and a third channel color distribution; and determining a current color compensation coefficient of the non-reference image relative to the reference image based on the current color distribution of the non-reference image and the current color distribution of the reference image includes: The current color compensation coefficient of the non-reference image corresponding to the reference image in each channel is determined according to the following formula:

[0009]

[0010]

[0011] Among them, the represents the current color compensation coefficient corresponding to the first channel of the nth non-reference image, represents the color distribution of the first channel of the nth non-reference image, represents the color distribution of the first channel of the reference image; represents the current color compensation coefficient corresponding to the second channel of the nth non-reference image, represents the color distribution of the second channel of the nth non-reference image, represents the color distribution of the second channel of the reference image; represents the current color compensation coefficient corresponding to the third channel of the nth non-reference image, represents the color distribution of the third channel of the nth non-reference image, Indicates the color distribution of the third channel of the reference image.

[0012] Optionally, the determining, for each non-reference image, a current color compensation coefficient of the non-reference image relative to the reference image based on a current color distribution of the non-reference image and a current color distribution of the reference image, includes: For each current image to be stitched among the K current images to be stitched, determine a non-motion area in the current image to be stitched according to the current image to be stitched and a historical image to be stitched corresponding to the current image to be stitched; the non-motion area refers to a background area in the current image to be stitched except for a moving object; For each non-reference image, a current color compensation coefficient of the non-reference image relative to the reference image is determined based on a current color distribution of a non-motion region in the non-reference image and a current color distribution of a non-motion region in the reference image.

[0013] Optionally, for each current image to be stitched among the K current images to be stitched, determining a non-motion area in the current image to be stitched according to the current image to be stitched and a historical image to be stitched corresponding to the current image to be stitched includes: For each current image to be stitched among the K current images to be stitched, determine a velocity vector of each pixel in the current image to be stitched according to the current image to be stitched and a historical image to be stitched corresponding to the current image to be stitched, wherein the velocity vector includes a motion speed change amount and a motion direction change amount of the pixel between acquisition moments of the historical image to be stitched and the current image to be stitched; For each pixel in the current image to be stitched, if it is determined that the change in the motion rate of the pixel is less than a first threshold, and the change in the motion direction of the pixel is less than a second threshold, the pixel is taken as a candidate pixel; A non-motion area in the current image to be stitched is determined based on a set of all candidate pixels.

[0014] According to an embodiment of the second aspect of the present application, a video splicing device is provided, the device comprising: An acquisition module, used to acquire K current images to be stitched from different perspectives, where K is not less than 2; A determination module, for determining, for each current image to be stitched, a target color compensation coefficient of the current image to be stitched based on a historical color compensation coefficient of a historical image to be stitched corresponding to the current image to be stitched; wherein the historical image to be stitched and the current image to be stitched are images at the same viewing angle in the same video; wherein the historical color compensation coefficient is determined as the target color compensation coefficient, or the target color compensation coefficient is determined based on the current color compensation coefficient of the current image to be stitched and the historical color compensation coefficient; A correction module, used for performing color correction on the current image to be stitched according to the target color compensation coefficient of the current image to be stitched, so as to obtain a target color image corresponding to the current image to be stitched; The splicing module is used to splice the target color images corresponding to the K current images to be spliced ​​to obtain a target spliced ​​image; the target spliced ​​image is used to form a target video.

[0015] Optionally, the determining module is specifically used to: If the difference between the current color distribution of each current image to be stitched and the historical color distribution of the historical image to be stitched is within a preset range, the historical color compensation coefficient of the historical image to be stitched is used as the target color compensation coefficient of the current image to be stitched; wherein the historical image to be stitched is a previous frame of the historical image to be stitched of the current image to be stitched; And / or, the determining module is specifically used for: If the difference between the current color distribution of any current image to be stitched and the historical color distribution of the historical image to be stitched is not within a preset range, a weighted operation is performed on the current color compensation coefficient of the current image to be stitched and the historical color compensation coefficient of the historical image to be stitched to obtain the target color compensation coefficient; wherein the historical image to be stitched is the previous M frames of the historical image to be stitched before the current image to be stitched, and M is greater than or equal to 1; And / or, the K current images to be stitched include a reference image and at least one non-reference image, and the reference image is any current image to be stitched selected from the K current images to be stitched; the determination module is further used for: For the reference image, obtaining a configured default color compensation coefficient, and determining the default color compensation coefficient as a current color compensation coefficient of the reference image; For each non-reference image, determining a current color compensation coefficient of the non-reference image relative to the reference image based on a current color distribution of the non-reference image and a current color distribution of the reference image; And / or, the current color distribution of the non-reference image and the current color distribution of the reference image include a first channel color distribution, a second channel color distribution, and a third channel color distribution; and the determination module is specifically configured to: The current color compensation coefficient of the non-reference image corresponding to the reference image in each channel is determined according to the following formula:

[0016]

[0017]

[0018] Among them, the represents the current color compensation coefficient corresponding to the first channel of the nth non-reference image, represents the color distribution of the first channel of the nth non-reference image, represents the color distribution of the first channel of the reference image; represents the current color compensation coefficient corresponding to the second channel of the nth non-reference image, represents the color distribution of the second channel of the nth non-reference image, represents the color distribution of the second channel of the reference image; represents the current color compensation coefficient corresponding to the third channel of the nth non-reference image, represents the color distribution of the third channel of the nth non-reference image, Indicates the color distribution of the third channel of the reference image; And / or, the determining module is specifically used for: For each current image to be stitched among the K current images to be stitched, determine a non-motion area in the current image to be stitched according to the current image to be stitched and a historical image to be stitched corresponding to the current image to be stitched; the non-motion area refers to a background area in the current image to be stitched except for a moving object; For each non-reference image, determining a current color compensation coefficient of the non-reference image relative to the reference image based on a current color distribution of a non-motion region in the non-reference image and a current color distribution of a non-motion region in the reference image; And / or, the determining module is specifically used for: For each current image to be stitched among the K current images to be stitched, determine a velocity vector of each pixel in the current image to be stitched according to the current image to be stitched and a historical image to be stitched corresponding to the current image to be stitched, wherein the velocity vector includes a motion speed change amount and a motion direction change amount of the pixel between acquisition moments of the historical image to be stitched and the current image to be stitched; For each pixel in the current image to be stitched, if it is determined that the change in the motion rate of the pixel is less than a first threshold, and the change in the motion direction of the pixel is less than a second threshold, the pixel is taken as a candidate pixel; A non-motion area in the current image to be stitched is determined based on a set of all candidate pixels.

[0019] According to an embodiment of the third aspect of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the program.

[0020] According to an embodiment of the fourth aspect of the present application, a computer program product is provided, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the method as described in the first aspect is implemented.

[0021] According to an embodiment of the fifth aspect of the present application, a machine-readable storage medium is provided, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by a processor; wherein the processor is used to execute the machine-executable instructions to implement the method described in the first aspect.

[0022] It can be seen from the above technical scheme that, in the case of obtaining K current images to be stitched of different perspectives, the present application determines, for each current image to be stitched, a target color compensation coefficient of the current image to be stitched based on a historical color compensation coefficient of a historical image to be stitched corresponding to the current image to be stitched. Specifically, the historical color compensation coefficient is determined as the target color compensation coefficient, or the target color compensation coefficient is determined based on the current color compensation coefficient of the current image to be stitched and the historical color compensation coefficient, thereby fully considering the influence of the color information of the historical images to be stitched on the color compensation coefficient of the current image to be stitched; further, the current image to be stitched is color corrected according to the target color compensation coefficient of the current image to be stitched to obtain a target color image corresponding to the current image to be stitched, and the target color images corresponding to the K current images to be stitched are stitched to obtain a target stitched image for constituting a target video, thereby achieving color balance in the video stitching process. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0024] Figure 1 A flowchart of a video stitching method provided in an embodiment of the present application; Figure 2A schematic diagram of the overall process provided for the embodiment of the present application; Figure 3 A schematic diagram of a specific process flow of the spatiotemporal color balancing stage provided in an embodiment of the present application; Figure 4 A schematic diagram of a specific process of the image alignment stage provided in an embodiment of the present application; Figure 5 A schematic diagram of a specific process flow of the transition zone processing stage provided in an embodiment of the present application; Fig. 6A This is an example diagram of spatial color balance provided in an embodiment of the present application; Figure 6B An example diagram of spatiotemporal color balance provided in an embodiment of the present application; Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application; Figure 8 A structural diagram of a video splicing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present application and to make the above-mentioned purposes, features and advantages of the embodiments of the present application more obvious and understandable, the technical solutions in the embodiments of the present application are further described in detail below in conjunction with the accompanying drawings.

[0026] In actual scenes, multiple frames of images captured by multiple scattered video acquisition devices at the same time are usually stitched together to obtain a target stitched image used to form a target video, in order to provide a larger, wider and more complete perspective. In the process of video stitching, in order to avoid misalignment at the seams, the current images to be stitched are usually aligned. However, in the process of video stitching, due to changes in scene lighting, differences between photosensitive devices, etc., even if the images are aligned, there will still be problems such as uneven color of the stitched image and unnatural image transitions.

[0027] It can be seen that in the process of video stitching, in addition to aligning images, it is also necessary to consider the color balance of the stitched image to make the image transition natural. For example, at each moment, the color of each image is balanced in the spatial dimension. The color balance in the spatial dimension means that the color of multiple images to be stitched collected at the same moment is balanced, so that the color transition of the stitched image obtained after stitching is more natural.

[0028] However, although the stitched image obtained by stitching multiple videos to be stitched that have been color-balanced in the spatial dimension is color-balanced in the current frame, it does not take into account the color jumps (between different frames) in the temporal dimension caused by changes in lighting or the passage of moving targets. That is, each frame in the stitched video stream that has been color-balanced in the spatial dimension is color-balanced individually, but there may be obvious color jumps between the previous and next frames, and there is a lack of temporal smoothness.

[0029] Please refer to Figure 1 , Figure 1 A flowchart of a video stitching method provided in an embodiment of the present application.

[0030] like Figure 1 As shown, the method may include the following steps: Step 101: Obtain K current images to be stitched from different perspectives.

[0031] In this embodiment, the K current images to be stitched refer to images captured at multiple different viewing angles at the current moment, where K is not less than 2.

[0032] As an embodiment, the K current images to be stitched may be images of the target scene at the current moment captured by multiple image acquisition devices at different viewing angles, or may be K aligned images obtained after image alignment of images of the target scene at the current moment captured by multiple image acquisition devices at different viewing angles. The present application does not impose any restrictions on this.

[0033] At this point, the description of step 101 ends, and step 102 is executed next.

[0034] Step 102 : for each current image to be stitched, determine a target color compensation coefficient of the current image to be stitched based on a historical color compensation coefficient of a historical image to be stitched corresponding to the current image to be stitched.

[0035] In this embodiment, the historical images to be stitched and the current images to be stitched are images at the same viewing angle in the same video. For example, the current images to be stitched at different times captured by the same camera device at the same viewing angle are the historical images to be stitched at the same viewing angle of the current image to be stitched.

[0036] In this embodiment, the historical image to be stitched may be one or more images, and each historical image to be stitched corresponds to a historical color compensation coefficient.

[0037] In this embodiment, the method of determining the target color compensation coefficient of the current image to be stitched based on the historical color compensation coefficient of the historical image to be stitched corresponding to the current image to be stitched includes the following two cases: The historical color compensation coefficient is determined as the target color compensation coefficient, or the target color compensation coefficient is determined based on the current color compensation coefficient of the current image to be stitched and the historical color compensation coefficient.

[0038] The above two situations are described below respectively.

[0039] (1) The historical color compensation coefficient is determined as the target color compensation coefficient.

[0040] As an embodiment, a specific method of determining the historical color compensation coefficient as the target color compensation coefficient may include: If the difference between the current color distribution of each current image to be stitched and the historical color distribution of the historical image to be stitched is within a preset range, the historical color compensation coefficient of the historical image to be stitched is used as the target color compensation coefficient of the current image to be stitched; wherein the historical image to be stitched is the previous frame of the historical image to be stitched before the current image to be stitched.

[0041] In this embodiment, the historical image to be stitched may be a previous frame of the historical image to be stitched before the current image to be stitched. Specifically, if the number of historical images to be stitched corresponding to the current image to be stitched is one, the difference between the historical color distribution of the historical image to be stitched and the current color distribution of the current image to be stitched is directly determined; if the number of historical images to be stitched corresponding to the current image to be stitched is multiple, the historical color distribution of the historical image to be stitched that has the closest image acquisition time to the current image to be stitched (i.e., the previous frame of the historical image to be stitched before the current image to be stitched) is selected and the difference operation is performed between the historical color distribution and the current color distribution of the current image to be stitched.

[0042] In this embodiment, if the difference between the current color distribution of each current image to be stitched of K current images to be stitched of different perspectives and the historical color distribution of the historical image to be stitched corresponding to the current image to be stitched is within a preset range, it indicates that the color distribution of the current image to be stitched and the corresponding historical image to be stitched is slightly different, that is, there is no obvious color jump in the timing.

[0043] In this case, the historical color compensation coefficient of the previous frame of the historical image to be stitched can be directly used as the target color compensation coefficient of the current image to be stitched, that is, the historical color compensation coefficient of the previous frame of the historical image to be stitched is used, and there is no need to recalculate the target color compensation coefficient of the current image to be stitched. In this way, computing resources can be saved and power consumption can be reduced.

[0044] Specifically, if the number of historical images to be stitched corresponding to each current image to be stitched is one, the historical color compensation coefficient of the historical images to be stitched corresponding to each current image to be stitched can be directly used as the target color compensation coefficient of the current image to be stitched.

[0045] If there are multiple historical images to be stitched corresponding to each current image to be stitched, the historical color compensation coefficient of the historical image to be stitched whose image acquisition time is closest to the current image to be stitched (that is, the previous frame of the historical image to be stitched before the current image to be stitched) is used as the target color compensation coefficient of the current image to be stitched.

[0046] This concludes the description of situation (1), i.e., determining the historical color compensation coefficient as the target color compensation coefficient.

[0047] (2) Determine a target color compensation coefficient based on the current color compensation coefficient and the historical color compensation coefficient of the current image to be stitched.

[0048] As an embodiment, a method for determining a target color compensation coefficient based on a current color compensation coefficient and a historical color compensation coefficient of the current image to be stitched may include: If the difference between the current color distribution of any current image to be stitched and the historical color distribution of the historical image to be stitched is not within a preset range, a weighted operation is performed on the current color compensation coefficient of the current image to be stitched and the historical color compensation coefficient of the historical image to be stitched to obtain a target color compensation coefficient; wherein the historical image to be stitched is the previous M frames of the historical image to be stitched before the current image to be stitched, and M is greater than or equal to 1.

[0049] In this embodiment, if the difference between the current color distribution of any current image to be stitched and the historical color distribution of the historical image to be stitched is not within the preset range, it indicates that the color distribution of the current image to be stitched and the corresponding historical image to be stitched is quite different, that is, an obvious color jump occurs in the timing.

[0050] In this case, the current color compensation coefficient of the current image to be spliced ​​and the historical color compensation coefficient of the historical images to be spliced ​​can be weighted to obtain the target color compensation coefficient. In this way, when determining the target color compensation coefficient of the current image to be spliced, the influence of the color information of the historical images to be spliced ​​is fully considered, so that the target spliced ​​image determined by the target color compensation coefficient can have more balanced colors in time sequence, thereby enhancing the viewing experience of the video.

[0051] In this case, different from the case (1), the historical images to be stitched may be the previous M frames of historical images to be stitched before the current image to be stitched, where M is greater than or equal to 1.

[0052] Since the historical images to be spliced ​​can be one or more, the obtained historical color compensation coefficients are also one or more. At this time, according to the weights set for each historical color compensation coefficient and the current color compensation coefficient of the current image to be spliced, a weighted operation can be performed on the current color compensation coefficient of the current image to be spliced ​​and the historical color compensation coefficients of the historical images to be spliced ​​to obtain the target color compensation coefficient.

[0053] The specific weighted calculation method will be described below and will not be repeated here.

[0054] This concludes the description of situation (2), that is, determining the target color compensation coefficient based on the current color compensation coefficient and the historical color compensation coefficient of the current image to be stitched.

[0055] It should be noted that the current color compensation coefficient of the current image to be stitched proposed in the above situation (2) can be determined by the following method.

[0056] As an embodiment, the K current images to be stitched include a reference image and at least one non-reference image, and the reference image is any current image to be stitched selected from the K current images to be stitched; the current color compensation coefficient of the current image to be stitched is determined by the following method: For the reference image, a configured default color compensation coefficient is obtained, and the default color compensation coefficient is determined as the current color compensation coefficient of the reference image; For each non-reference image, a current color compensation coefficient of the non-reference image relative to the reference image is determined based on the current color distribution of the non-reference image and the current color distribution of the reference image.

[0057] In this embodiment, since the color compensation coefficient is a coefficient used to adjust the color effect of the current image to be consistent with the color effect of the reference image, it is necessary to select one of the K current images to be stitched as the reference image and determine the remaining images as non-reference images.

[0058] For the reference image, which serves as the color standard of the non-reference image, there is no need to perform color compensation on the spatial dimensions of the K images to be stitched that are collected at the current moment. Its default color compensation coefficient can be 1 (that is, the compensated image is itself). The default color compensation coefficient can be directly determined as the current color compensation coefficient of the reference image. This application does not impose any restrictions on the setting of the default color compensation coefficient.

[0059] For each non-reference image, a current color compensation coefficient of the non-reference image relative to the reference image may be determined based on the current color distribution of the non-reference image and the current color distribution of the reference image.

[0060] In this embodiment, the current color distribution of the non-reference image and the current color distribution of the reference image include a first channel color distribution, a second channel color distribution, and a third channel color distribution; based on the current color distribution of the non-reference image and the current color distribution of the reference image, determining a current color compensation coefficient of the non-reference image relative to the reference image includes: The current color compensation coefficient of the non-reference image in each channel relative to the reference image is determined according to the following formula:

[0061]

[0062]

[0063] in, Indicates the current color compensation coefficient of the nth non-reference image corresponding to the first channel, represents the color distribution of the first channel of the nth non-reference image, represents the color distribution of the first channel of the reference image; Indicates the current color compensation coefficient of the nth non-reference image corresponding to the second channel, represents the color distribution of the second channel of the nth non-reference image, Represents the color distribution of the second channel of the reference image; Indicates the current color compensation coefficient corresponding to the third channel of the nth non-reference image. represents the color distribution of the third channel of the nth non-reference image, Indicates the color distribution of the third channel of the reference image.

[0064] The following first introduces the method for determining the color distribution of each channel.

[0065] As an embodiment, the current color distribution of the current image to be stitched may be determined by calculating the mean values ​​of the three YUV channels.

[0066] Among them, the YUV three-channel is a model for representing color in image and video processing, which can decompose the color in the image into three components: Y, U and V.

[0067] Specifically, the Y channel represents the brightness of the image (the intensity of light), which can be obtained by weighted calculation based on the red, green, and blue color channels of the image.

[0068] The U channel and V channel represent chroma (color component), which determines the hue and color saturation of the image respectively.

[0069] The U channel represents the difference in brightness between the blue portion of the color and is usually a combination of blue and green.

[0070] The V channel represents the difference in brightness between the red portion of the color, usually a combination of red and green.

[0071] For example, for the input current image to be stitched, the color distribution can be calculated as follows: Input images, and count the mean of the three YUV channels of different images respectively.

[0072]

[0073]

[0074] in, represents the color distribution of the Y channel of the i-th image, Represents the Y channel data of the i-th image, represents the color distribution of the U channel of the i-th image, Represents the U channel data of the i-th image, represents the color distribution of the V channel of the ith image, Represents the V channel data of the i-th image, Indicates average value.

[0075] get: , , , , , ,…, , , . That is, the average value of the three YUV channels of the current images to be stitched.

[0076] As an embodiment, the current color distribution of the current image to be stitched may also be described by other methods such as parameters such as RGB, HSV, or a more complex model, and the present application does not limit this.

[0077] As an embodiment, for the color distribution of multiple channels, each channel corresponds to a color compensation coefficient. Specifically, the color compensation coefficient may be calculated by taking any one of the K images to be stitched as a reference image, calculating the color compensation coefficients of the remaining non-reference images relative to the reference image, and taking the first image to be stitched as the reference image as an example, The color compensation coefficient of an image is calculated as follows:

[0078]

[0079]

[0080] in, Indicates the current color compensation coefficient corresponding to the Y channel of the Kth image, represents the color distribution of the Y channel of the Kth image, Indicates the color distribution of the Y channel of the first image (i.e. the reference image); Indicates the current color compensation coefficient corresponding to the U channel of the K-th image, represents the color distribution of the U channel of the Kth image, Indicates the color distribution of the U channel of the first image (i.e., the reference image); Indicates the current color compensation coefficient corresponding to the V channel of the Kth image, represents the color distribution of the V channel of the Kth image, Indicates the color distribution of the V channel of the first image (i.e., the reference image).

[0081] In this embodiment, in the case where the current image to be stitched corresponds to the first frame image in the target video, the historical image to be stitched corresponding to the current image to be stitched is the current image to be stitched itself.

[0082] For the reference image itself, a default color compensation coefficient is pre-configured, and the current color compensation coefficient of the reference image is the default color compensation coefficient. The default color compensation coefficient may be 1, and the present application does not impose any limitation on this.

[0083] The following is a brief introduction to the method of obtaining the target color compensation coefficient by weighted operation.

[0084] As an embodiment, the target color compensation coefficient of the current image to be stitched may be determined according to the current color compensation coefficient of each current image to be stitched and the historical color compensation coefficient of the historical image to be stitched corresponding to the current image to be stitched.

[0085] Specifically, for each current image to be stitched, a weighted average operation is performed on the current color compensation coefficient and the historical color compensation coefficient of the current image to be stitched according to a preset weight coefficient to obtain a target color compensation coefficient of the current image to be stitched.

[0086] In this embodiment, before executing the video stitching method proposed in this application, the number of recorded historical images to be stitched may be determined first, for example, the latest three historical images to be stitched may be retained as a basis for calculating the target color compensation coefficient.

[0087] In this embodiment, if only a historical color compensation coefficient of a historical image to be stitched of the current image to be stitched is recorded, the historical color compensation coefficient and the current color compensation coefficient of the current image to be stitched can be weighted averaged according to the preset weight coefficient of the historical color compensation coefficient (such as 0.4) and the weight coefficient of the current color compensation coefficient (such as 0.6) to obtain the target color compensation coefficient of the current image to be stitched. Exemplarily, the sum of the weight coefficients of the historical color compensation coefficient and the current color compensation coefficient can be 1.

[0088] In this embodiment, if the historical color compensation coefficients of multiple historical images to be spliced ​​of the current image to be spliced ​​are recorded, a weighted operation can be performed according to the weight coefficients respectively configured for the multiple historical color compensation coefficients and the current color compensation coefficient. For example, if the latest three historical images to be spliced ​​are retained, the historical color compensation coefficients and the current color compensation coefficient of the current image to be spliced ​​can be weighted averaged according to the preset weight coefficients respectively corresponding to the historical color compensation coefficients of the latest three historical images to be spliced, and the weight coefficient of the current color compensation coefficient, to obtain the target color compensation coefficient of the current image to be spliced. Exemplarily, the sum of the weight coefficients of each historical color compensation coefficient and the current color compensation coefficient can be 1.

[0089] As an embodiment, the target color compensation coefficient of the current image to be stitched may be calculated according to the following formula:

[0090] in, Indicates the target color compensation coefficient of the tth image to be stitched (here one channel is taken as an example, and the other channels are the same); represents the product of the current color compensation coefficient of the tth current image to be stitched and the corresponding weight coefficient; represents the product of the historical color compensation coefficient of the rth historical image to be stitched corresponding to the current image to be stitched and the corresponding weight coefficient; It represents the sum of the products of the historical color compensation coefficients of the s historical images to be stitched corresponding to the current image to be stitched and the corresponding weight coefficients, and s represents the number of historical images to be stitched that participate in the target color compensation coefficient calculation.

[0091] This concludes the description of the weighted calculation method.

[0092] In this embodiment, for each non-reference image, based on the current color distribution of the non-reference image and the current color distribution of the reference image, a method for determining a current color compensation coefficient of the non-reference image relative to the reference image may include: For each current image to be stitched among the K current images to be stitched, determine a non-motion area in the current image to be stitched according to the current image to be stitched and a historical image to be stitched corresponding to the current image to be stitched; the non-motion area refers to a background area in the current image to be stitched except for a moving object; For each non-reference image, a current color compensation coefficient of the non-reference image relative to the reference image is determined based on the current color distribution of the non-motion region in the non-reference image and the current color distribution of the non-motion region in the reference image.

[0093] In this embodiment, considering that there is a moving target in the current image to be stitched, it will greatly affect the color distribution difference between the current image to be stitched and the corresponding historical image to be stitched. If the moving target is taken into account, the color distribution at the current moment will be larger than the previous one, causing a sudden color change in the timing of the stitched video stream.

[0094] Therefore, the non-moving area in the current image to be stitched can be determined according to the current image to be stitched and the historical image to be stitched corresponding to the current image to be stitched, and the current color compensation coefficient can be further determined according to the current color distribution of the non-moving area. The non-moving area refers to the background area excluding the moving object in the current image to be stitched.

[0095] As an embodiment, a method for determining a non-motion area may include: For each current image to be stitched among the K current images to be stitched, determine a velocity vector of each pixel in the current image to be stitched according to the current image to be stitched and a historical image to be stitched corresponding to the current image to be stitched, where the velocity vector includes a change in motion speed and a change in motion direction of the pixel between acquisition moments of the historical image to be stitched and the current image to be stitched; For each pixel in the current image to be stitched, if it is determined that the change in motion rate of the pixel is less than a first threshold and the change in motion direction of the pixel is less than a second threshold, the pixel is taken as a candidate pixel; A non-motion area in the current image to be stitched is determined based on a set of all candidate pixels.

[0096] As an embodiment, when calculating the current color distribution, it is necessary to remove the moving target area that may cause influence (the non-moving area after removing the moving target area can be recorded as Mask), and determine the target color compensation coefficient only according to the color distribution of the non-moving area.

[0097]

[0098]

[0099]

[0100] in, represents the color distribution of the Y channel of the i-th image, Represents the Y channel data of the i-th image, represents the color distribution of the U channel of the i-th image, Represents the U channel data of the i-th image, represents the color distribution of the V channel of the ith image, Represents the V channel data of the i-th image, Indicates averaging, and Mask indicates the non-motion area.

[0101] At this point, the description of step 102 ends, and step 103 is executed next.

[0102] Step 103 , color correction is performed on the current image to be stitched according to the target color compensation coefficient of the current image to be stitched, so as to obtain a target color image corresponding to the current image to be stitched.

[0103] Specifically, the method of performing color correction on the current image to be stitched according to the target color compensation coefficient of the current image to be stitched to obtain the target color image corresponding to the current image to be stitched may include: The target color distribution of the current image to be spliced ​​is determined according to the target color compensation coefficient of the current image to be spliced ​​and the current color distribution of the current image to be spliced, and the image corresponding to the target color distribution is used as the target color image.

[0104] In this embodiment, Taking an image as an example, the method for determining the target color distribution can be:

[0105]

[0106]

[0107] in, Indicates the target color compensation coefficient corresponding to the Y channel of the Kth image, represents the target color distribution of the Y channel of the Kth image, Indicates the current color distribution of the Y channel of the Kth image; Indicates the target color compensation coefficient corresponding to the U channel of the Kth image, represents the target color distribution of the U channel of the Kth image, Represents the current color distribution of the U channel of the Kth image; Indicates the target color compensation coefficient corresponding to the V channel of the Kth image, represents the target color distribution of the V channel of the Kth image, Indicates the current color distribution of the V channel of the Kth image.

[0108] At this point, the description of step 103 ends, and step 104 is executed next.

[0109] Step 104 , stitching the target color images corresponding to the K current images to be stitched together to obtain a target stitched image.

[0110] The target stitched images are used to form a target video.

[0111] In this embodiment, for each current image to be stitched, a target color image of the current image to be stitched after color correction is obtained according to step 103. Further, the target color images corresponding to K current images to be stitched can be stitched to obtain a target stitched image for constituting a target video.

[0112] This concludes the description of step 104.

[0113] In this embodiment, the above method for determining the target color compensation coefficient corresponding to each current image to be stitched can also be determined based on a deep learning network. For example, the current image to be stitched and the historical image to be stitched corresponding to the current image to be stitched are input into a trained deep learning network, and the target color compensation coefficient corresponding to the current image to be stitched is directly obtained end-to-end, and further a step of performing color correction on the current image to be stitched according to the target color compensation coefficient of the current image to be stitched is performed to obtain the target color image corresponding to the current image to be stitched.

[0114] Alternatively, the target color image corresponding to the current image to be stitched can be directly obtained through a trained deep learning network. For example, the current image to be stitched and the historical image to be stitched corresponding to the current image to be stitched are input into the trained deep learning network, and the target color image corresponding to the current image to be stitched is directly obtained end-to-end. This application does not impose any restrictions on this.

[0115] This concludes Figure 1 Introduction to video stitching methods.

[0116] In the present application, when K current images to be stitched of different viewing angles are obtained, for each current image to be stitched, a historical color compensation coefficient is determined as a target color compensation coefficient, or the target color compensation coefficient is determined based on the current color compensation coefficient and the historical color compensation coefficient of the current image to be stitched, thereby fully considering the influence of the color information of the historical images to be stitched on the color compensation coefficient of the current image to be stitched; further, the current image to be stitched is color corrected according to the target color compensation coefficient of the current image to be stitched to obtain a target color image corresponding to the current image to be stitched, and the target color images corresponding to the K current images to be stitched are stitched to obtain a target stitched image for constituting a target video, thereby achieving color balance in the video stitching process.

[0117] Combine the following Figure 2 The overall process of the video stitching method is introduced.

[0118] Please refer to Figure 2 , Figure 2 A schematic diagram of the overall process provided for an embodiment of the present application.

[0119] like Figure 2 As shown, this scheme proposes a video stitching method that can synchronously achieve spatiotemporal color balance.

[0120] Specifically, firstly, K current images to be stitched at the current moment are obtained, and then the K current images to be stitched are stitched to obtain a stitched image. The stitching process can be divided into three stages, namely, the image alignment stage, the spatiotemporal color balancing stage and the transition zone processing stage.

[0121] Through the above three stages, a stitched image obtained by stitching K current images to be stitched can be obtained. If it is found that after the stitching of the current images to be stitched is completed, there are still current images to be stitched at the next moment, enter the processing of the next moment, and obtain the current image to be stitched corresponding to the next moment, until there is no current image to be stitched at the next moment, it is determined that the stitched images of all frames of the video to be stitched are obtained.

[0122] After all the stitched images are obtained, the stitched images are used as each frame in the video to be stitched, and video stitching is performed to obtain the target video.

[0123] In this embodiment, the entire video stitching process may be performed online, for example, the stitching operation may be performed while the video is being played or recorded, so that the video data is processed in real time and the stitching result is output or displayed immediately.

[0124] In this embodiment, the entire video stitching process can also be performed offline, such as performing the stitching operation after the video recording is completed, and this application does not impose any restrictions on this.

[0125] Each stage will be introduced below.

[0126] Please refer to Figure 3 , Figure 3 A schematic diagram of the specific process of the spatiotemporal color balancing stage provided in an embodiment of the present application.

[0127] like Figure 3 As shown, the spatiotemporal color equalization stage realizes the spatiotemporal color equalization of the video stream to be spliced.

[0128] Specifically, k current images to be stitched are obtained, where the current images to be stitched may be original images or aligned images that have been obtained through an image alignment phase.

[0129] First, the target color compensation coefficient of the current image to be stitched is determined according to the historical color compensation coefficient corresponding to the current image to be stitched, and the historical color compensation coefficient is updated (updated to the target color compensation coefficient of the current image to be stitched).

[0130] Secondly, based on the target color compensation coefficient corresponding to each current image to be stitched, color correction is performed on each current image to be stitched, so as to obtain k target color images with balanced colors.

[0131] This concludes Figure 3 Description.

[0132] Please refer to Figure 4 , Figure 4 A schematic diagram of the specific process of the image alignment stage provided in an embodiment of the present application.

[0133] like Figure 4 As shown, the image alignment stage realizes the image alignment of the video stream to be spliced ​​at each moment. At a specific moment, its input can be the current image to be spliced, and can also include a mapping matrix (i.e., used to map the image to be spliced ​​from one coordinate system to another coordinate system, so as to accurately align different images). When the input includes a mapping matrix, the mapping matrix can be directly applied to transform the current image to be spliced. Otherwise, the mapping matrix is ​​calculated first, and then the image transformation is performed. Finally, the aligned current image to be spliced ​​is output, and the mapping matrix can also be output at the same time.

[0134] Specifically, the alignment phase may include the following processes: 1. Initialization processing: Calculate the mapping matrix based on the current image to be stitched of the initial frame.

[0135] In this embodiment, since the position of the camera in video stitching is usually fixed, the mapping matrix calculated for the first frame can be used for subsequent frames to improve efficiency. For the current image to be stitched (taking T frames as an example), it can be determined whether the current image to be stitched (i.e., the t-th frame image) is the first frame image of the video. If it is the first frame image (i.e., t=1), the mapping matrix can be calculated.

[0136] The calculation process of the mapping matrix is ​​as follows: (1) Feature detection and matching. First, the feature points of the input image to be stitched are detected, such as using SIFT (Scale Invariant Feature Transform) and other methods to extract features and obtain the feature points of each image.

[0137] Feature matching is performed by using methods such as NN (Nearest Neighbor: neural network) to obtain matching point pairs between the current images to be stitched, which is not limited in this application.

[0138] In order to further improve the accuracy of matching point pairs, algorithms such as RANSAC (RANdom SAmpleConsensus: RANdom SAC sampling consistency algorithm) can be applied to filter out outliers and obtain matching point pairs with higher consistency.

[0139] (2) Calculation of the mapping matrix. The homography matrix can be solved as the mapping matrix. The homography matrix can realize the projection of feature points from one plane to another plane, which is expressed as follows:

[0140] Among them, the homography matrix H = , h11, h12, h13 describe the projection transformation from the source plane to the target plane in the x-axis direction, h21, h22, h23 describe the projection transformation from the source plane to the target plane in the y-axis direction, and the three parameters h31, h32, h33 are usually related to the perspective transformation. They control the transformation of the third coordinate so that the mapping takes into account the perspective effect; is the coordinate of the point on the target plane obtained after projection, are the coordinates of the point on the source plane.

[0141] The above matrix describes the geometric transformation from the plane of one image to the plane of another image. Through this matrix, a point (x1, y1) on an image can be mapped to a point (x2, y2) on another image through a homography transformation.

[0142] Since the homography matrix H is a homogeneous matrix, it contains 9 parameters, which are calculated from 4 pairs of matching points. In order to solve these parameters, at least 4 pairs of matching points are required. The least squares method can be used for the solution. The solution method of the homography matrix is ​​a common method in the relevant technology and will not be repeated here.

[0143] At this point, the mapping matrix between the current images to be stitched is obtained.

[0144] 2. Perform transformation processing on the current image to be stitched at each moment to output the aligned current image to be stitched.

[0145] If it is determined that the current image to be spliced ​​is not the first frame image in the video (ie, t≠1), the image transformation can be performed based on the determined mapping matrix.

[0146] In this embodiment, the reverse warping method can be used, that is, for each pixel of the target alignment image, the corresponding coordinates in the original image are solved, and the pixel value is obtained by interpolation methods such as bilinear interpolation. That is, for each pixel point of the given target image, the corresponding position in the original image is found using the inverse matrix of the homography matrix. Specifically, it can be calculated by the following formula:

[0147] Among them, the homography matrix H = , is the inverse matrix of the homography matrix, is the position of each pixel in the target image; is the coordinate of the point in the target image in the original image.

[0148] The above embodiments can realize image alignment, and the present application does not limit the specific method of image alignment.

[0149] After completing the alignment of the current image to be stitched corresponding to the current frame, the next frame image can be transformed and aligned based on the above mapping matrix (ie, t=t+1) until all T frame images included in the video have completed image alignment (ie, t=T).

[0150] This concludes Figure 4 Description.

[0151] Please refer to Figure 5 , Figure 5 A schematic diagram of the specific process flow of the transition zone processing stage provided in an embodiment of the present application.

[0152] like Figure 5As shown, the transition band processing stage realizes the transition band fusion of the video stream to be spliced ​​at each moment. The input of the transition band processing stage is T target spliced ​​images after alignment and color balance (taking a total of T frames as an example, each frame corresponds to a target spliced ​​image), and the transition band processing is performed on the above T target spliced ​​images to make the transition band transition naturally.

[0153] Specifically, the transition zone processing stage may perform the following steps for each target stitched image (i.e., t=1 to t=T, the 1st to the Tth target stitched image): 1. Seam search, used to determine the location of the seam.

[0154] First, the image alignment stage can obtain the aligned current image to be stitched, where there is an overlapping area between adjacent images. The stitching is to find a line in the overlapping area, which can be simply set as the center line of the area, or a method such as graph cut can be used to find the line with the smallest brightness difference between the left and right images.

[0155] 2. Transition zone fusion. The transition zone can be the entire overlapping area, or it can be set to an area within a specific range from the seam. Simple methods include feathering fusion, etc., and complex multi-band fusion methods can also be used to improve the naturalness of the transition.

[0156] The above method is performed on T target stitched images, and a natural and smooth transition of the transition zone area can be achieved. The above embodiment can achieve transition zone processing and obtain the final stitched video.

[0157] This concludes Figure 5 Description.

[0158] Please refer to Fig. 6A as well as Figure 6B , Fig. 6A This is an example diagram of spatial color balance provided in an embodiment of the present application; Figure 6B This is an example diagram of spatiotemporal color balance provided in an embodiment of the present application.

[0159] like Fig. 6A As shown, since there are many dynamic targets in the scene and the lighting conditions change with the weather and street lights, the color difference of the images before and after is large. According to the traditional method of stitching video obtained by only using spatial color balancing, the stitching images before and after have large color changes, which appear unnatural. When the images at these two moments are played as adjacent frames, there will be a visual perception of color jump.

[0160] like Figure 6B As shown, by applying the above spatiotemporal color balanced video stitching method, the color difference between the images at the previous and next moments is small, presenting a better visual effect, effectively alleviating the problem of temporal color inconsistency, and being more conducive to the natural and smooth stitching of the video.

[0161] Please refer to Figure 7 , Figure 7 It is a schematic structural diagram of an electronic device proposed in an embodiment of the present application. At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and of course may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a terminal interaction device at the logical level. Of course, in addition to the software implementation method, this application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0162] Please refer to Figure 8 , Figure 8 is a structural diagram of a video splicing device proposed in an embodiment of the present application. Figure 8 As shown, the device may include an acquisition module 801, a determination module 802, a correction module 803 and a splicing module 804. Specifically, the device includes: An acquisition module 801 is used to acquire K current images to be stitched from different perspectives, where K is not less than 2; A determination module 802 is used to determine, for each current image to be stitched, a target color compensation coefficient of the current image to be stitched based on a historical color compensation coefficient of a historical image to be stitched corresponding to the current image to be stitched; wherein the historical image to be stitched and the current image to be stitched are images at the same viewing angle in the same video; wherein the historical color compensation coefficient is determined as the target color compensation coefficient, or the target color compensation coefficient is determined based on the current color compensation coefficient and the historical color compensation coefficient of the current image to be stitched; A correction module 803 is used to perform color correction on the current image to be stitched according to the target color compensation coefficient of the current image to be stitched, so as to obtain a target color image corresponding to the current image to be stitched; The stitching module 804 is used to stitch the target color images corresponding to the K current images to be stitched together to obtain a target stitched image; the target stitched image is used to form a target video.

[0163] Optionally, the determination module 802 is specifically configured to: If the difference between the current color distribution of each current image to be stitched and the historical color distribution of the historical image to be stitched is within a preset range, the historical color compensation coefficient of the historical image to be stitched is used as the target color compensation coefficient of the current image to be stitched; wherein the historical image to be stitched is a previous frame of the historical image to be stitched of the current image to be stitched; And / or, the determination module 802 is specifically used for: If the difference between the current color distribution of any current image to be stitched and the historical color distribution of the historical image to be stitched is not within a preset range, a weighted operation is performed on the current color compensation coefficient of the current image to be stitched and the historical color compensation coefficient of the historical image to be stitched to obtain a target color compensation coefficient; wherein the historical image to be stitched is the previous M frames of the historical image to be stitched before the current image to be stitched, and M is greater than or equal to 1; And / or, the K current images to be stitched include a reference image and at least one non-reference image, and the reference image is any current image to be stitched selected from the K current images to be stitched; the determination module 802 is further used for: For the reference image, a configured default color compensation coefficient is obtained, and the default color compensation coefficient is determined as the current color compensation coefficient of the reference image; For each non-reference image, determining a current color compensation coefficient of the non-reference image relative to the reference image based on a current color distribution of the non-reference image and a current color distribution of the reference image; And / or, the current color distribution of the non-reference image and the current color distribution of the reference image include a first channel color distribution, a second channel color distribution, and a third channel color distribution; the determination module 802 is specifically used to: The current color compensation coefficient of the non-reference image in each channel relative to the reference image is determined according to the following formula:

[0164]

[0165]

[0166] in, Indicates the current color compensation coefficient of the nth non-reference image corresponding to the first channel, represents the color distribution of the first channel of the nth non-reference image, represents the color distribution of the first channel of the reference image; Indicates the current color compensation coefficient of the nth non-reference image corresponding to the second channel, represents the color distribution of the second channel of the nth non-reference image, Represents the color distribution of the second channel of the reference image; Indicates the current color compensation coefficient corresponding to the third channel of the nth non-reference image. represents the color distribution of the third channel of the nth non-reference image, Indicates the color distribution of the third channel of the reference image; And / or, the determination module 802 is specifically used for: For each current image to be stitched among the K current images to be stitched, determine a non-motion area in the current image to be stitched according to the current image to be stitched and a historical image to be stitched corresponding to the current image to be stitched; the non-motion area refers to a background area in the current image to be stitched except for a moving object; For each non-reference image, determining a current color compensation coefficient of the non-reference image relative to the reference image based on a current color distribution of a non-motion region in the non-reference image and a current color distribution of a non-motion region in the reference image; And / or, the determination module 802 is specifically used for: For each current image to be stitched among the K current images to be stitched, determine a velocity vector of each pixel in the current image to be stitched according to the current image to be stitched and a historical image to be stitched corresponding to the current image to be stitched, where the velocity vector includes a change in motion speed and a change in motion direction of the pixel between acquisition moments of the historical image to be stitched and the current image to be stitched; For each pixel in the current image to be stitched, if it is determined that the change in motion rate of the pixel is less than a first threshold and the change in motion direction of the pixel is less than a second threshold, the pixel is taken as a candidate pixel; A non-motion area in the current image to be stitched is determined based on a set of all candidate pixels.

[0167] So far, completed Figure 8 Description of the video stitching device.

[0168] Correspondingly, based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the video stitching method disclosed in the above example of the present application can be implemented.

[0169] The above-mentioned machine-readable storage medium may be any electronic, magnetic, optical or other physical storage device, which may contain or store information, such as executable instructions, data, etc. For example, the machine-readable storage medium may be: RAM (Radom Access Memory), volatile memory, non-volatile memory, flash memory, storage drive (such as hard disk drive), solid state drive, any type of storage disk (such as optical disk, DVD, etc.), or similar storage medium, or a combination thereof.

[0170] Based on the same application concept as the above method, the embodiment of the present application further provides a computer program product, which may include a computer program. When the computer program is executed by a processor, the video stitching method disclosed in the above example of the present application is implemented.

[0171] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of protection of the present application.

Claims

1. A video splicing method, characterized in that: The method includes: Obtain K current images to be stitched from different perspectives, where K is not less than 2; For each current image to be stitched, a target color compensation coefficient of the current image to be stitched is determined based on a historical color compensation coefficient of a historical image to be stitched corresponding to the current image to be stitched; wherein the historical image to be stitched and the current image to be stitched are images at the same viewing angle in the same video; wherein the historical color compensation coefficient is determined as the target color compensation coefficient, or the target color compensation coefficient is determined based on the current color compensation coefficient of the current image to be stitched and the historical color compensation coefficient; Performing color correction on the current image to be stitched according to the target color compensation coefficient of the current image to be stitched, to obtain a target color image corresponding to the current image to be stitched; The target color images corresponding to the K current images to be stitched are stitched to obtain a target stitched image; the target stitched image is used to form a target video.

2. The method according to claim 1, characterized in that The step of determining the historical color compensation coefficient as the target color compensation coefficient includes: If the difference between the current color distribution of each current image to be stitched and the historical color distribution of the historical image to be stitched is within a preset range, the historical color compensation coefficient of the historical image to be stitched is used as the target color compensation coefficient of the current image to be stitched; wherein the historical image to be stitched is the previous frame of the historical image to be stitched before the current image to be stitched.

3. The method according to claim 1, characterized in that The determining the target color compensation coefficient based on the current color compensation coefficient of the current image to be stitched and the historical color compensation coefficient includes: If the difference between the current color distribution of any current image to be stitched and the historical color distribution of the historical image to be stitched is not within a preset range, a weighted operation is performed on the current color compensation coefficient of the current image to be stitched and the historical color compensation coefficient of the historical image to be stitched to obtain the target color compensation coefficient; wherein the historical image to be stitched is the previous M frames of the historical image to be stitched before the current image to be stitched, and M is greater than or equal to 1.

4. The method according to claim 1 or 3, characterized in that: The K current images to be stitched include a reference image and at least one non-reference image, and the reference image is any current image to be stitched selected from the K current images to be stitched; the current color compensation coefficient of the current image to be stitched is determined by the following method: For the reference image, obtaining a configured default color compensation coefficient, and determining the default color compensation coefficient as a current color compensation coefficient of the reference image; For each non-reference image, a current color compensation coefficient of the non-reference image relative to the reference image is determined based on the current color distribution of the non-reference image and the current color distribution of the reference image.

5. The method according to claim 4, characterized in that The current color distribution of the non-reference image and the current color distribution of the reference image include a first channel color distribution, a second channel color distribution, and a third channel color distribution; and determining a current color compensation coefficient of the non-reference image relative to the reference image based on the current color distribution of the non-reference image and the current color distribution of the reference image includes: The current color compensation coefficient of the non-reference image corresponding to the reference image in each channel is determined according to the following formula: Among them, the represents the current color compensation coefficient corresponding to the first channel of the nth non-reference image, represents the color distribution of the first channel of the nth non-reference image, represents the color distribution of the first channel of the reference image; represents the current color compensation coefficient corresponding to the second channel of the nth non-reference image, represents the color distribution of the second channel of the nth non-reference image, represents the color distribution of the second channel of the reference image; represents the current color compensation coefficient corresponding to the third channel of the nth non-reference image, represents the color distribution of the third channel of the nth non-reference image, Indicates the color distribution of the third channel of the reference image.

6. The method according to claim 4, characterized in that The step of determining, for each non-reference image, a current color compensation coefficient of the non-reference image relative to the reference image based on the current color distribution of the non-reference image and the current color distribution of the reference image comprises: For each current image to be stitched among the K current images to be stitched, determine a non-motion area in the current image to be stitched according to the current image to be stitched and a historical image to be stitched corresponding to the current image to be stitched; the non-motion area refers to a background area in the current image to be stitched except for a moving object; For each non-reference image, a current color compensation coefficient of the non-reference image relative to the reference image is determined based on a current color distribution of a non-motion region in the non-reference image and a current color distribution of a non-motion region in the reference image.

7. The method according to claim 6, characterized in that The step of determining, for each current image to be stitched among the K current images to be stitched, a non-motion area in the current image to be stitched according to the current image to be stitched and a historical image to be stitched corresponding to the current image to be stitched, includes: For each current image to be stitched among the K current images to be stitched, determine a velocity vector of each pixel in the current image to be stitched according to the current image to be stitched and a historical image to be stitched corresponding to the current image to be stitched, wherein the velocity vector includes a motion speed change amount and a motion direction change amount of the pixel between acquisition moments of the historical image to be stitched and the current image to be stitched; For each pixel in the current image to be stitched, if it is determined that the change in the motion rate of the pixel is less than a first threshold, and the change in the motion direction of the pixel is less than a second threshold, the pixel is taken as a candidate pixel; A non-motion area in the current image to be stitched is determined based on a set of all candidate pixels.

8. A video splicing device, characterized in that: The device includes: An acquisition module, used to acquire K current images to be stitched from different perspectives, where K is not less than 2; A determination module, for determining, for each current image to be stitched, a target color compensation coefficient of the current image to be stitched based on a historical color compensation coefficient of a historical image to be stitched corresponding to the current image to be stitched; wherein the historical image to be stitched and the current image to be stitched are images at the same viewing angle in the same video; wherein the historical color compensation coefficient is determined as the target color compensation coefficient, or the target color compensation coefficient is determined based on the current color compensation coefficient of the current image to be stitched and the historical color compensation coefficient; A correction module, used for performing color correction on the current image to be stitched according to the target color compensation coefficient of the current image to be stitched, so as to obtain a target color image corresponding to the current image to be stitched; The splicing module is used to splice the target color images corresponding to the K current images to be spliced ​​to obtain a target spliced ​​image; the target spliced ​​image is used to form a target video.

9. The device according to claim 8, characterized in that The determination module is specifically used for: If the difference between the current color distribution of each current image to be stitched and the historical color distribution of the historical image to be stitched is within a preset range, the historical color compensation coefficient of the historical image to be stitched is used as the target color compensation coefficient of the current image to be stitched; wherein the historical image to be stitched is a previous frame of the historical image to be stitched of the current image to be stitched; And / or, the determining module is specifically used for: If the difference between the current color distribution of any current image to be stitched and the historical color distribution of the historical image to be stitched is not within a preset range, a weighted operation is performed on the current color compensation coefficient of the current image to be stitched and the historical color compensation coefficient of the historical image to be stitched to obtain the target color compensation coefficient; wherein the historical image to be stitched is the previous M frames of the historical image to be stitched before the current image to be stitched, and M is greater than or equal to 1; And / or, the K current images to be stitched include a reference image and at least one non-reference image, and the reference image is any current image to be stitched selected from the K current images to be stitched; the determination module is further used for: For the reference image, obtaining a configured default color compensation coefficient, and determining the default color compensation coefficient as a current color compensation coefficient of the reference image; For each non-reference image, determining a current color compensation coefficient of the non-reference image relative to the reference image based on a current color distribution of the non-reference image and a current color distribution of the reference image; And / or, the current color distribution of the non-reference image and the current color distribution of the reference image include a first channel color distribution, a second channel color distribution, and a third channel color distribution; and the determination module is specifically configured to: The current color compensation coefficient of the non-reference image corresponding to the reference image in each channel is determined according to the following formula: Among them, the represents the current color compensation coefficient corresponding to the first channel of the nth non-reference image, represents the color distribution of the first channel of the nth non-reference image, represents the color distribution of the first channel of the reference image; represents the current color compensation coefficient corresponding to the second channel of the nth non-reference image, represents the color distribution of the second channel of the nth non-reference image, represents the color distribution of the second channel of the reference image; represents the current color compensation coefficient corresponding to the third channel of the nth non-reference image, represents the color distribution of the third channel of the nth non-reference image, Indicates the color distribution of the third channel of the reference image; And / or, the determining module is specifically used for: For each current image to be stitched among the K current images to be stitched, determine a non-motion area in the current image to be stitched according to the current image to be stitched and a historical image to be stitched corresponding to the current image to be stitched; the non-motion area refers to a background area in the current image to be stitched except for a moving object; For each non-reference image, determining a current color compensation coefficient of the non-reference image relative to the reference image based on a current color distribution of a non-motion region in the non-reference image and a current color distribution of a non-motion region in the reference image; And / or, the determining module is specifically used for: For each current image to be stitched among the K current images to be stitched, determine a velocity vector of each pixel in the current image to be stitched according to the current image to be stitched and a historical image to be stitched corresponding to the current image to be stitched, wherein the velocity vector includes a motion speed change amount and a motion direction change amount of the pixel between acquisition moments of the historical image to be stitched and the current image to be stitched; For each pixel in the current image to be stitched, if it is determined that the change in the motion rate of the pixel is less than a first threshold, and the change in the motion direction of the pixel is less than a second threshold, the pixel is taken as a candidate pixel; A non-motion area in the current image to be stitched is determined based on a set of all candidate pixels.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

Images