A video splicing method for fully mechanized coal mining face

Abstract

The present invention provides a method for splicing videos of coal mine fully mechanized mining faces, and belongs to the field of video fusion technology. The method includes: according to a pre-deployed multi-eye camera, a video of the fully mechanized mining face of the coal mine is collected, and according to the deployment position of the pre-deployed multi-eye camera, the collected video is processed by video grouping; each group video is frame extracted, and each frame extracted image is numbered to obtain a sequence of images to be spliced ​​for each corresponding group, wherein the number of frames extracted for each group video is the same; the group target feature under the deployment position of the multi-eye camera is established, and the images in the corresponding sequence of images to be spliced ​​are standardized according to the group target feature to obtain a standard image; based on all standard images, video splicing is performed to obtain a spliced ​​video of the fully mechanized mining face of the coal mine. It is convenient to obtain a complete video of the fully mechanized mining face of the coal mine.

Description

Technical Field

[0001] The present invention relates to the technical field of video fusion, and in particular to a method for splicing videos of fully mechanized mining faces in coal mines. Background Art

[0002] At present, in the working environment of coal mine fully mechanized mining face, multiple cameras are usually set up to monitor and record the working site. These cameras can cover different angles and positions to obtain more comprehensive video information. However, the picture captured by one camera is limited and cannot fully present the situation of the entire fully mechanized mining face. This will lead to incomplete pictures and abnormal analysis errors, causing management difficulties and safety problems for the fully mechanized mining face.

[0003] Therefore, the present invention proposes a method for splicing videos of fully mechanized mining faces in coal mines. Summary of the invention

[0004] The present invention provides a method for splicing videos of a coal mine fully mechanized mining face, which is used for monitoring and shooting the coal mine fully mechanized mining face through a multi-camera, and after processing the videos shot by multiple cameras, they are matched according to the degree of overlap and spliced ​​into a complete video of the coal mine fully mechanized mining face, which is convenient for subsequent management.

[0005] The present invention provides a method for splicing videos of fully mechanized coal mining faces, comprising:

[0006] Step 1: Collect videos of the fully-mechanized mining face of the coal mine according to the pre-deployed multi-eye camera, and group the collected videos according to the deployment positions of the pre-deployed multi-eye camera;

[0007] Step 2: Frame extraction is performed on each grouped video, and each frame extracted image is numbered to obtain a sequence of images to be spliced ​​for each corresponding group, wherein the number of frames extracted from each grouped video is the same;

[0008] Step 3: establishing group target features at the deployment position of the multi-camera, and performing standardization processing on the images in the corresponding image sequence to be stitched according to the group target features to obtain a standard image;

[0009] Step 4: Perform video stitching based on all standard images to obtain a stitched video of the coal mine fully mechanized mining face.

[0010] In one possible implementation method, video collection of the fully mechanized mining face of a coal mine is performed based on a pre-deployed multi-camera, including:

[0011] Based on the basic environmental conditions of the fully-mechanized mining face of the coal mine and the area that needs to be covered and photographed, the initial deployment position and shooting angle of the multi-eye camera are planned, and a corresponding relationship between the initial deployment position and the number of the multi-eye camera at each initial deployment position is established;

[0012] Multi-cameras deployed based on number correspondence are used to capture video of the fully mechanized mining face in coal mines.

[0013] In a possible implementation, the captured video is subjected to video grouping processing, including:

[0014] All collected videos are grouped and processed according to the number correspondence between each working area of ​​the fully mechanized mining face of the coal mine;

[0015] According to the grouping processing result, the grouped video of the corresponding working area is obtained.

[0016] In a possible implementation, frames are extracted from each grouped video, and each extracted frame image is numbered to obtain a sequence of images to be spliced ​​for each corresponding group, including:

[0017] Decoding each grouped video into an original image sequence using a video decoder, wherein the original image sequence is composed of unequal amounts of image frames under multiple acquired videos;

[0018] Determine the average duration according to the video duration of each collected video in each grouped video, and determine the number of extracted frames of the current group according to the average-extracted frame table;

[0019] Extracting a plurality of images in the same collected video according to the number of extracted frames from the original image sequence under the same grouped video at the time interval of the corresponding collected video to obtain a still image, preprocessing the extracted still image to obtain an initial image sequence to be spliced, wherein the initial image sequence to be spliced ​​is composed of equal image frames under multiple collected videos;

[0020] Each frame of the image in the initial sequence of images to be stitched is numbered in a temporal logical order to obtain a sequence of images to be stitched of the target captured video.

[0021] In a possible implementation manner, establishing group target features at the deployment positions of the multi-cameras includes:

[0022] Performing pixel alignment processing on still images extracted from the same acquisition video in the same grouped video to obtain multi-feature images under the corresponding acquisition video, wherein each acquisition video corresponds to a multi-feature image;

[0023] All multi-feature images under the same group video are spliced ​​and edge-fused to obtain the target fused image;

[0024] Calculate the gradient of each pixel in the target fused image and construct a gradient set of the target fused image: , assuming that the target fusion image has a total of n pixels, represents the gradient of the i11th pixel, and the value range of i11 is [1, n];

[0025] Take a point s in the target fusion image and select 3x3 pixels around it to construct the gradient covariance matrix. :

[0026] ;in, Represents the covariance based on the gradient of point s; Represents the covariance based on the gradient on the upper left side of point s; Represents the covariance based on the gradient on the left side of point s; Represents the covariance based on the gradient on the lower left side of point s; Represents the covariance based on the gradient on the upper right side of point s; Represents the covariance based on the gradient on the right side of point s; Represents the covariance based on the gradient on the lower right side of point s; Represents the covariance based on the gradient on the upper side of point s; Represents the covariance based on the gradient under point s;

[0027] Based on the gradient covariance matrix , respectively calculate The retained value for each point in :

[0028] ; where r represents The retention value of the corresponding point in ; express The covariance of corresponding points in ; express The maximum value of all covariances in ; express The average of all covariances in ; Represents the covariance matrix of corresponding points The modulus value of Represents the covariance matrix The modulus value; max represents the maximum value symbol;

[0029] Determine the gradient covariance matrix The average value of all retained values ​​in the target feature image is obtained, and the points corresponding to the retained values ​​greater than the average value are regarded as feature points, so as to obtain all feature points on the target feature image, and then obtain the grouped target features.

[0030] In a possible implementation, performing standardization processing on the images in the corresponding sequence of images to be stitched according to the grouping target feature includes:

[0031] Performing perspective change processing on the multi-feature images under the same group video to obtain a perspective change matrix, substituting the group target features into the perspective change matrix, and obtaining a standard conversion function corresponding to the multi-feature image;

[0032] The standard conversion function is converted into the still images of equal frames in the corresponding captured video in the corresponding images to be spliced ​​in the same group to obtain a first image sequence corresponding to the captured video;

[0033] The first image sequences are grouped together to obtain standard image sequences in corresponding groups.

[0034] In a possible implementation, video stitching is performed based on all standard image sequences to obtain a stitched video of the fully mechanized mining face of the coal mine, including:

[0035] Perform edge matching on the kth standard image of the left video and the kth standard image of the right video that are adjacent to each other in the same group to generate a transformation matrix;

[0036] The k-th standard image of the left video and the original video image corresponding to the k-th standard image of the right video are fused using the transformation matrix to obtain the k-th fused image of the grouped spliced ​​video;

[0037] All frame fusion images under all groups are stitched together to obtain a stitched video of the fully mechanized mining face of the coal mine.

[0038] In a possible implementation manner, the process of edge matching the kth standard image of the left video and the kth standard image of the right video that are adjacent to each other in the same group further includes:

[0039] Perform image alignment processing on the kth standard image of the left video and the kth standard image of the right video, and screen the edge points of the left edge line and the right edge line at equal distances to construct a point array, and obtain a linear change sequence based on the point array ,in, represents the linear value of the j01th column; Represents the value of the first point in the j01th column of the point array; Represents the value of the second point in the j01th column of the point array; represents the standard value of the j01th column in the point array; m01 represents the number of columns contained in the corresponding point array, and each column contains an edge point pair;

[0040] According to the linear variation sequence, the error coefficient of edge matching is determined:

[0041] ;in, Represents the corresponding error coefficient; m02 represents the corresponding point array The number of items not within the preset range; A difference function representing linear values ​​that are not within a preset range; represents the difference function of the corresponding point array; Indicates the j02th linear value that is not within the preset range;

[0042] Based on the error coefficient, the edge point pairs that are not within the preset range are adjusted to achieve subsequent edge matching.

[0043] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0044] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0046] Figure 1 The present invention is a flowchart of a method for splicing videos of a fully mechanized coal mining face in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0048] Embodiment 1:

[0049] The present invention provides a method for splicing videos of fully mechanized coal mining faces, such as Figure 1 As shown, including:

[0050] Step 1: Collect videos of the fully-mechanized mining face of the coal mine according to the pre-deployed multi-eye camera, and group the collected videos according to the deployment positions of the pre-deployed multi-eye camera;

[0051] Step 2: Frame extraction is performed on each grouped video, and each frame extracted image is numbered to obtain a sequence of images to be spliced ​​for each corresponding group, wherein the number of frames extracted from each grouped video is the same;

[0052] Step 3: establishing group target features at the deployment position of the multi-camera, and performing standardization processing on the images in the corresponding image sequence to be stitched according to the group target features to obtain a standard image;

[0053] Step 4: Perform video stitching based on all standard images to obtain a stitched video of the coal mine fully mechanized mining face.

[0054] In this embodiment, a multi-eye camera refers to multiple cameras installed on the same device. These cameras can simultaneously capture the scene images of the coal mine fully mechanized mining face at different angles or positions, thereby providing a more comprehensive perspective and richer working face information. Among them, there are multiple multi-eye cameras, that is, there are multiple deployment positions, all of which are deployed in advance to ensure that all images of the coal mine fully mechanized mining face can be collected.

[0055] In this embodiment, for example, the coal mine has area 1, area 2 and area 3, and there are multi-eye cameras 01 and 02 in area 1, there is a multi-eye camera 03 in area 2, and there are multi-eye cameras 04 and 05 in area 3. At this time, multi-eye cameras 01 and 02 are grouped as a group, multi-eye camera 03 is grouped as a group, and multi-eye cameras 04 and 05 are grouped as a group.

[0056] In this embodiment, video acquisition refers to converting the video signal of the coal mine comprehensive mining face captured in real time into digital data through a multi-camera video device input device for the purpose of processing, storage and transmission.

[0057] In this embodiment, the deployment position of the pre-deployed multi-eye camera is to first survey and detect the comprehensive mining face of the current coal mine before the camera is deployed to determine the deployment position and shooting angle of all multi-eye cameras.

[0058] In this embodiment, frame extraction refers to the process of selecting specific video frames from a continuous video stream as separate image frames for processing or display.

[0059] In this embodiment, the image sequence to be spliced ​​is a group of image sequences to be spliced ​​obtained after frame extraction of all videos under a current group and labeling them in chronological order. For example, group 1 contains video a1 and video a2. At this time, s0 frame images are extracted from video a1, and s0 frame images are extracted from video a2. Thus, the extracted frame images are labeled in chronological order: ta11...ta1s0 and ta21...ta2s0, which is the image sequence to be spliced.

[0060] In this embodiment, the grouping target feature refers to the grouping target feature of the current group composed of multiple feature points obtained after feature processing in the grouped image sequence to be stitched together. That is, the grouping target feature is a concentrated stitching representation of the most feature points existing in each video in the same group involved.

[0061] In this embodiment, the standardization process is a process of subjecting the still images of equal frames under the captured video to perspective change processing - a standard conversion function.

[0062] In this embodiment, the standard image is an image obtained by standardizing a still image of an equal frame under a captured video.

[0063] In this embodiment, video stitching refers to performing edge connection processing on images that are adjacent to each other to obtain a stitched video.

[0064] The working principle and beneficial effects of the above technical solution are: the video stitching method of coal mine comprehensive mining face based on the shooting input of multi-eye camera can monitor and splice the video of coal mine comprehensive mining face in real time, ensure that the video can be presented completely, and realize effective monitoring of coal mine comprehensive mining face.

[0065] Embodiment 2:

[0066] On the basis of the above-mentioned embodiment 1, this embodiment provides a method for splicing videos of a fully-mechanized coal mining face. In step 1, a video of the fully-mechanized coal mining face is collected according to a pre-deployed multi-eye camera, including:

[0067] Based on the basic environmental conditions of the fully-mechanized mining face of the coal mine and the area that needs to be covered and photographed, the initial deployment position and shooting angle of the multi-eye camera are planned, and a corresponding relationship between the initial deployment position and the number of the multi-eye camera at each initial deployment position is established;

[0068] Multi-cameras deployed based on number correspondence are used to capture video of the fully mechanized mining face in coal mines.

[0069] In this embodiment, the basic environmental conditions are the basic environmental conditions of the comprehensive mining face of a coal mine, which refer to some environmental indicators when comprehensive mining operations are carried out underground in a coal mine, including: geological conditions, size and shape of the working face, mechanical properties of the coal and rock mass, gas and coal dust conditions, ambient temperature and humidity, and other environmental indicators.

[0070] In this embodiment, the number correspondence is to set the initial deployment position and the multi-eye camera with their own unique numbers respectively, and the preparation work before deployment requires a one-to-one correspondence between the number of the deployment position and the number of the multi-eye camera.

[0071] The working principle and beneficial effects of the above technical solution are: the multi-eye camera can record and collect the working status and production conditions of the coal mine comprehensive mining face in all directions, avoiding the disadvantages that ordinary cameras capture incomplete images of the coal mine comprehensive mining face and the acquired data cannot support complete analysis.

[0072] Embodiment 3:

[0073] On the basis of the above-mentioned embodiment 1, this embodiment provides a method for splicing videos of fully mechanized coal mining faces. In step 1, the collected videos are grouped and processed, including:

[0074] All collected videos are grouped and processed according to the number correspondence between each working area of ​​the fully mechanized mining face of the coal mine;

[0075] According to the grouping processing result, the grouped video of the corresponding working area is obtained.

[0076] In this embodiment, the working area of ​​the fully mechanized coal mining face refers to the specific area where comprehensive coal mining operations are carried out, including: coal seam development area, coal mining area, support area, and transportation area.

[0077] The working principle and beneficial effects of the above technical solution are: by performing video grouping processing based on the corresponding relationship between numbers, the collected video data can be used more effectively to realize the monitoring, evaluation and analysis of each working area, which can solve the problem of heavy splicing process caused by too many videos and the long splicing working time.

[0078] Embodiment 4:

[0079] On the basis of the above-mentioned embodiment 1, this embodiment provides a method for splicing videos of a fully mechanized coal mining face. In step 2, a frame is extracted from each grouped video, and each frame of the extracted image is numbered to obtain a sequence of images to be spliced ​​for each corresponding group, including:

[0080] Decoding each grouped video into an original image sequence using a video decoder, wherein the original image sequence is composed of unequal amounts of image frames under multiple acquired videos;

[0081] Determine the average duration according to the video duration of each collected video in each grouped video, and determine the number of extracted frames of the current group according to the average-extracted frame table;

[0082] Extracting a plurality of images in the same collected video according to the number of extracted frames from the original image sequence under the same grouped video at the time interval of the corresponding collected video to obtain a still image, preprocessing the extracted still image to obtain an initial image sequence to be spliced, wherein the initial image sequence to be spliced ​​is composed of equal image frames under multiple collected videos;

[0083] Each frame of the image in the initial sequence of images to be stitched is numbered in a temporal logical order to obtain a sequence of images to be stitched of the target captured video.

[0084] In this embodiment, the video decoder is a software or hardware device for decoding compressed video files, and decodes the compressed and encoded coal mine collected video data to restore it into images and sounds.

[0085] In this embodiment, the original image sequence is the original image group obtained after the coal mine collection video under each group is processed by the video decoder. For example, there is a1 video under group 1. The a1 video can be processed by the video decoder to obtain a group of continuous image frames a1e1, a1e2...a1en. These n continuous image frames are the original image group of the a1 video.

[0086] In this embodiment, the unequal image frame amounts mean that the number of image frames contained in the video per second is not constant. For example, there are s0 frames of images originally in the video a1, and there are s1 frames of images originally in the video a2.

[0087] In this embodiment, the average-extracted frame table is a correspondence table based on the average duration of all videos in the group and the corresponding number of extracted frames required, which facilitates direct matching to obtain the number of extracted frames after obtaining the average duration.

[0088] In this embodiment, the preprocessing is to extract the number of frames from a video and then perform processing such as resizing, cropping, filtering, etc. on the extracted still images.

[0089] In this embodiment, the initial image sequence to be stitched is a group of still images obtained after preprocessing. For example, group 1 includes video a1 and video a2. The image size extracted from video a1 is 1000x1000 pixels, while the image size extracted from video a2 is 700x700. It is necessary to unify the image sizes of image a1 and image a2 to facilitate subsequent operations.

[0090] The working principle and beneficial effects of the above technical solution are: extracting an appropriate number of image frames from each captured video for processing can reduce unnecessary repeated or redundant images, and the time sequence sorting can also ensure the correct order of images when splicing, and provide a basis for subsequent image processing and editing.

[0091] Embodiment 5:

[0092] On the basis of the above-mentioned embodiment 1, this embodiment provides a method for splicing videos of a fully mechanized coal mining face. In step 3, a grouping target feature under the deployment position of the multi-eye camera is established, including:

[0093] Performing pixel alignment processing on still images extracted from the same acquisition video in the same grouped video to obtain multi-feature images under the corresponding acquisition video, wherein each acquisition video corresponds to a multi-feature image;

[0094] All multi-feature images under the same group video are spliced ​​and edge-fused to obtain the target fused image;

[0095] Calculate the gradient of each pixel in the target fused image and construct a gradient set of the target fused image: , assuming that the target fusion image has a total of n pixels, represents the gradient of the i11th pixel, and the value range of i11 is [1, n];

[0096] Take a point s in the target fusion image and select 3x3 pixels around it to construct the gradient covariance matrix. :

[0097] ;in, Represents the covariance based on the gradient of point s; Represents the covariance based on the gradient on the upper left side of point s; Represents the covariance based on the gradient on the left side of point s; Represents the covariance based on the gradient on the lower left side of point s; Represents the covariance based on the gradient on the upper right side of point s; Represents the covariance based on the gradient on the right side of point s; Represents the covariance based on the gradient on the lower right side of point s; Represents the covariance based on the gradient on the upper side of point s; Represents the covariance based on the gradient under point s;

[0098] Based on the gradient covariance matrix , respectively calculate The retained value for each point in :

[0099] ; where r represents The retention value of the corresponding point in ; express The covariance of corresponding points in ; express The maximum value of all covariances in ; express The average of all covariances in ; Represents the covariance matrix of corresponding points The modulus value of Represents the covariance matrix The modulus value; max represents the maximum value symbol;

[0100] Determine the gradient covariance matrix The average value of all retained values ​​in the target feature image is obtained, and the points corresponding to the retained values ​​greater than the average value are regarded as feature points, so as to obtain all feature points on the target feature image, and then obtain the grouped target features.

[0101] In this embodiment, the pixel alignment process refers to aligning all still images in the same group video with a predefined grid or coordinate system. A pixel refers to the smallest display unit in an image, which is one of the components of an image.

[0102] In this embodiment, the multi-feature image is a superimposed image obtained after the same grouped video is processed by pixel points.

[0103] In this embodiment, edge fusion is to seamlessly merge the edges of multiple feature images in the same group to create an overall consistent image.

[0104] In this embodiment, the target fused image is the final image obtained by splicing and edge fusion of all multi-feature images in the same grouped video.

[0105] In this embodiment, the gradient of a pixel point refers to the grayscale change rate at each pixel position in the image, which indicates the degree of change in brightness or color of the image at that position.

[0106] In this embodiment, the retained value is a value representing the characteristics of the pixel point through calculation in the neighborhood around the current pixel point.

[0107] In this embodiment, the feature point refers to the average of all retained values ​​of 8 points in a pixel area. If the retained value of the current pixel is greater than the average value, the current pixel is determined to be a feature point.

[0108] The working principle and beneficial effects of the above technical solution are: by superimposing the pixel points of the images under multiple videos in the group, and splicing and fusing the multi-feature images of multiple videos, the fused image of the current group is extracted, and the grouping target features of the current group are obtained based on the gradient calculation of the group-based fused image, providing a basis for subsequent video splicing.

[0109] Embodiment 6:

[0110] On the basis of the above-mentioned embodiment 1, this embodiment provides a method for splicing videos of a fully mechanized coal mining face. In step 3, the images in the corresponding image sequence to be spliced ​​are standardized according to the grouping target features, including:

[0111] Performing perspective change processing on the multi-feature images under the same group video to obtain a perspective change matrix, substituting the group target features into the perspective change matrix, and obtaining a standard conversion function corresponding to the multi-feature image;

[0112] The standard conversion function is converted into the still images of equal frames in the corresponding captured video in the corresponding images to be spliced ​​in the same group to obtain a first image sequence corresponding to the captured video;

[0113] The first image sequences are grouped together to obtain standard image sequences in corresponding groups.

[0114] In this embodiment, perspective change processing is an image processing technology used to map an image on one plane to another plane and distort and transform the shape according to the perspective relationship so that the original image can be completely projected onto the other plane.

[0115] In this embodiment, the perspective change matrix is ​​a mathematical representation for performing perspective transformation, and is usually a 3x3 matrix. It is a calculation matrix generated based on the characteristic of perspective transformation that maintains linearity.

[0116] In this embodiment, the standard conversion function is a function that substitutes the group target features into the perspective transformation matrix and normalizes it into a specific form. The function represents the perspective change calculation function of the current group and provides a basis for subsequent image conversion.

[0117] In this embodiment, the first image sequence is the first image sequence of the corresponding captured video obtained by performing perspective change processing - standard conversion function processing operation on all still images in the corresponding captured video in the image to be stitched under the same group. That is, after determining the group target features under the corresponding group, the standard conversion function is obtained to include all possible feature situations. For example, the feature of feature point 1 is y1, and the converted function is r1. At this time, the corresponding standard conversion function is: r1=zh(y1,1), zh(y1,1) is the conversion function for the feature of feature point 1 with the feature y1, and then the standard conversion function is used as the basis for image conversion to obtain the first image sequence. That is, the image is first converted, and then the image is numbered to obtain the first image sequence.

[0118] The working principle and beneficial effect of the above technical solution are: by applying multi-feature images to the same perspective transformation matrix, it is possible to ensure that the images remain consistent in space, eliminate the influence of factors such as different perspectives, distortion or scale differences on the images, and make the images more comparable.

[0119] Embodiment 7:

[0120] On the basis of the above-mentioned embodiment 1, this embodiment provides a method for splicing videos of a fully-mechanized coal mining face. In step 4, video splicing is performed based on all standard image sequences to obtain a spliced ​​video of the fully-mechanized coal mining face, including:

[0121] Perform edge matching on the kth standard image of the left video and the kth standard image of the right video that are adjacent to each other in the same group to generate a transformation matrix;

[0122] The k-th standard image of the left video and the original video image corresponding to the k-th standard image of the right video are fused using the transformation matrix to obtain the k-th fused image of the grouped spliced ​​video;

[0123] All frame fusion images under all groups are stitched together to obtain a stitched video of the fully mechanized mining face of the coal mine.

[0124] In this embodiment, edge matching can identify and match the edge or outline of an object in an image. Through edge matching, similar edge features in two adjacent standard images are found and matched.

[0125] In this embodiment, the transformation matrix is ​​a geometric matrix obtained by obtaining edge changes after edge matching of two adjacent standard images.

[0126] The working principle and beneficial effect of the above technical solution is that through edge matching and fusion operations, the discontinuity and misalignment between the left and right images in the spliced ​​video can be reduced, thereby improving the image quality and enabling a clearer observation of the coal mine fully mechanized mining face. Moreover, based on the change matrix, the left and right image conversion is smoother, and the spliced ​​video picture is more coherent.

[0127] Embodiment 8:

[0128] On the basis of the above-mentioned embodiment 1, this embodiment provides a method for splicing videos of a fully mechanized coal mining face, wherein in the process of edge matching the kth standard image of the left video and the kth standard image of the right video that are in an adjacent relationship within the same group, the method further includes:

[0129] Perform image alignment processing on the kth standard image of the left video and the kth standard image of the right video, and screen the edge points of the left edge line and the right edge line at equal distances to construct a point array, and obtain a linear change sequence based on the point array ,in, represents the linear value of the j01th column; Represents the value of the first point in the j01th column of the point array; Represents the value of the second point in the j01th column of the point array; represents the standard value of the j01th column in the point array; m01 represents the number of columns contained in the corresponding point array, and each column contains an edge point pair;

[0130] According to the linear variation sequence, the error coefficient of edge matching is determined:

[0131] ;in, Represents the corresponding error coefficient; m02 represents the corresponding point array The number of items not within the preset range; A difference function representing linear values ​​that are not within a preset range; represents the difference function of the corresponding point array; Indicates the j02th linear value that is not within the preset range;

[0132] Based on the error coefficient, the edge point pairs that are not within the preset range are adjusted to achieve subsequent edge matching.

[0133] In this embodiment, the equidistant edge points refer to key pixel points that are distributed at equidistant intervals on the left edge line and the right edge line in two adjacent still images.

[0134] In this embodiment, the point array refers to a sequence of a group of pixel points obtained by screening equidistant edge points on the left edge line and the right edge line in two adjacent still images.

[0135] In this embodiment, the linear change sequence refers to an array of corresponding points obtained from two images. There is a linear change relationship between the corresponding two edge points, so the linear change relationship between the corresponding two edge points of all point arrays constitutes a linear change sequence.

[0136] In this embodiment, the linear value indicates the degree to which two corresponding edge points in the linear change sequence gradually change along a linear relationship.

[0137] In this embodiment, an edge point pair is a point group consisting of two edge points that represent a corresponding relationship in the point array.

[0138] In this embodiment, the error coefficient is a measure of the error between the result of quantifying the matching edge point and the real edge.

[0139] In this embodiment, the difference function is a function representing the difference between two linear values.

[0140] The working principle and beneficial effects of the above technical solution are: through edge matching and error calculation, the gaps at the splicing position of the left and right videos can be effectively eliminated, making the spliced ​​video visually smoother and more continuous, avoiding information faults and visual incoherence caused by splicing gaps, and reducing information loss problems caused by video discontinuity in video monitoring, operation guidance and troubleshooting.

[0141] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for splicing videos of fully mechanized coal mining faces, characterized in that: include: Step 1: Collect videos of the fully-mechanized mining face of the coal mine according to the pre-deployed multi-eye camera, and group the collected videos according to the deployment positions of the pre-deployed multi-eye camera; Step 2: Frame extraction is performed on each grouped video, and each frame extracted image is numbered to obtain a sequence of images to be spliced ​​for each corresponding group, wherein the number of frames extracted from each grouped video is the same; Step 3: establishing group target features at the deployment position of the multi-camera, and performing standardization processing on the images in the corresponding image sequence to be stitched according to the group target features to obtain a standard image; Step 4: Perform video stitching based on all standard images to obtain a stitched video of the fully mechanized mining face of the coal mine; In step 2, each grouped video is frame extracted, and each frame extracted image is numbered to obtain a sequence of images to be spliced ​​for each corresponding group, including: Decoding each grouped video into an original image sequence using a video decoder, wherein the original image sequence is composed of unequal amounts of image frames under multiple acquired videos; Determine the average duration according to the video duration of each collected video in each grouped video, and determine the number of extracted frames of the current group according to the average-extracted frame table; Extracting a plurality of images in the same collected video according to the number of extracted frames from the original image sequence under the same grouped video at the time interval of the corresponding collected video to obtain a still image, preprocessing the extracted still image to obtain an initial image sequence to be spliced, wherein the initial image sequence to be spliced ​​is composed of equal image frames under multiple collected videos; Number each frame of the initial image sequence to be stitched according to the time logical sequence to obtain the image sequence to be stitched of the target acquisition video; In step 3, the grouped target features at the deployment position of the multi-camera are established, including: Performing pixel alignment processing on still images extracted from the same acquisition video in the same grouped video to obtain multi-feature images under the corresponding acquisition video, wherein each acquisition video corresponds to a multi-feature image; All multi-feature images under the same group video are spliced ​​and edge-fused to obtain the target fused image; Calculate the gradient of each pixel in the target fused image and construct a gradient set of the target fused image: , assuming that the target fusion image has a total of n pixels, represents the gradient of the i11th pixel, and the value range of i11 is [1, n]; Take a point s in the target fusion image and select 3x3 pixels around it to construct the gradient covariance matrix. : ;in, Represents the covariance based on the gradient of point s; Represents the covariance based on the gradient on the upper left side of point s; Represents the covariance based on the gradient on the left side of point s; Represents the covariance based on the gradient on the lower left side of point s; Represents the covariance based on the gradient on the upper right side of point s; Represents the covariance based on the gradient on the right side of point s; Represents the covariance based on the gradient on the lower right side of point s; Represents the covariance based on the gradient on the upper side of point s; Represents the covariance based on the gradient under point s; Based on the gradient covariance matrix , respectively calculated The retained value for each point in : ; where r represents The retention value of the corresponding point in ; express The covariance of corresponding points in ; express The maximum value of all covariances in ; express The average of all covariances in ; Represents the covariance matrix of corresponding points The modulus of Represents the covariance matrix The modulus value; max represents the maximum value symbol; Determine the gradient covariance matrix The average value of all retained values ​​in the target feature image is obtained, and the points corresponding to the retained values ​​greater than the average value are regarded as feature points, thereby obtaining all feature points on the target feature image, and then obtaining the grouped target features; In step 4, video stitching is performed based on all standard image sequences to obtain a stitched video of the fully mechanized mining face of the coal mine, including: Perform edge matching on the kth standard image of the left video and the kth standard image of the right video that are adjacent to each other in the same group to generate a transformation matrix; The k-th standard image of the left video and the original video image corresponding to the k-th standard image of the right video are fused using the transformation matrix to obtain the k-th fused image of the grouped spliced ​​video; All frame fusion images in all groups are stitched together to obtain a stitched video of the fully mechanized mining face of the coal mine; The process of edge matching the kth standard image of the left video and the kth standard image of the right video which are adjacent to each other in the same group also includes: Perform image alignment processing on the kth standard image of the left video and the kth standard image of the right video, and screen the edge points of the left edge line and the right edge line at equal distances to construct a point array, and obtain a linear change sequence based on the point array ,in, represents the linear value of the j01th column; Represents the value of the first point in the j01th column of the point array; Represents the value of the second point in the j01th column of the point array; represents the standard value of the j01th column in the point array; m01 represents the number of columns contained in the corresponding point array, and each column contains an edge point pair; According to the linear variation sequence, the error coefficient of edge matching is determined: ;in, Represents the corresponding error coefficient; m02 represents the corresponding point array The number of items not within the preset range; A difference function representing linear values ​​that are not within a preset range; represents the difference function of the corresponding point array; Indicates the j02th linear value that is not within the preset range; Based on the error coefficient, the edge point pairs that are not within the preset range are adjusted to achieve subsequent edge matching.

2. A method for splicing videos of fully mechanized coal mining faces according to claim 1, characterized in that: In step 1, video collection of the fully mechanized mining face of the coal mine is performed based on the pre-deployed multi-eye camera, including: Based on the basic environmental conditions of the fully-mechanized mining face of the coal mine and the area that needs to be covered and photographed, the initial deployment position and shooting angle of the multi-eye camera are planned, and a corresponding relationship between the initial deployment position and the number of the multi-eye camera at each initial deployment position is established; Multi-cameras deployed based on number correspondence are used to capture video of the fully mechanized mining face in coal mines.

3. A method for splicing videos of fully mechanized coal mining faces according to claim 2, characterized in that: In step 1, the collected videos are processed into video groups, including: All collected videos are grouped and processed according to the number correspondence between each working area of ​​the fully mechanized mining face of the coal mine; According to the grouping processing result, the grouped video of the corresponding working area is obtained.

4. A method for splicing videos of fully mechanized coal mining faces according to claim 1, characterized in that: In step 3, the images in the corresponding sequence of images to be stitched are standardized according to the grouping target features, including: Perform perspective change processing on the multi-feature images under the same group video to obtain a perspective change matrix, substitute the group target features into the perspective change matrix, and obtain a standard conversion function corresponding to the multi-feature image; The standard conversion function is converted into the still images of equal frames in the corresponding captured video in the corresponding images to be spliced ​​in the same group to obtain a first image sequence corresponding to the captured video; The first image sequences are grouped together to obtain standard image sequences in corresponding groups.

Images