Working face dynamic machine-following panoramic video stitching method and device

Abstract

The invention provides a working face dynamic machine-following panoramic video stitching method and device. The method comprises the following steps: calling an original video stream of a target camera; wherein the target cameras are a plurality of cameras which are determined according to the position information of the coal mining machine and are within a preset range with the coal mining machine as the center; mapping the plurality of original video streams to a global coordinate system of the same working face according to the position information through a dynamic coordinate system conversion algorithm to obtain a machine-following panoramic video stream; performing local correction on the machine-following panoramic video stream through a local deformation correction algorithm to obtain a corrected video stream; fusing the overlapping areas of the adjacent cameras in the machine-following panoramic video stream through a multi-band fusion algorithm to obtain a fused video stream; the dynamic panoramic video which takes the coal mining machine as the center and continuously changes along with the movement of the coal mining machine is generated, and the remote control operation efficiency is remarkably improved. And the visual quality and naturalness of the dynamic panoramic video are ensured.

Description

Technical Field

[0001] This invention relates to the field of coal mine monitoring technology, and in particular to a method and device for dynamic panoramic video stitching of working faces. Background Technology

[0002] In underground coal mines, to improve the ability to perceive working conditions, it is necessary to combine the images from multiple cameras into a panoramic view. Currently, common solutions can be categorized into two types.

[0003] One approach is a static stitching scheme based on fixed markers. This scheme involves placing specific markers, such as dustproof signs, underground and capturing video streams containing these markers using cameras. The video stitching system identifies and detects these pre-set markers and uses dynamic adjustment components to adjust and stitch the images. Essentially, this method relies on the identification of fixed reference points to align and fuse the images.

[0004] Another approach is a static stitching scheme based on image feature matching. This scheme does not rely on specific markers but utilizes image processing techniques. The system acquires video signals from various sources through the video encoding module, and then the image stitching control module uses a multi-scale feature point detection and matching algorithm to find natural image features, such as textures and corners, within the overlapping areas of adjacent camera feeds. Based on these features, the images are stitched together to ultimately generate a more comprehensive static panoramic image, which is then displayed on a computer.

[0005] The common feature of the above solutions is that their stitching goal is to generate a fixed, static panoramic image covering a specific area, such as the entire working face or the vicinity of the coal mining machine. The cameras are typically fixed in place, and their viewing angle and coverage area are determined and remain unchanged after system deployment.

[0006] Although the above solution achieves basic video stitching functionality, in practical applications, especially in the context of control centers for remote, intelligent, and unmanned mining, the following problems exist.

[0007] The aforementioned solution generates a static panoramic view, which cannot track the movement of the coal mining machine in real time and continuously. When the coal mining machine moves to different positions, its working area may only occupy a small part in the static panoramic view. The remote operator cannot obtain a continuously changing panoramic view that always centers on the coal mining machine and follows its position. This results in a fragmented and disjointed visual experience, failing to provide a coherent and immersive visual background for remote operation, and seriously affecting the overall grasp of the real-time working status of the coal mining machine.

[0008] In addition, due to the lack of sophisticated image processing, the current solution is prone to image perspective distortion due to differences in camera perspective during stitching, and the edge blending in overlapping areas is unnatural, with obvious seams or ghosting, affecting the visualization effect and judgment accuracy. Summary of the Invention

[0009] This invention provides a method for stitching dynamic panoramic videos of the coal mining face, addressing the shortcomings of existing static panoramic images that cannot continuously track the movement of the coal mining machine in real time, and lacking fine image processing. It generates a dynamic panoramic video centered on the coal mining machine that continuously changes as it moves, significantly improving the immersive experience and operational efficiency of remote control. Furthermore, it effectively eliminates image distortion and stitching seams caused by differences in camera angle and perspective, ensuring the visual quality and naturalness of the dynamic panoramic video.

[0010] This invention provides a method for dynamic panoramic video stitching of a working face, comprising the following steps.

[0011] Call the raw video stream of the target camera; where the target camera is a set of multiple cameras within a preset range centered on the coal mining machine, determined based on the location information of the coal mining machine. By using a dynamic coordinate system transformation algorithm, multiple original video streams are mapped to the same global coordinate system of the working surface based on the position information to obtain a follow-up panoramic video stream; The local deformation correction algorithm is used to perform local correction on the following panoramic video stream to obtain the corrected video stream; By using a multi-band fusion algorithm, the overlapping areas of adjacent cameras in the panoramic video stream are fused to obtain the fused video stream.

[0012] According to the present invention, a method for dynamic panoramic video stitching of a working face is provided. This method uses a dynamic coordinate system transformation algorithm to map multiple original video streams to the same global coordinate system of the working face based on position information, thereby obtaining a panoramic video stream. The method includes: For multiple raw video streams, feature point matching with a fixed reference object is performed to obtain the matching results; Based on the location information, the observation area corresponding to each original video stream is calculated to obtain the camera's observation area; By using a dynamic coordinate system transformation algorithm, multiple original video streams are mapped to the same global coordinate system of the working surface based on the matching results and the camera's observation area, thus obtaining a panoramic video stream for following the camera.

[0013] According to the present invention, a method for dynamic panoramic video stitching of a working face is provided, which performs local correction on the panoramic video stream using a local deformation correction algorithm to obtain a corrected video stream, including: By using a local deformation correction algorithm, the linear structural features in the live panoramic video stream are identified, and linear feature points are obtained. A mesh deformation model is constructed using straight line feature points as geometric constraints. The local deformation correction algorithm is used to adjust the position of the mesh vertices in the mesh deformation model to obtain the corrected video stream.

[0014] According to the present invention, a method for dynamic following panoramic video stitching of a working face is provided, which uses a multi-band fusion algorithm to fuse overlapping areas of adjacent cameras in a following panoramic video stream to obtain a fused video stream, including: By using a multi-band fusion algorithm, the overlapping areas of adjacent cameras in the following panoramic video stream are weighted to obtain a weighted video stream; The weighted video stream is decomposed using the Laplacian pyramid to obtain multiple different frequency bands; For each frequency band, the weighted video streams are fused using a multi-band fusion algorithm to obtain the fused video stream.

[0015] The method for dynamic panoramic video stitching of a work surface according to the present invention further includes: Determine the positions of multiple cameras in the merged video stream; Add clickable icons and corresponding click commands to the camera position to obtain the processed video stream; Output and display the processed video stream.

[0016] According to the present invention, a method for dynamic panoramic video stitching of a work surface, after outputting and displaying the fused video stream, further includes: When a click command is received for a clickable icon, the raw video stream from the corresponding camera is displayed.

[0017] According to the present invention, a method for dynamic panoramic video stitching of a working face is provided, which outputs and displays the fused video stream, including: Real-time acquisition of audio data from the coal mining machine's working area; The audio data and the fused video stream are synchronized and merged to obtain an audio-video stream. Output audio and video streams.

[0018] The present invention also provides a working face dynamic following panoramic video stitching device, comprising the following modules: The location and video acquisition module is used to call the raw video stream of the target camera; wherein, the target camera is a number of cameras within a preset range centered on the coal mining machine, determined based on the location information of the coal mining machine; The coordinate system mapping module is used to map multiple original video streams to the same global coordinate system of the working surface based on the position information through a dynamic coordinate system transformation algorithm, so as to obtain a follow-up panoramic video stream; The video correction module is used to perform local correction on the following panoramic video stream using a local deformation correction algorithm to obtain a corrected video stream; The video fusion module is used to fuse the overlapping areas of adjacent cameras in the following panoramic video stream using a multi-band fusion algorithm to obtain a fused video stream.

[0019] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the working face dynamic following panoramic video stitching method as described above.

[0020] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the working face dynamic following panoramic video stitching method as described above.

[0021] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the working surface dynamic following panoramic video stitching method as described above.

[0022] This invention provides a method and apparatus for dynamic panoramic video stitching of a working face. It utilizes the original video streams from target cameras, which are multiple cameras within a preset range centered on the coal mining machine, determined based on the machine's location information. A dynamic coordinate system transformation algorithm maps these original video streams to the same global coordinate system of the working face, resulting in a panoramic video stream. A local deformation correction algorithm locally corrects the panoramic video stream, yielding a corrected video stream. A multi-band fusion algorithm fuses overlapping areas of adjacent cameras within the panoramic video stream, resulting in a fused video stream. Compared to existing static panoramic images that cannot continuously track the movement of the coal mining machine in real time and lack fine image processing, this invention generates a dynamic panoramic video centered on the coal mining machine that continuously changes with its movement, significantly improving the immersive experience and operational efficiency of remote control. Furthermore, it effectively eliminates image distortion and stitching seams caused by differences in camera angles and perspective, ensuring the visual quality and naturalness of the dynamic panoramic video. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating the dynamic panoramic video stitching method for the working face provided by the present invention.

[0025] Figure 2 This is a structural schematic diagram of the working face dynamic following panoramic video stitching device provided by the present invention.

[0026] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0028] The following is combined Figures 1 to 3 This invention is described.

[0029] The system architecture design of this invention requires the construction of a layered distributed system framework, including a perception layer, a transmission layer, a processing layer, and an application layer. The perception layer deploys a high-definition camera array, positioning sensors, and microphones; the transmission layer adopts an industrial Ethernet + 5G / WiFi6 wireless redundancy approach; the processing layer is configured with edge computing nodes to achieve real-time video processing; and the application layer provides a human-machine interface and a data analysis platform.

[0030] Cameras: One explosion-proof high-definition camera is installed for every three hydraulic supports. The lens type is optimized according to the location. A standard-angle camera is used in the middle of the working face, and a wide-angle camera is used in the triangular coal area.

[0031] Positioning System: A high-precision UWB positioning device is installed on the body of the coal mining machine, which is combined with the inertial navigation system of the coal mining machine to ensure positioning accuracy of ±10cm.

[0032] Microphones: Explosion-proof microphones are deployed near each camera and key components of the coal mining machine.

[0033] Figure 1 This is one of the flowcharts illustrating the dynamic panoramic video stitching method for the working face provided by the present invention, such as... Figure 1 As shown, the method includes the following: Step 101: Call the original video stream of the target camera; wherein, the target camera is a number of cameras within a preset range centered on the coal mining machine, determined based on the location information of the coal mining machine.

[0034] Step 102: Using a dynamic coordinate system transformation algorithm, multiple original video streams are mapped to the same global coordinate system of the working surface based on the position information to obtain the following panoramic video stream.

[0035] In step 102 above, the upper-level application platform acquires the real-time position information of the coal mining machine, including coordinates X, Y, Z and attitude angle θ. Based on this position, it dynamically determines the set of surrounding cameras that need to be called. For example, cameras within the range of N sets of supports in front of and behind the coal mining machine.

[0036] Optionally, step 102 includes steps A1 to A3: Step A1: Perform feature point matching on multiple original video streams with a fixed reference object to obtain the matching results.

[0037] Step A2: Calculate the observation area corresponding to each original video stream based on the location information to obtain the camera's observation area.

[0038] Step A3: Using a dynamic coordinate system transformation algorithm, based on the matching results and the camera's observation area, multiple original video streams are mapped to the same global coordinate system on the working surface to obtain the following panoramic video stream.

[0039] In steps A1 to A3 above, a coordinate system transformation algorithm based on feature point matching is employed. First, the SIFT / SURF algorithm is used to extract feature points from fixed reference objects in the field of view of each camera, such as the support outline and the connection point of the scraper conveyor. Then, based on the real-time position of the coal mining machine, the theoretical projection position of each camera's video frame in the unified working face global coordinate system is calculated. Finally, the RANSAC algorithm is used to eliminate mismatched points, and the homography matrix is ​​solved to achieve accurate mapping of each video stream to the global coordinate system.

[0040] Step 103: Perform local correction on the following panoramic video stream using a local deformation correction algorithm to obtain the corrected video stream.

[0041] Optionally, step 103 includes steps B1 to B3: Step B1: Using a local deformation correction algorithm, identify the straight line structural features in the following panoramic video stream to obtain straight line feature points.

[0042] Step B2: Construct a mesh deformation model using straight line feature points as geometric constraints.

[0043] Step B3: Adjust the position of the mesh vertices in the mesh deformation model using a local deformation correction algorithm to obtain the corrected video stream.

[0044] In steps B1 to B3 above, a mesh-based local deformation correction algorithm is used to address the perspective distortion present in the mapped image. By detecting straight line features in the image, such as the support columns and the edge lines of the scraper conveyor chute, a mesh deformation model is constructed, and nonlinear optimization correction is performed.

[0045] Step 104: Using a multi-band fusion algorithm, the overlapping areas of adjacent cameras in the following panoramic video stream are fused to obtain the fused video stream.

[0046] Optionally, step 104 includes steps C1 to C3: Step C1: Using a multi-band fusion algorithm, the overlapping areas of adjacent cameras in the following panoramic video stream are weighted to obtain a weighted video stream.

[0047] Step C2: Decompose the weighted video stream using the Laplacian pyramid to obtain multiple different frequency bands.

[0048] Step C3: For each frequency band, the weighted video stream is fused using a multi-band fusion algorithm to obtain the fused video stream.

[0049] In steps C1 to C3 above, a multi-band blending algorithm is used in the overlapping areas of adjacent camera videos. First, brightness differences are processed through adaptive weighting, and then multi-scale fusion is performed using Laplacian pyramid decomposition to eliminate seams and ensure a natural transition.

[0050] With the above algorithm combination, the system design goal is to achieve a stitching error of ≤25 pixels (at 1080P resolution).

[0051] This invention effectively eliminates image distortion and stitching seams caused by differences in camera viewpoint and perspective through specific deformation correction and edge fusion algorithms, ensuring the visual quality and naturalness of dynamic panoramic video, making the stitched image closer to the real scene, and reducing operator visual fatigue and misjudgment.

[0052] The system supports the dynamic stitching of all camera videos across the entire working face, or segments as needed, to generate a "scroll-like" panoramic video covering the entire coal mining face.

[0053] Operators can select the automatic cruise mode on the client interface. The system will automatically switch or stitch together videos from different areas according to a preset path, such as from front to back of the camera or by speed. Operators can also manually select the area of ​​interest.

[0054] This invention solves the problem of static and fragmented stitched images in existing technologies by introducing real-time position-driven video stitching of the coal mining machine. It can generate a dynamic panoramic video centered on the coal mining machine and continuously changing as it moves, providing remote operators with unprecedented continuous and global visual perception capabilities, and significantly improving the immersive experience and operational efficiency of remote control.

[0055] Optionally, after step 104, steps D1 to D3 are also included: Step D1: Determine the positions of multiple cameras in the merged video stream.

[0056] Step D2: Add a clickable icon and corresponding click command to the camera position to obtain the processed video stream.

[0057] Step D3: Output and display the processed video stream.

[0058] In steps D1 to D3 above, clickable icons, such as small camera icons, are dynamically overlaid on the panoramic video or work surface topology map displayed on the client.

[0059] When an operator clicks an icon with the mouse, the client sends a request to the processing layer. The system immediately retrieves the original high-definition video stream or a partially optimized and stitched video from the camera at that location, which can be displayed in a separate window or in picture-in-picture mode. There is no need to remember the camera's number or location.

[0060] This invention can also enable multi-view video systems to call up camera videos from specific locations on the application interface, thus complementing panoramic video.

[0061] Specifically, in special working conditions such as coal cutting at triangular coal seams, the system automatically activates or prompts operators to call up the video from the dedicated wide-angle camera deployed in the end area.

[0062] This special perspective video can be displayed in picture-in-picture or split-screen mode with a partial panoramic video of the current coal mining machine location, providing remote operators with more comprehensive information on cutting posture and coal wall status.

[0063] This invention introduces a point-and-view interactive method. Operators can manually switch between spliced ​​images of different areas, and the system supports the installation of a single camera on each support frame to form a "scroll"-like video coverage. Dedicated point cameras are deployed, especially in complex areas such as the triangular coalfield, complementing the panoramic video and providing precise visual guidance for special operating scenarios. This solves the problems of operators needing to remember camera numbers and locations and the cumbersome manual switching of images. Through intuitive graphical interaction, operators can quickly and accurately view detailed images from any location, achieving seamless switching between global monitoring and local detail viewing, greatly improving the system's usability and operational efficiency.

[0064] The design combines multi-view and panoramic video with camera deployment at specific locations to enhance the accuracy of remote control operations.

[0065] Optionally, after step D3, step E1 is also included: Step E1: Upon receiving a click command for a clickable icon, display the raw video stream from the corresponding camera.

[0066] Optionally, step D3 includes steps F1 to F3: Step F1: Acquire audio data of the coal mining machine's working area in real time.

[0067] Step F2: Perform audio-video synchronization alignment and merging processing on the audio data and the fused video stream to obtain the audio-video stream.

[0068] Step F3: Output audio and video streams.

[0069] In steps F1 to F3 above, the audio signals collected by each microphone are transmitted through the same transmission channel as the video. Hardware timestamps are used for strict time synchronization of the audio and video streams. At the processing layer, the audio and video streams at corresponding positions are aligned at the frame level and merged into a single audio and video stream for output. The system design goal is to achieve an audio-visual synchronization error of ≤50ms.

[0070] This invention adds an audio dimension. By synchronously fusing on-site audio and video streams, it overcomes the limitations of existing systems that only provide visual information. Operators in the control center can simultaneously acquire on-site acoustic information, such as abnormal equipment noises, achieving multimodal perception and enhancing the ability to comprehensively assess production status and provide an immersive experience.

[0071] This invention can also realize the storage and playback of historical videos. The historical video playback function supports the storage and retrieval of single-channel video, dynamically stitched video, and full-area stitched video.

[0072] A distributed storage architecture, such as Ceph or HDFS, is adopted. Three independent storage paths are designed for the three types of video: Single-camera video: the raw video stream; Dynamic panoramic video following the machine: indexed by timestamp and coal mining machine location; Full-face stitched video: indexed by working face area and time.

[0073] The client supports searching and replaying the above three types of videos by time, location, coal mining machine coordinates, and camera number, among other criteria. It also supports playback speed adjustment and keyframe localization.

[0074] The historical video playback function supports three types of video storage and retrieval, improving the ability to trace the operation process.

[0075] This invention provides a method for dynamic panoramic video stitching of a working face, which calls upon the original video streams of target cameras. The target cameras are multiple cameras within a preset range centered on the coal mining machine, determined based on the machine's location information. A dynamic coordinate system transformation algorithm maps the multiple original video streams to the same global coordinate system of the working face, resulting in a panoramic video stream. A local deformation correction algorithm locally corrects the panoramic video stream, yielding a corrected video stream. A multi-band fusion algorithm fuses overlapping areas of adjacent cameras in the panoramic video stream, resulting in a fused video stream. The fused video stream is then output and displayed. Compared to existing static panoramic images that cannot continuously track the movement of the coal mining machine in real time and lack fine image processing, this invention generates a dynamic panoramic video centered on the coal mining machine that continuously changes with its movement, significantly improving the immersive experience and operational efficiency of remote control. Furthermore, it effectively eliminates image distortion and stitching seams caused by differences in camera angles and perspective, ensuring the visual quality and naturalness of the dynamic panoramic video.

[0076] The following describes the working face dynamic following panoramic video stitching device provided by the present invention. The working face dynamic following panoramic video stitching device described below and the working face dynamic following panoramic video stitching method described above can be referred to in correspondence.

[0077] like Figure 2 As shown, the present invention provides a dynamic panoramic video stitching device for working surfaces, comprising the following modules: The location and video acquisition module 201 is used to call the original video stream of the target camera; wherein, the target camera is a plurality of cameras within a preset range centered on the coal mining machine, determined according to the location information of the coal mining machine.

[0078] The mapping coordinate system module 202 is used to map multiple original video streams to the same working surface global coordinate system based on the position information through a dynamic coordinate system transformation algorithm, so as to obtain a follow-up panoramic video stream.

[0079] The video correction module 203 is used to perform local correction on the following panoramic video stream using a local deformation correction algorithm to obtain the corrected video stream.

[0080] The video fusion module 204 is used to fuse the overlapping areas of adjacent cameras in the following panoramic video stream using a multi-band fusion algorithm to obtain a fused video stream.

[0081] The present invention provides a dynamic panoramic video stitching device for the working face, which utilizes the original video streams from target cameras. These target cameras are multiple cameras within a preset range centered on the coal mining machine, determined based on the machine's location information. A dynamic coordinate system transformation algorithm maps these multiple original video streams to the same global coordinate system of the working face, resulting in a panoramic video stream. A local deformation correction algorithm performs local correction on the panoramic video stream, yielding a corrected video stream. A multi-band fusion algorithm fuses overlapping areas of adjacent cameras within the panoramic video stream, resulting in a fused video stream. Compared to existing static panoramic images that cannot continuously track the movement of the coal mining machine in real time and lack fine image processing, this invention generates a dynamic panoramic video centered on the coal mining machine that continuously changes with its movement, significantly improving the immersive experience and operational efficiency of remote control. Furthermore, it effectively eliminates image distortion and stitching seams caused by differences in camera angles and perspective, ensuring the visual quality and naturalness of the dynamic panoramic video.

[0082] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840. The processor 810, communications interface 820, and memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions from the memory 830 to execute a dynamic, follow-up panoramic video stitching method for the working surface.

[0083] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0084] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the working surface dynamic following panoramic video stitching method provided by the above methods.

[0085] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the working-face dynamic following panoramic video stitching method provided by the above methods.

[0086] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0087] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for dynamic panoramic video stitching of a work surface, characterized in that, include: The original video stream of the target camera is invoked; wherein, the target camera is a plurality of cameras within a preset range centered on the coal mining machine, determined based on the location information of the coal mining machine; By using a dynamic coordinate system transformation algorithm, multiple original video streams are mapped to the same global coordinate system of the working surface based on the location information to obtain a follow-up panoramic video stream; The local deformation correction algorithm is used to locally correct the following panoramic video stream to obtain the corrected video stream; The overlapping areas of adjacent cameras in the following panoramic video stream are fused using a multi-band fusion algorithm to obtain a fused video stream.

2. The method for dynamic panoramic video stitching of the working face according to claim 1, characterized in that, The step of mapping multiple original video streams to the same global coordinate system of the working surface using a dynamic coordinate system transformation algorithm based on the position information to obtain a follow-up panoramic video stream includes: For multiple original video streams, feature point matching with a fixed reference object is performed to obtain matching results; Based on the location information, the observation area corresponding to each original video stream is calculated to obtain the camera observation area; By using a dynamic coordinate system transformation algorithm, based on the matching results and the camera's observation area, multiple original video streams are mapped to the same global coordinate system on the working surface to obtain a follow-up panoramic video stream.

3. The method for dynamic panoramic video stitching of the working face according to claim 1, characterized in that, The step of performing local deformation correction on the following panoramic video stream using a local deformation correction algorithm to obtain a corrected video stream includes: By using a local deformation correction algorithm, the straight line structural features in the following panoramic video stream are identified, and straight line feature points are obtained; A mesh deformation model is constructed using the aforementioned straight line feature points as geometric constraints; The mesh vertex positions are adjusted in the mesh deformation model using a local deformation correction algorithm to obtain the corrected video stream.

4. The method for dynamic panoramic video stitching of the working face according to claim 1, characterized in that, The step of fusing overlapping areas of adjacent cameras in the following panoramic video stream using a multi-band fusion algorithm to obtain a fused video stream includes: The overlapping areas of adjacent cameras in the following panoramic video stream are weighted by a multi-band fusion algorithm to obtain a weighted video stream. The weighted video stream is decomposed using the Laplacian pyramid to obtain multiple different frequency bands; For each frequency band, the weighted video stream is fused using a multi-band fusion algorithm to obtain a fused video stream.

5. The method for dynamic panoramic video stitching of the working face according to claim 1, characterized in that, Also includes: Determine the positions of multiple cameras in the fused video stream; Clickable icons and corresponding click commands are added to the camera location to obtain the processed video stream; Output and display the processed video stream.

6. The method for dynamic panoramic video stitching of the working face according to claim 5, characterized in that, After outputting and displaying the fused video stream, the method further includes: Upon receiving a click instruction from the clickable icon, the raw video stream from the corresponding camera is displayed.

7. The method for dynamic panoramic video stitching of the working face according to claim 5, characterized in that, The output and display of the fused video stream includes: Real-time acquisition of audio data from the coal mining machine's working area; The audio data and the fused video stream are then subjected to audio-video synchronization alignment and merging processing to obtain an audio-video stream. Output the audio and video streams.

8. A dynamic panoramic video stitching device for a work surface, characterized in that, include: The location and video acquisition module is used to call the original video stream of the target camera; wherein, the target camera is a plurality of cameras within a preset range centered on the coal mining machine, determined based on the location information of the coal mining machine; The coordinate system mapping module is used to map multiple original video streams to the same global coordinate system of the working surface based on the position information using a dynamic coordinate system transformation algorithm, so as to obtain a follow-up panoramic video stream; The video correction module is used to perform local correction on the following panoramic video stream using a local deformation correction algorithm to obtain a corrected video stream; The video fusion module is used to fuse the overlapping areas of adjacent cameras in the following panoramic video stream using a multi-band fusion algorithm to obtain a fused video stream.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the working face dynamic following panoramic video stitching method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the working face dynamic following panoramic video stitching method as described in any one of claims 1 to 7.

Images