Progressive panoramic image stitching method and device for large equipment of tunneling face

Abstract

This disclosure relates to a progressive panoramic image stitching method and apparatus for large equipment in a tunneling face. The method includes: acquiring an original image sequence of the large equipment in the tunneling face; performing noise reduction processing on each image in the original image sequence to obtain a denoised image sequence; selecting one image from the denoised image sequence as a reference image; stitching the images sequentially in the direction of increasing sequence index of the denoised image sequence, starting from the reference image, to obtain a first stitched image; stitching the images sequentially in the direction of decreasing sequence index of the denoised image sequence, starting from the reference image, to obtain a second stitched image; and combining the first stitched image with the second stitched image to generate a panoramic image. This solution achieves high-precision, low-distortion panoramic stitching of images surrounding large-size equipment.

Description

Technical Field

[0001] This disclosure relates to the field of coal mining technology, and in particular to a progressive panoramic image stitching method and apparatus for large equipment in tunneling faces. Background Technology

[0002] In related technologies, in underground coal mine tunneling faces, large equipment (such as roadheaders and anchor transporters) requires the acquisition of multiple surround images to stitch together a complete panoramic view due to their massive size. However, uneven lighting and high dust concentration in the underground environment lead to severe image noise. Furthermore, due to the large size of the equipment and the wide span of the image sequence, if a direct stitching method is used, the difference in perspective between distant images will cause significant errors in the calculated homography matrix, resulting in severe distortion and feature point warping in the stitched result, making it difficult to meet practical application requirements. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides a progressive panoramic image stitching method and apparatus for large equipment in tunneling faces.

[0004] According to a first aspect of the present disclosure, a progressive panoramic image stitching method for large equipment in a tunneling face is provided, comprising:

[0005] Acquire raw image sequences of large equipment at the tunneling face; the raw image sequences are arranged in chronological or spatial order of acquisition. Each image in the original image sequence is subjected to noise reduction processing to obtain a noise-reduced image sequence; One image is selected from the denoised image sequence as the reference image; Starting from the reference image, the images are stitched together sequentially in the direction of increasing the sequence index of the denoised image sequence to obtain the first stitched image; Starting from the reference image, the images are stitched together sequentially in the direction of decreasing sequence index of the denoised image sequence to obtain the second stitched image. The first stitched image and the second stitched image are merged to generate a panoramic image.

[0006] According to a second aspect of the present disclosure, a progressive panoramic image stitching device for large equipment in a tunneling face is provided, comprising: An acquisition unit is used to acquire the original image sequence of large equipment at the tunneling face; the original image sequence is arranged in order of acquisition time or space. A noise reduction unit is used to perform noise reduction processing on each image in the original image sequence to obtain a noise-reduced image sequence; The selection unit is used to select one image from the denoised image sequence as a reference image; The first stitching unit is used to stitch together the image sequentially in the direction of increasing the sequence index of the denoised image sequence, starting from the reference image, to obtain the first stitched image. The second stitching unit is used to stitch together the reference image in the direction of decreasing sequence index of the denoised image sequence to obtain the second stitched image. The fusion unit is used to fuse the first stitched image and the second stitched image to generate a panoramic image.

[0007] According to a third aspect of the present disclosure, an electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as described in any one of the first aspects.

[0008] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of the first aspects.

[0009] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method as described in any one of the first aspects.

[0010] This invention provides a progressive panoramic image stitching method for large equipment in tunneling faces. It acquires a sequence of original images arranged chronologically or spatially, and performs noise reduction on each image, effectively eliminating noise interference from the high dust and low illumination environment underground, providing a high-quality image data foundation for subsequent stitching. By selecting one image from the denoised image sequence as a reference image, a unified stitching reference system is established. Based on this, stitching is performed sequentially in both the increasing and decreasing index directions, starting from the reference image. This avoids excessive distortion and feature point distortion caused by direct matching of distant images, achieving orderly and progressive alignment of the surrounding images of large equipment. Finally, by fusing the first and second stitched images, a complete panoramic image is generated. This enables high-precision, low-distortion panoramic stitching of large equipment surrounding images under complex working conditions in tunneling faces, providing clear and reliable image data support for underground visual tasks and remote monitoring.

[0011] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0013] Figure 1 This is a flowchart illustrating a progressive panoramic image stitching method for large equipment in a tunneling face, according to an exemplary embodiment.

[0014] Figure 2 This is a block diagram illustrating a progressive panoramic image stitching device for large equipment at a tunneling face, according to an exemplary embodiment.

[0015] Figure 3 This is a block diagram illustrating an apparatus for a progressive panoramic image stitching method for large equipment at a tunneling face, according to an exemplary embodiment. Detailed Implementation

[0016] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0017] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the present disclosure. The singular forms “a” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0018] It should be understood that although the terms first, second, third, etc., may be used to describe various information in embodiments of this disclosure, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of embodiments of this disclosure, and similarly, second information may also be referred to as first information. Depending on the context, the words “if” and “suppose” as used herein may be interpreted as “when”, “when”, or “in response to a determination”.

[0019] Furthermore, various forms of processes shown in the embodiments of this disclosure can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and no limitation is imposed herein.

[0020] It should be noted that the collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0021] Figure 1 This is a flowchart illustrating a progressive panoramic image stitching method for large equipment in a tunneling face, according to an exemplary embodiment. Figure 1 As shown, it should be noted that the progressive panoramic image stitching method for large equipment in a tunneling face, as described in this embodiment, is applied in a progressive panoramic image stitching device for large equipment in a tunneling face. Figure 1 As shown, the method may include the following steps: Step 101: Obtain the original image sequence of large equipment at the tunneling face.

[0022] The original image sequence can be arranged in order of acquisition time or space.

[0023] In this embodiment of the disclosure, a mining intrinsically safe dust removal camera deployed on large equipment (such as a tunneling machine, anchor transport machine, etc.) at the tunneling face can be used to take surround shots of the environment around the equipment, thereby obtaining an original image sequence covering the entire equipment.

[0024] It should be noted that arranging the images according to the acquisition time or spatial order can directly correspond to the physical positional relationship of the images around the device, providing a logical basis for subsequently establishing the stitching direction and reference image.

[0025] Step 102: Denoise each image in the original image sequence to obtain a denoised image sequence.

[0026] In this embodiment of the disclosure, noise reduction processing is performed on each image in the original image sequence to eliminate the interference of the complex underground environment on image quality, providing clean input data for subsequent accurate stitching.

[0027] Specifically, for high-intensity noise (such as Gaussian noise, salt-and-pepper noise) introduced into the original image sequence due to factors such as uneven downhole lighting, dust scattering, and equipment vibration, a pre-trained deep learning denoising model (such as a model based on convolutional neural networks or Transformer architecture) can be used to process each image independently. By learning the noise distribution characteristics of a large number of real-world downhole scenes, the deep learning denoising model can effectively remove noise while preserving the edge texture and detail information of the image to the maximum extent, avoiding the loss of feature points due to excessive smoothing. After denoising, the image sequence exhibits a significantly improved signal-to-noise ratio, and the detectability and matching accuracy of feature points are greatly enhanced, thus laying a reliable data foundation for the subsequent steps of accurately calculating the homography matrix between adjacent images and achieving progressive stitching.

[0028] Step 103: Select one image from the denoised image sequence as the reference image.

[0029] In this embodiment of the disclosure, a reference image is selected from the denoised image sequence as a unified reference system to provide the spatial coordinate origin for the progressive alignment of all subsequent images.

[0030] As an example of a possible implementation, the image with the best overall quality can be selected as the reference image based on the denoised image sequence by calculating quality evaluation indicators such as sharpness, contrast, or information entropy of each image; or, considering the spatial symmetry of panoramic images, an image in the middle of the sequence can be directly selected as the reference.

[0031] In one embodiment, the aforementioned reference image is fixed as the origin of the target coordinate system for all transformations in the subsequent stitching process; that is, its viewpoint and position remain unchanged, while all other images are aligned to this reference image through progressively accumulating affine transformations. Selecting a suitable reference image can effectively reduce accumulated errors, control the overall offset range of the panoramic image, and provide a clear starting point for distinguishing between the two stitching paths, "towards the index increasing direction" and "towards the index decreasing direction," in subsequent steps, thereby ensuring the orderliness and accuracy of the entire progressive stitching process.

[0032] Step 104: Starting from the reference image, the images are stitched together sequentially in the direction of increasing the sequence index of the denoised image sequence to obtain the first stitched image.

[0033] In this embodiment of the disclosure, a reference image can be used as a spatial reference system. Following the increasing order of the image sequence index, each image to the right of the reference image (that is, in the direction of increasing sequence index) can be gradually aligned and stitched onto a unified stitching result through iterative cumulative transformation.

[0034] Specifically, the reference image is used as the initial image for the current stitching result, and a unit cumulative transformation matrix is ​​initialized. Starting from the image immediately adjacent to the reference image, each image is processed sequentially to the right: For the current image to be stitched and its left-side neighboring image, the homography matrix between them is calculated, which accurately describes the viewpoint change relationship between adjacent images; using this homography matrix and the current cumulative transformation matrix, the cumulative transformation matrix that directly transforms the image to be stitched to the coordinate system of the current stitching result is derived; after the image to be stitched undergoes an affine transformation using this cumulative transformation matrix, it is image fused with the current stitching result, thereby seamlessly embedding the new image into the stitching result; at the same time, the cumulative transformation matrix is ​​updated so that it can directly apply to the next subsequent image.

[0035] It should be noted that, through the progressive strategy of calculating the homography matrix by “image-image” and cumulative transformation by “image-stitching sequence”, all images with indices greater than the baseline are gradually and accurately stitched onto the same image, ultimately forming the first stitched image. This effectively avoids the problems of excessive distortion and feature point distortion caused by direct matching of distant images, and ensures the geometric consistency of the stitching results.

[0036] In some embodiments of this disclosure, step 104 may specifically include the following steps: Step a1: Initialize the current stitching result w as the reference image and initialize the cumulative transformation matrix H as the identity matrix.

[0037] In one embodiment, the current stitching result w is set as the reference image, that is, the starting point of the stitching is the reference image itself, and its coordinate system will be used as the target reference system for all subsequent image transformations.

[0038] In addition, a 3×3 identity matrix can be initialized as the cumulative transformation matrix H. The purpose of H is to record the cumulative perspective transformation relationship from the current image to be processed to the reference image w. Initializing it as an identity matrix indicates that the transformation relationship between the reference image and itself is an identity transformation, providing a mathematical basis for subsequent iterative updates.

[0039] For each image whose index is greater than the base image index, perform the following steps in sequence: Step a2, calculate the current image x j Towards the next image x j+1 The homography matrix M.

[0040] It is understandable that the homography matrix M represents the current image x. j With the next image x j+1 The geometric transformation relationship between them. M satisfies .

[0041] As an example, feature points can be extracted from the overlapping region of two images, feature matching can be performed, and mismatched points can be eliminated using a random sampling consensus algorithm. Finally, the accurate homography matrix can be calculated using the Direct Linear Transform (DLT) method. Since the viewpoint changes between adjacent images are small, the calculated homography matrix has high accuracy, providing a reliable foundation for subsequent cumulative transformations.

[0042] Step a3: Based on the homography matrix M and the current cumulative transformation matrix H, obtain the result from image x j+1 The transformation matrix to the current splicing result w .

[0043] In some embodiments of this disclosure, the homography matrix precisely describes the viewpoint change between the current image and the next image, while the inverse of the current cumulative transformation matrix reflects the coordinate correspondence between the current image and the stitched result. By organically combining these two transformation relationships, the overall transformation parameters required to directly transform from the next image to the current stitched result can be obtained.

[0044] In some embodiments of this disclosure, step a3 may specifically include: use Image x j+1 Transform to image x j From the perspective of [the first transformation], the first transformation result is obtained; Using H 1 Image x j By changing the perspective to the current stitching result w, the second transformation result is obtained; By combining the results of the first and second transformations, we obtain So that the image x j+1 Change to the perspective of the current stitched result w.

[0045] In this embodiment of the disclosure, the inverse matrix of the homography matrix between adjacent images obtained in step a2 is used to map the next image x. j+1 Transform to the current image x j From the perspective of, and then using the inverse matrix H of the current cumulative transformation matrix. 1 , set the current image x j The perspective is then further transformed to the viewpoint of the stitched result. By merging these two levels of transformation, a composite transformation relationship is obtained that directly maps the next image to the current stitched result.

[0046] Step a4, use the transformation matrix to transform the image x j+1 Perform an affine transformation to obtain the transformed image. Then, concatenate the transformed image with the current concatenation result w to obtain the updated w.

[0047] In one embodiment, in image stitching technology, affine transformation is a key mathematical means to achieve geometric alignment of images. Through a linear transformation matrix and a two-dimensional translation vector, it works together on the coordinates of each pixel in the image, thereby achieving rotation, scaling, translation and a small number of shearing operations on the image while preserving the image's straightness (i.e., straight lines remain straight lines after transformation) and parallelism (i.e., parallel lines remain parallel after transformation).

[0048] In the progressive stitching process disclosed herein, step a4 uses the composite transformation matrix derived in step a3 to perform an affine transformation on the image to be stitched. Essentially, this involves gradually adjusting the overall perspective of the image to align with the coordinate system of the current stitching result w. Since the perspective changes between adjacent images at the tunneling face are relatively small, the affine transformation is sufficient to describe these changes with high precision and is computationally efficient. It effectively avoids distortion caused by overly complex perspective transformations, ensuring that the new image can be smoothly and accurately embedded into the stitched panoramic image.

[0049] Step a5, update the cumulative transformation matrix .

[0050] In one embodiment, the composite transformation relationship calculated in the current iteration is redefined as the cumulative transformation benchmark for the next iteration, thereby achieving continuous and progressive transformation of subsequent images. This update strategy ensures that each iteration advances one step based on the previous transformation, making the entire stitching process exhibit a cumulative progression characteristic.

[0051] Step a6: Assign j the value j+1, then return to calculate the current image x. j With the next image x j+1 The steps of the homography matrix M between the images are repeated until all images with indices greater than the reference image index are processed, resulting in the first stitched image.

[0052] In one embodiment, by incrementing index j, the current processing focus shifts from the current image pair to the next set of adjacent images, and the process returns to step a3 to recalculate the new homography matrix. This process is repeated until all images with indices greater than the reference image index have been processed. The ingenuity of this iterative mechanism lies in the fact that each loop processes only one pair of adjacent images, but through the continuous updating of the cumulative transformation matrix, all right-hand images are eventually and systematically unified into the coordinate system of the reference image, thus obtaining a complete first stitched image. This progressive processing method effectively avoids the error accumulation and distortion problems caused by direct matching of distant images.

[0053] Step 105: Starting from the reference image, the images are stitched together sequentially in the direction of decreasing sequence index of the denoised image sequence to obtain the second stitched image.

[0054] In this embodiment, using a reference image as a spatial reference, each image to the left of the reference image (i.e., in the direction of decreasing sequence index) is progressively aligned and stitched onto a unified stitching result, following the decreasing order of the image sequence index. Similar to right-side stitching, a progressive strategy of calculating the homography matrix from "image-image" and accumulating transformation from "image-stitching sequence" is also adopted. However, since the left-side image will have negative coordinate regions after transformation, an additional translation correction operation needs to be introduced in each iteration to adjust all pixel coordinates to a non-negative range, ensuring that the image can be correctly stored and displayed.

[0055] By processing each image with an index smaller than the reference image in turn, all left-side images are transformed into the coordinate system of the reference image in an orderly manner, ultimately forming a second stitched image corresponding to the first stitched image, laying a solid foundation for subsequent fusion to generate a complete panoramic image.

[0056] In some embodiments of this disclosure, step 105 may specifically include the following steps: Step b1: Initialize the current stitching result w as the reference image and initialize the cumulative transformation matrix H as the identity matrix.

[0057] In one embodiment, the current stitching result w is set as the reference image, that is, the starting point of the stitching is the reference image itself, and its coordinate system will be used as the target reference system for all subsequent image transformations.

[0058] In addition, a 3×3 identity matrix can be initialized as the cumulative transformation matrix H. The purpose of H is to record the cumulative perspective transformation relationship from the current image to be processed to the reference image w. Initializing it as an identity matrix indicates that the transformation relationship between the reference image and itself is an identity transformation, providing a mathematical basis for subsequent iterative updates.

[0059] For each image whose index is less than the base image index, perform the following steps in sequence: Step b2, calculate the current image x j Towards the next image x j-1 The homography matrix M.

[0060] In the embodiments of this disclosure, the homography matrix M can be calculated using the homography matrix M calculation method proposed in any embodiment of this disclosure, and will not be elaborated here.

[0061] Step b3: Based on the homography matrix M and the current cumulative transformation matrix H, obtain the result from image x. j-1 The transformation matrix to the current splicing result w .

[0062] In some embodiments of this disclosure, the homography matrix precisely describes the viewpoint change between the current image and the next image, while the inverse of the current cumulative transformation matrix reflects the coordinate correspondence between the current image and the stitched result. By organically combining these two transformation relationships, the overall transformation parameters required to directly transform from the next image to the current stitched result can be obtained.

[0063] Step b4, using the transformation matrix For image x j-1 Perform an affine transformation to obtain the transformed image. .

[0064] In the embodiments of this disclosure, the specific implementation of affine transformation can be achieved by adopting the affine transformation method proposed in any embodiment of this disclosure, and will not be elaborated here.

[0065] Step b5, for image x j-1 Perform translation correction to obtain the translated image, so that the image x j-1 All pixel coordinates are non-negative.

[0066] In this embodiment of the disclosure, since some pixels of the left image will be mapped to the left side of the reference image coordinate system after the affine transformation, that is, there will be areas with negative horizontal coordinates. However, computer image storage requires that all pixel coordinates be non-negative, so translation correction must be performed before each stitching.

[0067] The image after translation correction retains the correct geometric alignment with the current stitching result and meets the basic requirements for image data storage, thus laying the coordinate foundation for subsequent seamless fusion.

[0068] In some embodiments of this disclosure, step b5 may specifically include the following steps: Calculate the transformed image The coordinates of the four corner points; Take the minimum x-coordinate of the four corner points and the minimum x-coordinate of 0 as x. min Take the minimum value of the y-coordinate and the minimum value of 0 as y. min ; Construct the translation matrix T:

[0069] Applying the translation matrix T to the transformed image yields the translation-corrected image. .

[0070] In this embodiment of the disclosure, the coordinates of the four corner points of the transformed image are calculated, the minimum negative values ​​of the horizontal and vertical coordinates are determined, and a translation matrix is ​​constructed based on these two minimum values ​​to translate the entire image to the right and down, so that all pixel coordinates become non-negative values.

[0071] Step b6: Combine the translated and corrected image with the current stitching result w to obtain the updated w.

[0072] Step b7, update the cumulative transformation matrix .

[0073] In one embodiment, the composite transformation relationship calculated in the current iteration is redefined as the cumulative transformation benchmark for the next iteration, thereby achieving continuous and progressive transformation of subsequent images. This update strategy ensures that each iteration advances one step based on the previous transformation, making the entire stitching process exhibit a cumulative progression characteristic.

[0074] Step b8: Assign j the value j-1, and return to calculate the current image x. j With the next image x j-1 The process of processing the homography matrix M between images continues until all images with indices smaller than the reference image index are processed, resulting in the second stitched image.

[0075] In this embodiment, by decrementing index j, the current processing focus is moved from the stitched image pair to the next set of adjacent images, and the process returns to step b1 to recalculate the new homography matrix. This iterative mechanism corresponds symmetrically to the incremental loop in the rightward stitching. Each loop processes only one pair of adjacent images, but through the continuous updating of the cumulative transformation matrix, all left-side images are successively and orderly transformed into the coordinate system of the reference image. When all images with indices smaller than the reference image index have been processed, the complete second stitched image is obtained. This progressive iterative control strategy ensures the continuity and stability of the left-side image stitching process, while avoiding the distortion and error accumulation problems caused by direct matching of distant images.

[0076] Step 106: Merge the first stitched image and the second stitched image to generate a panoramic image.

[0077] In some embodiments of this disclosure, step 106 may specifically include: stitching the first stitched image and the second stitched image together to generate a panoramic image.

[0078] In this embodiment, since both the first and second stitched images use the reference image as the origin of their coordinate systems, they are mathematically already in the same spatial reference system and have an overlapping area centered on the reference image. Therefore, the core task of this step is not to re-align the geometrically, but to perform pixel-level fusion processing on the overlapping area of ​​the two images to eliminate possible brightness differences, stitching seams, or slight misalignments, achieving a smooth transition.

[0079] In this embodiment of the disclosure, two images are directly stitched together. That is, based on the aligned coordinate relationship, the second stitched image is placed to the left of the first stitched image, and the two are naturally superimposed in the overlapping area, ultimately generating a seamless panoramic image covering the entire equipment, providing complete and clear visual data support for downhole visual monitoring and remote control.

[0080] This invention provides a progressive panoramic image stitching method for large equipment in tunneling faces. It acquires a sequence of original images arranged chronologically or spatially, and performs noise reduction on each image, effectively eliminating noise interference from the high dust and low illumination environment underground, providing a high-quality image data foundation for subsequent stitching. By selecting one image from the denoised image sequence as a reference image, a unified stitching reference system is established. Based on this, stitching is performed sequentially in both the increasing and decreasing index directions, starting from the reference image. This avoids excessive distortion and feature point distortion caused by direct matching of distant images, achieving orderly and progressive alignment of the surrounding images of large equipment. Finally, by fusing the first and second stitched images, a complete panoramic image is generated. This enables high-precision, low-distortion panoramic stitching of large equipment surrounding images under complex working conditions in tunneling faces, providing clear and reliable image data support for underground visual tasks and remote monitoring.

[0081] Figure 2 This is a block diagram illustrating a progressive panoramic image stitching device for large equipment at a tunneling face, according to an exemplary embodiment. (Refer to...) Figure 2 The device includes an acquisition unit 201, a noise reduction unit 202, a selection unit 203, a first splicing unit 204, a second splicing unit 205, and a fusion unit 206.

[0082] Among them, the acquisition unit 201 is used to acquire the original image sequence of large equipment at the tunneling face; the original image sequence is arranged in the order of acquisition time or space. The noise reduction unit 202 is used to perform noise reduction processing on each image in the original image sequence to obtain a noise-reduced image sequence; The selection unit 203 is used to select one image from the denoised image sequence as a reference image; The first stitching unit 204 is used to stitch together the image sequentially in the direction of increasing the sequence index of the denoised image sequence, starting from the reference image, to obtain the first stitched image. The second stitching unit 205 is used to stitch together the image from the reference image in the direction of decreasing sequence index of the denoised image sequence to obtain the second stitched image. The fusion unit 206 is used to fuse the first stitched image and the second stitched image to generate a panoramic image.

[0083] In some embodiments of this disclosure, the first splicing unit 204 may specifically be used for: Initialize the current stitching result w as the reference image, and initialize the cumulative transformation matrix H as the identity matrix; For each image whose index is greater than the base image index, perform the following steps in sequence: Calculate the current image x j Towards the next image x j+1 The homography matrix M; Based on the homography matrix M and the current cumulative transformation matrix H, we obtain the result from image x. j+1 The transformation matrix to the current splicing result w ; Using the transformation matrix to transform the image x j+1 Perform an affine transformation to obtain the transformed image, then concatenate the transformed image with the current concatenation result w to obtain the updated w; Update the cumulative transformation matrix ; Assign j the value of j+1 and return to calculate the current image x. j With the next image x j+1 The steps of the homography matrix M between the images are repeated until all images with indices greater than the reference image index are processed, resulting in the first stitched image.

[0084] In some embodiments of this disclosure, the first splicing unit 204 may specifically be used for: use Image x j+1 Transform to image x j From the perspective of [the first transformation], the first transformation result is obtained; Using H 1 Image x j By changing the perspective to the current stitching result w, the second transformation result is obtained; By combining the results of the first and second transformations, we obtain So that the image x j+1 Change to the perspective of the current stitched result w.

[0085] In some embodiments of this disclosure, the second splicing unit 205 may specifically be used for: Initialize the current stitching result w as the reference image, and initialize the cumulative transformation matrix H as the identity matrix; For each image whose index is less than the base image index, perform the following steps in sequence: Calculate the current image x j Towards the next image x j-1 The homography matrix M; Based on the homography matrix M and the current cumulative transformation matrix H, we obtain the result from image x. j-1 The transformation matrix to the current splicing result w ; Using the transformation matrix For image x j-1 Perform an affine transformation to obtain the transformed image. ; For image x j-1 Perform translation correction to obtain the translated image, so that the image x j-1 All pixel coordinates are non-negative; The translated and corrected image is then stitched together with the current stitching result w to obtain the updated w; Update the cumulative transformation matrix ; Assign j the value j-1 and return to calculate the current image x. j With the next image x j-1 The process of processing the homography matrix M between images continues until all images with indices smaller than the reference image index are processed, resulting in the second stitched image.

[0086] In some embodiments of this disclosure, the second splicing unit 205 may specifically be used for: Calculate the transformed image The coordinates of the four corner points; Take the minimum x-coordinate of the four corner points and the minimum x-coordinate of 0 as x. min Take the minimum value of the y-coordinate and the minimum value of 0 as y. min ; Construct the translation matrix T:

[0087] Applying the translation matrix T to the transformed image yields the translation-corrected image. .

[0088] In some embodiments of this disclosure, the fusion unit 206 may be specifically used to: fuse the first stitched image and the second stitched image to generate a panoramic image.

[0089] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0090] This invention provides a progressive panoramic image stitching method for large equipment in tunneling faces. It acquires a sequence of original images arranged chronologically or spatially, and performs noise reduction on each image, effectively eliminating noise interference from the high dust and low illumination environment underground, providing a high-quality image data foundation for subsequent stitching. By selecting one image from the denoised image sequence as a reference image, a unified stitching reference system is established. Based on this, stitching is performed sequentially in both the increasing and decreasing index directions, starting from the reference image. This avoids excessive distortion and feature point distortion caused by direct matching of distant images, achieving orderly and progressive alignment of the surrounding images of large equipment. Finally, by fusing the first and second stitched images, a complete panoramic image is generated. This enables high-precision, low-distortion panoramic stitching of large equipment surrounding images under complex working conditions in tunneling faces, providing clear and reliable image data support for underground visual tasks and remote monitoring.

[0091] Figure 3 This is a block diagram illustrating an apparatus for a progressive panoramic image stitching method for large equipment at a tunneling face, according to an exemplary embodiment. For example, apparatus 300 may be an electronic device, such as a mobile phone, computer, digital broadcasting terminal, messaging device, tablet device, personal digital assistant, etc.

[0092] Reference Figure 3 The device 300 may include one or more of the following components: processing component 302, memory 304, power component 306, multimedia component 308, audio component 310, input / output (I / O) interface 312, sensor component 314, and communication component 316.

[0093] Processing component 302 typically controls the overall operation of device 300, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 302 may include one or more processors 320 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 302 may include one or more modules to facilitate interaction between processing component 302 and other components. For example, processing component 302 may include a multimedia module to facilitate interaction between multimedia component 308 and processing component 302.

[0094] Memory 304 is configured to store various types of data to support the operation of device 300. Examples of such data include instructions for any application or method operating on device 300, contact data, phonebook data, messages, pictures, videos, etc. Memory 304 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0095] The power supply component 306 provides power to the various components of the device 300. The power supply component 306 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 300.

[0096] Multimedia component 308 includes a screen that provides an output interface between the device 300 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 308 includes a front-facing camera and / or a rear-facing camera. When the device 300 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0097] Audio component 310 is configured to output and / or input audio signals. For example, audio component 310 includes a microphone (MIC) configured to receive external audio signals when device 300 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 304 or transmitted via communication component 316. In some embodiments, audio component 310 also includes a speaker for outputting audio signals.

[0098] I / O interface 312 provides an interface between processing component 302 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, start buttons, and lock buttons.

[0099] Sensor assembly 314 includes one or more sensors for providing status assessments of various aspects of device 300. For example, sensor assembly 314 may detect the on / off state of device 300, the relative positioning of components such as the display and keypad of device 300, changes in the position of device 300 or a component of device 300, the presence or absence of user contact with device 300, the orientation or acceleration / deceleration of device 300, and temperature changes of device 300. Sensor assembly 314 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 314 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 314 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0100] Communication component 316 is configured to facilitate wired or wireless communication between device 300 and other devices. Device 300 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 316 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 316 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0101] In an exemplary embodiment, the apparatus 300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0102] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 304 including instructions, which can be executed by a processor 320 of the device 300 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0103] In an exemplary embodiment, a computer program product is also provided, including a computer program that implements the above-described method when executed by the processor 320 of the device 300.

[0104] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.

[0105] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A progressive panoramic image stitching method for large equipment in a tunneling face, characterized in that, include: Acquire raw image sequences of large equipment at the tunneling face; the raw image sequences are arranged in chronological or spatial order of acquisition. Each image in the original image sequence is subjected to noise reduction processing to obtain a noise-reduced image sequence; One image is selected from the denoised image sequence as the reference image; Starting from the reference image, the images are stitched together sequentially in the direction of increasing the sequence index of the denoised image sequence to obtain the first stitched image; Starting from the reference image, the images are stitched together sequentially in the direction of decreasing sequence index of the denoised image sequence to obtain the second stitched image. The first stitched image and the second stitched image are merged to generate a panoramic image.

2. The progressive panoramic image stitching method for large equipment in a tunneling face according to claim 1, characterized in that, The first stitched image is obtained by sequentially stitching the images together in the direction of increasing the sequence index of the denoised image sequence, starting from the reference image, including: Initialize the current stitching result w as the reference image, and initialize the cumulative transformation matrix H as the identity matrix; For each image whose index is greater than the base image index, perform the following steps in sequence: Calculate the current image x j Towards the next image x j+1 The homography matrix M; Based on the homography matrix M and the current cumulative transformation matrix H, the image x is obtained. j+1 The transformation matrix to the current splicing result w ; Using the transformation matrix to transform image x j+1 Perform an affine transformation to obtain the transformed image, and then concatenate the transformed image with the current stitching result w to obtain the updated w; Update the cumulative transformation matrix ; Assign j the value j+1 and return to perform the calculation of the current image x. j With the next image x j+1 The steps of homography matrix M between the images are repeated until all images with indices greater than the reference image index are processed to obtain the first stitched image.

3. The progressive panoramic image stitching method for large equipment in a tunneling face according to claim 2, characterized in that, The process involves obtaining the image x from the homography matrix M and the current cumulative transformation matrix H. j+1 The transformation matrix to the current splicing result w ,include: use Image x j+1 Transform to image x j From the perspective of [the first transformation], the first transformation result is obtained; Using H 1 Image x j By changing the perspective to the current stitching result w, the second transformation result is obtained; By combining the results of the first and second transformations, we obtain So that the image x j+1 Change to the perspective of the current stitched result w.

4. The progressive panoramic image stitching method for large equipment in a tunneling face according to claim 1, characterized in that, The step of stitching together the second stitched image by starting with the reference image and proceeding sequentially in the direction of decreasing sequence index of the denoised image sequence includes: Initialize the current stitching result w as the reference image, and initialize the cumulative transformation matrix H as the identity matrix; For each image whose index is less than the base image index, perform the following steps in sequence: Calculate the current image x j Towards the next image x j-1 The homography matrix M; Based on the homography matrix M and the current cumulative transformation matrix H, the image x is obtained. j-1 The transformation matrix to the current splicing result w ; Using the transformation matrix For image x j-1 Perform an affine transformation to obtain the transformed image. ; For image x j-1 Perform translation correction to obtain the translated image, so that the image x j-1 All pixel coordinates are non-negative; The translated and corrected image is then stitched together with the current stitching result w to obtain the updated w; Update the cumulative transformation matrix ; Assign j the value j-1 and return to perform the calculation of the current image x. j With the next image x j-1 The steps of homography matrix M between the images are repeated until all images with indices smaller than the reference image index are processed, resulting in the second stitched image.

5. The progressive panoramic image stitching method for large equipment in a tunneling face according to claim 4, characterized in that, The image x j-1 After performing translation correction, the image after translation correction is obtained, including: Calculate the transformed image The coordinates of the four corner points; Take the minimum x-coordinate of the four corner points and the minimum x-coordinate of 0 as x. min Take the minimum value of the y-coordinate and the minimum value of 0 as y. min ; Construct the translation matrix T: Applying the translation matrix T to the transformed image yields the translation-corrected image. .

6. The progressive panoramic image stitching method for large equipment in a tunneling face according to claim 1, characterized in that, The process of fusing the first stitched image and the second stitched image to generate a panoramic image includes: The first stitched image and the second stitched image are stitched together to generate the panoramic image.

7. A progressive panoramic image stitching device for large equipment in a tunneling face, characterized in that, include: An acquisition unit is used to acquire the original image sequence of large equipment at the tunneling face; the original image sequence is arranged in order of acquisition time or space. A noise reduction unit is used to perform noise reduction processing on each image in the original image sequence to obtain a noise-reduced image sequence; The selection unit is used to select one image from the denoised image sequence as a reference image; The first stitching unit is used to stitch together the image sequentially in the direction of increasing the sequence index of the denoised image sequence, starting from the reference image, to obtain the first stitched image. The second stitching unit is used to stitch together the reference image in the direction of decreasing sequence index of the denoised image sequence to obtain the second stitched image. The fusion unit is used to fuse the first stitched image and the second stitched image to generate a panoramic image.

8. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the method as described in any one of claims 1 to 6.

Images