Annular lens tube network video data processing method, device and equipment and storage medium

Abstract

The application discloses a kind of ring belt lens pipe network video data processing method, device, equipment and storage medium, it is related to image processing technical field, comprising: the ring belt lens pipe network video data of acquisition is collected, and coordinate transformation is carried out to ring belt lens pipe network video data, and the ring belt video data after coordinate transformation is obtained;Image linearization processing is carried out to the ring belt video data after coordinate transformation, and initial rectangular image is obtained;Linear interpolation processing is carried out to initial rectangular image, and target rectangular image is obtained;Feature point detection and extraction are carried out to target rectangular image, and initial feature point pair is obtained;Random sample consistency algorithm is used to filter initial feature point pair, and target feature point pair is obtained;Mean coordinate fusion processing is carried out to target feature point pair, and fusion mosaic panorama is obtained.Through the above scheme, the fusion mosaic panorama of high quality is obtained, more close to actual pipe network inner wall imaging feature, and the correction distortion precision is greatly improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to methods, apparatus, equipment and storage media for processing video data in a ring-shaped lens network. Background Technology

[0002] In existing research, "axial imaging" and "radial imaging" are the two main methods for imaging inside pipes. "Axial imaging" involves placing the imaging device along the pipe axis with its line of sight for single-viewpoint imaging, such as pipe closed-circuit television camera imaging, fisheye lens camera imaging, single-reflection pipe camera imaging, catadioptric panoramic ring lens camera imaging, and omnidirectional vision sensor imaging. Due to its fast speed and better panoramic unfolding effect of the ring image compared to "radial imaging," "axial imaging" has developed rapidly. However, under "axial imaging," the mapping relationship of the inner surface of the pipe on the resulting image is always a regular and symmetrical ring image. Moreover, with the increase of axial depth, the image distortion and resolution distribution are different. The unfolding effect of the ring image inside the pipe depends on the regularity of the pipe, the degree of coincidence between the line of sight and the pipe axis, and the distortion correction algorithm during the unfolding process. Any sudden change in pipe morphology and line of sight shift can cause a sharp decrease in the image unfolding effect. In particular, the radial barrel distortion caused by fisheye lenses is often difficult to completely correct. Since panoramic pipeline images are an indirect reflection of the pipe's inner wall structure—the image formed by light reflecting off the pipe's inner wall onto the camera probe's conical mirror—the inner wall structure determines the composition of the video image's color depth. The varying reflectivity of areas such as cracks, fracture surfaces, holes, and breakage zones within the pipeline wall results in either bright or dark colors in the video image, posing a significant challenge to image feature matching and mosaicking methods.

[0003] Existing panoramic image mosaicking methods are not ideal and cannot meet the high-precision identification requirements of existing pipeline network defect video detection. Considering the complex field conditions requiring accuracy, efficiency, and computational power for defect identification in pipeline network video detection, existing methods suffer from the following main problems: Traditional internal wall scanning strip stacking mosaicking methods rely on a limited amount of parameter data. Furthermore, in actual field surveys, it is difficult to ensure that the internal camera probe moves at a very slow and uniform speed, resulting in recorded video images that fluctuate in speed and are accompanied by irregular rotation and back-and-forth jitter. This leads to severe distortion of the final internal wall image, with periods of expansion and contraction. When converting the video images obtained by the ring-shaped camera probe into a usable panoramic internal wall image, the imaging characteristics of the pipeline internal wall structure are circular shapes from near to far. During the acquisition equipment's movement, the pipe wall structure is imaged continuously from near to far, resulting in circular distortion and overlapping parts in the video image. Traditional pinhole imaging model correction and fusion mosaicking methods struggle to correct and fuse these circular distortions. Summary of the Invention

[0004] The main objective of this application is to provide a method, apparatus, device, and storage medium for processing video data in a ring-shaped lens network, aiming to solve the technical problems of image distortion and inability to fuse images when analyzing and processing panoramic video images within a pipe.

[0005] To achieve the above objectives, this application proposes a method for processing video data in a ring-shaped lens network, the method comprising: Acquire the collected ring-shaped lens network video data, and perform coordinate transformation on the ring-shaped lens network video data to obtain the coordinate-transformed ring-shaped video data; The transformed ring video data is linearized to obtain an initial rectangular image. The initial rectangular image is subjected to linear interpolation to obtain the target rectangular image; Feature point detection and extraction are performed on the target rectangular image to obtain initial feature point pairs; The initial feature point pairs are filtered using a random sampling consensus algorithm to obtain target feature point pairs; The target feature point pairs are subjected to mean coordinate fusion processing to obtain a fused mosaic panoramic image.

[0006] In one embodiment, the step of acquiring the collected ring-shaped lens network video data and performing coordinate transformation on the ring-shaped lens network video data to obtain coordinate-transformed ring-shaped video data includes: Acquire the collected video data from the ring-shaped lens network; A rectangular coordinate system is established based on the video data of the ring-shaped lens network. Obtain the coordinates of the center point in the video data of the ring-shaped lens network in the Cartesian coordinate system; Calculate the radial length and radial angle between any point in the ring-shaped lens network video data and the center point; The coordinates of the annular region of any point in the annular lens network video data are obtained based on the radial length and the radial angle. Based on the correspondence between the starting point of the annular region and the corresponding point in the rectangular region, the coordinates of the annular region are transformed to obtain the annular video data after coordinate transformation.

[0007] In one embodiment, the step of performing image linearization processing on the coordinate-transformed ring-shaped video data to obtain an initial rectangular image includes: Tangential direction image linearization processing is performed on the coordinate-transformed ring video data to map the wedge-shaped part in the coordinate-transformed ring video data to a rectangular array, thereby obtaining a rectangular image after tangential direction linearization processing; The radial direction image linearization processing is performed on the coordinate-transformed ring video data to obtain multiple outer circular points of the coordinate-transformed ring video data; The center coordinates of the ring video data after coordinate transformation are obtained based on the multiple outer circular points, and the outer circle radius is calculated using the center coordinates and the multiple outer circular points. Determine the inner circle point on the inner circumference of the ring video data after coordinate transformation; The radius of the inner circle is determined based on the coordinates of the inner circle point and the center of the circle. The width and height of the image in pixels are calculated based on the inner circle radius and the outer circle radius, and a rectangular image after radial linearization is obtained based on the width and the height. The initial rectangular image is obtained by linearizing the rectangular image in the tangent direction and the rectangular image in the radial direction.

[0008] In one embodiment, the step of performing linear interpolation on the initial rectangular image to obtain the target rectangular image includes: The initial rectangular image is back-mapping processed to map the initial rectangular image to a preset position on the coordinate-transformed ring video data; Obtain the vertices surrounding the preset position; The vertices are interpolated horizontally to obtain the first interpolation point and the second interpolation point; Vertical interpolation is performed based on the first interpolation point and the second interpolation point to obtain the interpolation point value; The target rectangular image is obtained based on the interpolation point values.

[0009] In one embodiment, the step of detecting and extracting feature points from the target rectangular image to obtain initial feature point pairs includes: The blur level of the target rectangular image is adjusted to construct the scale space of the image; Local extreme points are identified in the scale space, and these local extreme points are used as candidate key points. Calculate the position and scale of the candidate key points, and remove key points whose contrast and edge response do not meet the preset conditions according to the position and scale to obtain the target key points; Feature point matching is performed on the target key points to obtain initial feature point pairs.

[0010] In one embodiment, the step of using a random sampling consensus algorithm to filter the initial feature point pairs to obtain target feature point pairs includes: The random sampling consensus algorithm is used to randomly sample multiple data points from the initial feature point pair, and the multiple data points are not collinear. Calculate the transformation matrix parameters for multiple sample data; The transformation matrix is ​​obtained based on the transformation matrix parameters; Calculate the projection error between each feature point pair in the initial feature point pair and the transformation matrix; When the projection error is less than a preset projection error threshold, the feature point pair is retained; When the projection error is greater than or equal to the projection error threshold, the feature point pair is removed from the initial feature point pair to obtain the target feature point pair.

[0011] In one embodiment, the step of performing mean coordinate fusion processing on the target feature point pairs to obtain a fused mosaic panoramic image includes: The target image is obtained based on the target feature point pairs; The target image is subjected to boundary recognition using an edge detection algorithm to obtain the foreground image boundary and the background image boundary. Based on the foreground image boundary and the background image boundary, the target image is segmented into multiple polygonal regions; The difference between multiple polygonal regions based on their internal pixels is calculated using the mean coordinate method. The difference is added to the original pixel value of each pixel in the foreground image to obtain a fused mosaic panoramic image.

[0012] Furthermore, to achieve the above objectives, this application also proposes a ring-shaped lens network video data processing device, which includes: The acquisition module is used to acquire the collected ring-shaped lens network video data and perform coordinate transformation on the ring-shaped lens network video data to obtain the coordinate-transformed ring-shaped video data. The linearization processing module is used to perform image linearization processing on the coordinate-transformed ring video data to obtain an initial rectangular image. An interpolation processing module is used to perform linear interpolation processing on the initial rectangular image to obtain a target rectangular image; The feature point detection and extraction module is used to detect and extract feature points in the target rectangular image to obtain initial feature point pairs; The filtering module is used to filter the initial feature point pairs using a random sampling consensus algorithm to obtain target feature point pairs; The fusion module is used to perform mean coordinate fusion processing on the target feature point pairs to obtain a fused mosaic panoramic image.

[0013] In addition, to achieve the above objectives, this application also proposes a ring-loop lens network video data processing device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the ring-loop lens network video data processing method described above.

[0014] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the above-described ring-band lens network video data processing method.

[0015] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the ring-shaped lens network video data processing method described above.

[0016] One or more technical solutions proposed in this application have at least the following technical effects: 1) The solution proposed in this application can effectively process the video data of the ring-shaped lens network, solving the technical problems of image distortion and inability to fuse. By preprocessing and correcting the distortion of the panoramic ring image, the restoration capability of the panoramic ring image is enhanced, and the distortion of the panoramic ring image is reduced, thus consolidating the theoretical foundation for distortion correction of the video data of the ring-shaped lens network.

[0017] 2) The quality of the fused image is far superior to that of the traditional line drawing and stacking method. The resolution and clarity of the fused image are improved by an order of magnitude. The image stitching and fusion method is simple and easy to operate, enabling intelligent and automated processing of panoramic images for pipeline video inspection. It meets the development needs of fixed-focus panoramic ring lens distortion correction in terms of simplicity, convenience, and low cost.

[0018] 3) By establishing a Cartesian coordinate system and obtaining the center point coordinates, the video data acquired by the ring-shaped lens can be precisely geometrically calibrated. Calculating the radial length and radial angle between any point and the center point yields the ring-shaped region coordinates, effectively eliminating perspective distortion or geometric warping caused by the lens's special properties, resulting in more realistic and accurate video data. Converting the data to ring-shaped region coordinates based on the radial length and radial angle preserves and accurately represents the lens's unique perspective information, better adapting to subsequent data processing and analysis needs. After converting the ring-shaped region coordinates to rectangular region coordinates using coordinate transformation methods, subsequent processing can utilize a standard rectangular coordinate system, improving processing efficiency and reducing the complexity of data preprocessing.

[0019] 4) By linearizing the tangential and radial directions, the wedge-shaped and radially curved portions of the annular lens can be effectively mapped into a standard rectangular array. This reduces geometric distortion, ensuring accurate preservation of image precision and detail, and avoiding image distortion common in traditional processing methods. Mapping the coordinate-transformed annular video data into a rectangular image makes image processing more standardized and uniform. By obtaining precise inner and outer radii, as well as their width and height, the image size and proportions can be guaranteed to meet actual requirements, facilitating subsequent image processing and algorithm applications. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating an embodiment of the video data processing method for a ring-shaped lens network in this application. Figure 2 This is a schematic diagram of the imaging principle of an annular lens lens provided in an embodiment of the annular lens network video data processing method of this application; Figure 3 This is a schematic diagram of a cylindrical and planar projection model provided in an embodiment of the video data processing method for a ring-shaped lens network of this application; Figure 4 This is a schematic diagram illustrating the establishment of a rectangular coordinate system and the transformation of the annular data image coordinates, provided in an embodiment of the annular lens network video data processing method of this application. Figure 5 A schematic diagram of the transformed image provided in an embodiment of the video data processing method for the ring-shaped lens network of this application; Figure 6 This is a schematic diagram of a backward mapping algorithm from an initial rectangular image to the original image provided in an embodiment of the video data processing method for a ring-shaped lens network of this application; Figure 7 This is a schematic diagram illustrating horizontal and vertical interpolation of vertices in an embodiment of the video data processing method for a ring-shaped lens network of this application. Figure 8 This is a schematic diagram illustrating the definition of the mean coordinate angle in an embodiment of the video data processing method for a ring-shaped lens network according to this application. Figure 9 This is a schematic diagram of image segmentation provided in an embodiment of the video data processing method for a ring-shaped lens network according to this application; Figure 10 This is a flowchart illustrating Embodiment 2 of the video data processing method for the ring-shaped lens network of this application. Figure 11 This is a schematic diagram of the annular image linearization processing provided in an embodiment of the annular lens network video data processing method of this application; Figure 12 This is a schematic diagram of annular image mapping onto a rectangular array provided in an embodiment of the video data processing method for annular lens network of this application; Figure 13 This is a simplified flowchart illustrating a method for processing video data in a ring-shaped lens network, as provided in one embodiment of the method for processing video data in a ring-shaped lens network according to this application.

[0023] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0024] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0025] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0026] Current technologies for analyzing and processing panoramic video images inside pipes primarily rely on methods for processing panoramic data. Traditional methods for mosaicking and stacking internal wall scanning strips are limited by the amount of parameter data available. Furthermore, during actual field surveys, it's difficult to ensure the camera probe inside the pipe moves at a very slow, uniform speed. This results in recorded video images that fluctuate in speed and are accompanied by irregular rotation and jitter. Consequently, the final internal wall image is severely distorted, appearing stretched and shrunken. Without image correction processing before fusion and stitching, the resulting panoramic mosaic image quality is poor. In the process of converting the video images obtained by the ring-shaped camera probe into a usable panoramic image of the inner wall of the pipe, the imaging characteristics of the inner wall structure are a ring shape from near to far. During the movement of the acquisition equipment, the pipe wall structure is imaged continuously from near to far. The images in the video have ring deformation and overlapping parts. Traditional pinhole imaging model correction and fusion mosaicking methods are difficult to achieve the correction and fusion mosaicking of ring distortion. Affected by lens tilt, cable entanglement or shaking of the detection platform, the ring-shaped camera lens will inevitably experience rotation and jitter. In particular, when the camera probe is stretched by the cable and jittered by human factors, the electronic compass and encoder itself cannot self-correct and find the correct position and direction, resulting in rotation and distortion of the video image data inside the pipe. Because the video images recorded by the ring-shaped camera possess absolute continuity and inherent image characteristics, existing methods fail to comprehensively incorporate these continuity and inherent characteristics to more accurately record or correct the orientation and distance information in the pipeline network video images. Current software implementation methods lack adaptive and intelligent processing capabilities, and subsequent data corrections such as distance and orientation correction heavily rely on manual processing and practical experience. This necessitates dedicated data processing for several days or even weeks to improve the quality of pipeline network inner wall images and for subsequent analysis and processing of survey data, which is time-consuming and labor-intensive. There is an urgent need for intelligent and automated correction, fusion, mosaicking, and stitching processing of pipeline network inner wall video images to improve the real-time performance and efficiency of video surveys of pipeline network inner wall health conditions.

[0027] This application provides a solution, (1) coordinate transformation of video data of the ring lens; (2) linearization of ring data; (3) interpolation of radiance value of image data; (4) feature extraction and registration; (5) fusion mosaicking: generation of fusion mosaicking panoramic image based on mean coordinate socket.

[0028] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of performing the above functions, such as a ring-band lens network video data processing device. The following description uses a ring-band lens network video data processing device as an example to illustrate this embodiment and the subsequent embodiments.

[0029] Based on this, embodiments of this application provide a method for processing video data in a ring-shaped lens network, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the video data processing method for the ring-shaped lens network of this application.

[0030] In this embodiment, the method for processing video data in the ring-shaped lens network includes steps S10 to S40: Step S10: Acquire the collected ring-shaped lens network video data and perform coordinate transformation on the ring-shaped lens network video data to obtain the coordinate-transformed ring-shaped video data.

[0031] It should be noted that fixed-focus annular lens pipe network video imaging is based on planar cylindrical perspective imaging, which uses a projection method to transform the inner wall scenery of the cylindrical region inside the pipe cavity around the lens into a two-dimensional finite plane, projecting a 360° panoramic view of the pipe's inner wall space onto a two-dimensional finite plane, and generating a panoramic annular image of the pipe's inner wall within it. This special lens video camera has an infinite depth of field. Its unique imaging performance gives the video observation system composed of annular lenses a prominent feature: no moving parts, allowing simultaneous observation of objects 360° around the lens inside the pipe. However, the annular image of the pipe network's inner wall formed by this lens is not convenient for observation and measurement, and exhibits significant geometric distortion and aberration. Therefore, it is necessary to find suitable image processing methods to better restore the original image, thus facilitating human observation without affecting measurement accuracy. Using coordinate transformation, through linearization and interpolation processing of the annular image, the original image can be restored with good results.

[0032] A ring-shaped lens is a special type of catadioptric lens that can form a planar ring-shaped image within a 360° lateral field of view around the optical axis. This image is a ring-shaped virtual image inside the lens, and subsequent lenses pull the image out to form ring-shaped video data. For example... Figure 2 As shown, Figure 2 This is a schematic diagram of the imaging principle of a ring lens. The ring lens optical system consists of a ring lens and a subsequent image-rotating lens group. The ring lens projects the scene in the surrounding 360° space onto a virtual image plane, which is the key to realizing panoramic ring imaging. The image-rotating lens can further image the virtual image onto the detector.

[0033] In pipeline network video inspection, the inner cavity of the pipe can be viewed as a cylinder rather than a sphere. Therefore, the image can be unfolded into a plane using a stretching method, representing the three-dimensional cylindrical surface in two dimensions. This phenomenon is called planar cylindrical perspective. It is the basis of fixed-focus ring-band lens imaging. In planar cylindrical projection mapping, all parallel light rays focus at a single point, while in the traditional pinhole planar perspective method, parallel lines in different directions focus at different points on a (horizontal) line. Fixed-focus ring-band lens imaging applies a novel planar cylindrical projection imaging principle, replacing the pinhole planar projection principle of previous pinhole optical imaging.

[0034] Therefore, this embodiment can perform coordinate transformation on the ring-shaped lens network video data. Specifically, it uses a ring-shaped lens planar cylindrical perspective imaging projection model to perform coordinate transformation on the ring-shaped image data, thereby obtaining the coordinate-transformed ring-shaped video data. For example... Figure 3 As shown, Figure 3 This diagram illustrates the cylindrical and planar projection models. The cylindrical field of view (region α in the diagram), encompassing 360° of the optical axis of the optical system, is projected onto a ring-shaped region on a two-dimensional plane, facilitating real-time panoramic imaging of the 360° annular space. The width of the ring-shaped region on the image plane corresponds to the object space field of view angle α. Each concentric circle within the ring-shaped region represents the projection of all points on the optical cone surface in object space that have a certain angle with the optical axis. The vanishing point is at the center of the circle. The angle range represented by β becomes the imaging blind zone, i.e., the center of the distant part of the pipe. The obtained annular image requires special processing to unfold the annular image formed by the panoramic annular lens into a rectangular image. Currently, there is no perfect method to represent a non-rectangular image as a rectangular image format without any distortion. This embodiment aims to minimize this distortion.

[0035] In one feasible implementation, step S10 may include steps A11 to A16: Step A11: Acquire the collected video data of the ring-shaped lens network; In practice, a ring-shaped lens can be used to capture video of the inside of the pipeline network, obtaining 360° panoramic video data of the pipeline's inner wall. This data typically contains detailed information about the pipeline's inner wall, but due to the special nature of the ring-shaped lens, the video data may suffer from geometric distortion and loss of quality.

[0036] Step A12: Establish a rectangular coordinate system based on the video data of the ring-shaped lens network; To process the video data from the ring-shaped lens network, a Cartesian coordinate system needs to be established first. This coordinate system will be used for subsequent calculations and coordinate transformations. For example... Figure 4 As shown, Figure 4 A schematic diagram illustrating the coordinate transformation of the annular data image to establish a rectangular coordinate system. Figure 5 This is a schematic diagram of the transformed image.

[0037] Step A13: Obtain the coordinates of the center point in the video data of the ring lens network on the rectangular coordinate system; In practice, the center point is the center point of the inner wall of the pipe, which is also the reference point for subsequent coordinate transformation. The coordinates of the center point O of the annular region are (xc, yc).

[0038] Step A14: Calculate the radial length and radial angle between any point in the ring-shaped lens network video data and the center point; In practical implementation, let P(x, y) be any point in the rectangular array, P′(x′, y′) be a point in the corresponding annular region, ρ be the distance of point P′ from the center point O(xc, yc) of the annular region, which is called the radial length, and θ be the angle between OP′ and the x-axis, which is called the radial angle.

[0039] Step A15: Obtain the coordinates of the annular region at any point in the annular lens network video data based on the radial length and the radial angle; In practical implementation, for each pixel in the video data, we can construct a ring-shaped region coordinate system using its radial length and radial angle. For example, P′(ρ, θ), then the coordinates of any point P′ in the ring-shaped lens network video data are given by Equation 1: (Equation 1) In the above formula, x′ and y′ are the coordinates of point P′ in the rectangular coordinate system, ρ is the radial length, θ is the radial angle, and (xc, yc) are the coordinates of the center point O.

[0040] Step A16: Based on the correspondence between the starting point of the annular region and the corresponding rectangular region point in the annular region coordinates, perform coordinate transformation on the annular region coordinates to obtain the annular video data after coordinate transformation.

[0041] In practical implementation, according to the principle of equal maximum scale, the observation points in the annular region that maintain the same radial length and radial angle as the center point O correspond to a certain row and a certain column in the rectangular region, respectively. That is, for every increase of Δρ and Δθ in ρ and θ, the corresponding x and y in the rectangular region increase by Δx and Δy, respectively. Therefore: (Equation 2) In the formula, Δρ, Δx, Δθ, and Δy are all in pixels. Assuming the starting point A′(ρ0, θ0) of the linearization of the circular image corresponds to point A(x0, y0) in the rectangular region, then: (Equation 3) Therefore, we have: (Equation 4) Substituting equation 4 into x′ and y′ in equation 1, we get: (Equation 5) At this point, the coordinate transformation between point P(x, y) in the rectangular region and point P′(x′, y′) in the corresponding annular region is completed. By setting their gray values ​​to be equal, g(x, y) = f(x′, y′), the coordinate transformation is complete.

[0042] By establishing a Cartesian coordinate system and obtaining the coordinates of the center point, precise geometric calibration can be performed on video data acquired by the ring-shaped lens. Calculating the radial length and radial angle between any point and the center point, and thus obtaining the coordinates of the ring area, effectively eliminates perspective distortion or geometric warping caused by the lens's special properties, making the video data more realistic and accurate. By converting the data to ring-shaped coordinates based on the radial length and radial angle, the lens's unique perspective information can be preserved and accurately represented, better adapting to subsequent data processing and analysis needs. After converting the ring-shaped coordinates to rectangular coordinates using coordinate transformation methods, subsequent processing can be performed using a standard rectangular coordinate system, improving processing efficiency and reducing the complexity of data preprocessing.

[0043] Step S20: Perform image linearization processing on the coordinate-transformed ring video data to obtain an initial rectangular image.

[0044] In a specific implementation, the ring-shaped image formed by the ring lens can be obtained from the ring-shaped video data after coordinate transformation, and the ring-shaped image formed by the ring lens can be linearized and unfolded into a rectangular image in the tangential and radial directions, respectively, i.e., the initial rectangular image.

[0045] Step S30: Perform linear interpolation on the initial rectangular image to obtain the target rectangular image.

[0046] Understandably, because a portion of the source image before linearization is magnified during the linearization process, the one-to-one mapping between pixels in the source image and the initial rectangular image cannot be guaranteed. When the image is magnified, one pixel in the source image may map to multiple pixels in the initial rectangular image. This mapping from source pixels to pixels in the initial rectangular image is achieved through perspective mapping of the initial rectangular image. To ensure that each pixel in the initial rectangular image has a given value, linear interpolation can be performed using inverse mapping. Without inverse mapping, there is no one-to-one association between source pixels and pixels in the initial rectangular image, and it cannot be guaranteed that a source pixel can be mapped to every pixel in the initial rectangular image. During the mapping process, some pixels will have no value, and this gap will degrade the initial rectangular image.

[0047] It should be noted that the target rectangular image is the image after linear interpolation.

[0048] In one feasible implementation, step S30 may include steps A21 to A25: Step A21: Perform backward mapping processing on the initial rectangular image to map the initial rectangular image to a preset position on the coordinate-transformed ring video data; It's important to note that the inverse mapping operates on one pixel in the initial rectangular image space at a time, with the transformation function determining which pixel from the source image is used to generate the initial rectangular image pixel. However, inverse mapping introduces a problem: fractional pixel addresses. This problem arises when calculating the contribution of a single source pixel to a target pixel. To handle these fractional addresses, interpolation is used to calculate the new pixel value at non-integer addresses (x, y) between pixels with known values.

[0049] like Figure 6 As shown, Figure 6 This diagram illustrates a backward mapping algorithm from an initial rectangular image to the original image (the loop video data after coordinate transformation). The left side of the diagram shows the original image, and the right side shows the initial rectangular image. For the point P being calculated in the diagram, the pixels with known gray values ​​surrounding it have a much greater impact than distant points. Assuming these points are closely packed and the gray values ​​do not change significantly with position, simple linear interpolation can be used.

[0050] When a point P in the initial rectangular image is back-mapped to a preset position (the corresponding point of point P), a fractional address (x, y) is generated. The value of pixel P at this non-integer address (x, y) can be obtained from the values ​​of four points A, B, C, and D according to their distance from point P.

[0051] Step A22: Obtain the vertices surrounding the preset position; In practice, since the four vertices of the preset position surround the preset position, the grayscale values ​​and coordinate positions of vertices A, B, C, and D can be obtained.

[0052] Step A23: Perform horizontal interpolation on the vertices to obtain the first interpolation point and the second interpolation point; In practice, a two-dimensional interpolation is achieved by three linear interpolations. First, the grayscale values ​​of A and B are interpolated, and then C and D are interpolated. Using the two values ​​obtained in this way as endpoints, linear interpolation is performed in the vertical direction to obtain the grayscale value of the final point.

[0053] During linearization, a portion of the original image is magnified. To prevent some output pixels from being missed due to a lack of input pixels being mapped to their vicinity, thus degrading the output image, we employ a backward mapping algorithm. This involves mapping each output pixel back to the input image one by one to determine its grayscale level. However, backward mapping typically maps output pixels to non-integer positions in the input image. To determine the grayscale value corresponding to that position, interpolation is necessary. Given the grayscale values ​​of the four vertices A, B, C, and D of the unit square enclosing point P, we aim to obtain their values ​​through bilinear interpolation.

[0054] Specifically, we can let f(x, y) be a function of two variables, and then use the bilinear equation... Let's define a hyperbolic paraboloid and fit it to four known points. Here, a, b, c, and d are undetermined coefficients used to describe the shape of the hyperbolic paraboloid. This can be solved using the known coordinates and grayscale values ​​of the four vertices A, B, C, and D. f(x,y) represents the function value at position (x,y), which can be understood as the grayscale value of a pixel. Figure 7 As shown, Figure 7 To illustrate the horizontal and vertical interpolation of vertices, we first interpolate points A and B, obtaining the following: (Equation 6) Where, f(x,y) ab f represents the function value obtained by interpolation at (x, y). a f b The function value (or pixel value) representing known points A and B.

[0055] Interpolating C and D yields: (Equation 7) f(x,y) cd f represents the function value obtained by interpolation at (x, y). c f d These are the function values ​​(or pixel values) for points C and D.

[0056] Using equations 6 and 7 above, we obtain the first interpolation point and the second interpolation point.

[0057] Step A24: Perform vertical interpolation based on the first interpolation point and the second interpolation point to obtain the interpolation point value; In practical implementation, the first and second interpolation points can be used as endpoints for vertical interpolation, resulting in: (Equation 8) By substituting equations 6 and 7 into equation 8 and combining them, we can obtain: (Equation 9) As shown in Equation 9 above, this formula is similar in form to the bilinear equation mentioned above, and therefore it is indeed bilinear. Verification confirms that Equation 9 does indeed satisfy the known values ​​of f(x, y) at the four vertices of the unit square.

[0058] Step A25: Obtain the target rectangular image based on the interpolation point values.

[0059] In practice, the interpolated point values ​​obtained by interpolating multiple points can be summarized to obtain the target rectangular image.

[0060] Step S40: Perform feature point detection and extraction on the target rectangular image to obtain initial feature point pairs.

[0061] It should be noted that feature point detection and extraction can be performed on the target rectangular image. For example, the SIFT (Scale-Invariant Feature Transform) algorithm can be used to detect and extract feature points to obtain matching feature point pairs. The feature points that can form feature point pairs are then connected to obtain the initial feature point pairs.

[0062] In one feasible implementation, step S40 may include steps A31 to A34: Step A31: Adjust the blur level of the target rectangular image to construct the scale space of the image; In practice, since the SIFT algorithm detects feature points based on changes in grayscale values, if the target rectangular image is a color image, it is first necessary to convert it to a grayscale image and apply techniques such as Gaussian filtering to remove image noise in order to reduce the false detection rate in subsequent steps.

[0063] SIFT feature point detection mainly involves altering the blur level of a target rectangular image to construct its scale space, which helps in detecting key points at different scales.

[0064] Step A32: Determine local extreme points in the scale space and use these local extreme points as candidate key points; It should be noted that after determining the scale space, extremum detection can be performed, specifically by finding local extrema points in the scale space as candidate keypoints. These points are local maxima or minima at their respective scales and locations.

[0065] Step A33: Calculate the position and scale of the candidate key points, and remove key points whose contrast and edge response do not meet the preset conditions according to the position and scale to obtain the target key points; It should be noted that after obtaining subsequent key points, suitable key points can be located again with precision. Therefore, the position and scale of candidate key points can be accurately calculated using the quadratic interpolation method. Based on the position and scale, key points that do not meet the preset conditions for contrast and edge response can be eliminated. The preset conditions are low contrast and excessive edge response. Thus, key points with low contrast and excessive edge response are eliminated, and the target key points are obtained.

[0066] Step A34: Perform feature point matching on the target key points to obtain initial feature point pairs.

[0067] It is understandable that after obtaining the target key points, one or more main directions can be assigned to each key point according to the gradient direction distribution of its neighboring pixels. Based on the selected directions, the gradient magnitude and direction in the area surrounding the key point are calculated to construct a 128-dimensional feature descriptor vector.

[0068] Then, using feature descriptor vectors and data structures such as KD-trees, the system finds the closest match between the feature descriptors of the current image and another image. Euclidean distance is typically used to measure similarity.

[0069] To improve matching accuracy, a distance ratio test can be introduced (for example, selecting matching pairs whose ratio of nearest neighbor distance to second nearest neighbor distance is less than a certain threshold) to eliminate bad matches and thus obtain initial feature point pairs.

[0070] For each pair of successfully matched feature points (i.e., corresponding feature points found in the two images), a straight line segment can be drawn connecting the two points between the original and target images. This step is usually done during visualization to visually represent the matching results.

[0071] Step S50: Use the random sampling consensus algorithm to filter the initial feature point pairs to obtain target feature point pairs.

[0072] It's important to note that during feature point pair matching, there are many mismatched feature point pairs. These mismatched pairs can affect the final image stitching result. Therefore, the RANSAC (Random Sample Consensus) algorithm can be used to filter the initial feature point pairs, resulting in filtered feature point pairs, i.e., the target feature point pairs. The RANSAC algorithm assumes that the data contains both correct and outlier data. It iteratively estimates the parameters of a mathematical model from a dataset containing mismatched points. Randomness and assumptions are the core of this algorithm, which can effectively eliminate mismatched points and robustly estimate the model parameters.

[0073] In one feasible implementation, step S50 may include steps A41 to A46: Step A41: Use the random sampling consensus algorithm to randomly select multiple sample data from the initial feature point pair, wherein the multiple sample data are not collinear; In practice, a random sampling consensus algorithm can be used to randomly extract multiple sample data from the initial feature point pair, such as four sample data, and the four sample data cannot be collinear.

[0074] Step A42: Calculate the transformation matrix parameters of the multiple sample data; It should be noted that the transformation matrix parameters corresponding to four sample data points can be calculated, for example, by selecting four non-collinear matching points and establishing a system of linear equations. The system of equations can then be converted into matrix form and solved to obtain the transformation matrix parameters.

[0075] Step A43: Obtain the transformation matrix based on the transformation matrix parameters; In practice, the transformation matrix can be obtained by combining the transformation matrix parameters.

[0076] Step A44: Calculate the projection error between each feature point pair in the initial feature point pair and the transformation matrix; It should be noted that the projection error between all data and the transformation matrix in the initial feature point pair can be calculated.

[0077] In practical implementation, for each data point {x i ,y i The error in calculating and transforming matrix M: (Equation 10) In equation 10 above, Let {x} be the coordinates of the transformed data points. For each data point {x} i ,y i The transformation matrix M maps the coordinates from the original coordinate system to the new coordinate system, resulting in the transformed coordinates {x}. i ′,y i ′}.

[0078] Step A45: When the projection error is less than a preset projection error threshold, retain the feature point pair; It should be noted that a projection error threshold can be set in advance. If the projection error is less than the preset projection error threshold, the system will automatically adjust the projection error accordingly. If the error is less than the threshold, then the point is considered an interior point, and the feature point pair is retained. The interior point set contains all points that satisfy the condition that the error is less than the threshold.

[0079] Step A46: When the projection error is greater than or equal to the projection error threshold, the feature point pair is removed from the initial feature point pair to obtain the target feature point pair.

[0080] If the projection error is greater than or equal to the projection error threshold, the feature point pair is discarded. If the number of elements in the current inlier set is greater than the optimal inlier set, the inlier set is updated again, and the iteration count is updated. During each iteration, the number of feature point pairs that have been accurately matched is counted. If the number of iterations reaches the set maximum number of iterations, the process stops and exits; otherwise, the iteration continues, and the above steps are repeated. Finally, the transformation matrix with the most correct matches is recalculated, and its matching status is checked to obtain the target feature point pair.

[0081] Step S60: Perform mean coordinate fusion processing on the target feature point pairs to obtain a fused mosaic panoramic image.

[0082] In practical implementation, after obtaining the target feature point pairs, data fusion processing can be performed to obtain the final synthesized image. The algorithm for image fusion is as follows: While Poisson fusion can effectively generate a fused image with global consistency, its problem lies in the fact that the equation is not closed, resulting in a large computational load and slow processing speed. Therefore, this embodiment uses Mean-Value Coordinates (MVC) fusion processing, which can be considered an alternative to Poisson fusion. It calculates interpolation based on median coordinates, which can well simulate Poisson fusion and approximate the final fusion effect. Furthermore, after optimization, MVC fusion has lower computational complexity and significantly improved processing speed compared to Poisson fusion. The main method of MVC fusion is to calculate the difference between pixels at the edges of the foreground and background images. This difference is used as boundary conditions and coordinates to calculate the interpolation between each pixel in the foreground image and the background image. The calculated interpolation is then added to the pixel value of each pixel to obtain the synthesized image.

[0083] In practice, a fused mosaic panoramic image can be obtained by performing mean coordinate fusion processing on the target feature point pairs.

[0084] In one feasible implementation, step S60 may include steps A51 to A55: Step A51: Obtain the target image based on the target feature point pairs; It should be noted that research on Poisson fusion has shown that interpolation is smoother when far from the boundary. Therefore, it is not necessary to calculate the interpolation for each pixel. Thus, it is possible to first obtain the target image composed of target feature point pairs.

[0085] Step A52: Perform boundary recognition on the target image using an edge detection algorithm to obtain the foreground image boundary and the background image boundary; In more specific implementations, edge detection algorithms (such as Canny, Sobel, etc.) can be used to perform edge recognition on the target image, thereby obtaining the boundaries between the foreground and background images.

[0086] Step A53: Based on the foreground image boundary and the background image boundary, segment the target image into multiple polygonal regions; In practice, the image is divided into several polygonal regions based on the available boundary information. The boundary of each region should match the outline of the actual object as closely as possible. Dividing the image into multiple polygonal regions facilitates subsequent interpolation calculations.

[0087] Step A54: Calculate the difference between multiple polygonal regions based on their internal pixels using the mean coordinate method; It should be noted that for each edge of each polygon, the difference between corresponding pixels in the foreground and background images is calculated. These differences will serve as boundary conditions for subsequent interpolation calculations.

[0088] Its internal pixels can be quickly calculated using median coordinates, such as Figure 8 As shown, Figure 8 This is a schematic diagram defining the angle of the mean coordinate system, and its boundary curve is... P = ( p 0, p 1, p 2, p n = p 0), p 0∈R2, where p i Denotes the vertices on the boundary, and p 0 and p n Given a common vertex, a closed two-dimensional polygon is formed. The value of a point x inside the polygon is the weighted average of all points on the curve of the closed two-dimensional polygon, that is: (Equation 11) In the above formula, the interpolation value is f(x), and λ is... i (x) is the boundary point corresponding to point x. p i The weighting coefficient, where x represents a point inside the polygon, needs to be calculated using the mean coordinate interpolation method, f( p i ) represents a boundary point p i The function value.

[0089] The weighting coefficients are calculated as follows: (Equation 12) (Equation 12) W i This indicates that point x corresponds to boundary point p. i The weight. This represents the sum of the weights of all boundary points relative to point x, used to normalize the weight coefficients. Weight W i The calculation is as follows: Equation 13: (Equation 13) α i 1 and α i These represent the angles formed by two adjacent vertices on the polygon boundary and the interior point x, respectively. Represents boundary point p i The Euclidean distance between the interior point x and the interior point x.

[0090] Step A55: Add the difference to the original pixel value of each pixel in the foreground image to obtain the fused mosaic panoramic image.

[0091] The calculated interpolation is added to the original pixel value of each pixel in the foreground image. The final composite image is generated by ensuring that the pixel values ​​of the composite image are within a reasonable range (e.g., between 0 and 255). Figure 9 As shown, Figure 9 This is a schematic diagram of image segmentation.

[0092] This embodiment provides a method for processing video data from a pipe network using a ring-shaped lens. The solution described in this application effectively processes video data from a pipe network using a ring-shaped lens, solving the technical problems of image distortion and inability to fuse the images. Through preprocessing and distortion correction of the panoramic ring image, the method enhances the ability to restore the panoramic ring image and reduces its distortion, thus solidifying the theoretical foundation for distortion correction of video data from a pipe network using a ring-shaped lens. The quality of the fused image is far superior to that of traditional line-stack methods, with a significant improvement in resolution and clarity. The image stitching and fusion method is simple and easy to implement, enabling intelligent and automated processing of panoramic images for pipe video inspection. This method meets the development needs of fixed-focus panoramic ring-shaped lens distortion correction in terms of simplicity, convenience, and low cost.

[0093] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 10 Step 20 includes steps S201 to S207: Step S201: Perform tangential direction image linearization processing on the coordinate-transformed ring video data, and map the wedge-shaped part in the coordinate-transformed ring video data to a rectangular array to obtain a rectangular image after tangential direction linearization processing.

[0094] It's important to note that unfolding the ring image formed by the panoramic ring lens into a rectangular image involves two linearization steps: tangential linearization and radial linearization. Accurately identifying and using the tangential direction helps preserve the original shape characteristics and avoid distortion when unfolding or deforming a ring structure. Radial operations, on the other hand, focus more on converting circular information into a more easily processed form (such as a rectangle), which may rely more directly on geometric relationships than on local linear approximation.

[0095] In the tangential direction image linearization process, the wedge-shaped portion of the coordinate-transformed annular video data can be mapped to a rectangular array, such as... Figure 11 As shown, Figure 11 This is a schematic diagram of the linearization process for a ring image. For linearization of the tangent direction, the resulting ring image can be represented by polar coordinates f(r, θ) with its center as the origin. Converting it to a rectangular coordinate system with the lower left corner of the image as the origin can be done according to the following relationship: (Equation 14) Through the above processing, a matrix image after tangent direction linearization can be obtained.

[0096] Step S202: Perform radial direction image linearization processing on the coordinate-transformed ring video data to obtain multiple outer circular points of the coordinate-transformed ring video data.

[0097] In practice, since the ring image is not linear radially, the resulting rectangular image also needs to be stretched vertically, a process known as radial linearization. Therefore, multiple outer circular points of the ring video data after coordinate transformation can be obtained first. The first step in linearization using a computer is to determine the image size. The coordinates of four points need to be input into the computer. The first three points are selected on the outer circumference of the image, thus obtaining three outer circular points.

[0098] Step S203: Obtain the center coordinates of the ring video data after coordinate transformation based on the multiple outer circular points, and calculate the outer circle radius using the center coordinates and the multiple outer circular points.

[0099] In practice, the image and the center (xc, yc) of the circle can be calculated based on three outer points, denoted as P1(x1, y1), P2(x2, y2), and P3(x3, y3). Using these three points, the center (xc, yc) and radius R can be found by solving simultaneous equations or using geometric methods. For example, the property that the perpendicular bisectors of the two chords formed by these three points intersect at the center can be used to solve the problem. Once the location of the center is found, the outer radius R can be obtained by calculating the distance from any point to the center, for example: .

[0100] Step S204: Determine the inner circle point on the inner circumference of the ring video data after coordinate transformation.

[0101] In practice, the remaining point is arbitrarily selected on the circumference of the inner circle. Therefore, the inner circle point P4(x4, y4) can be determined on the circumference of the inner circle in the ring video data after coordinate transformation.

[0102] Step S205: Determine the inner circle radius based on the coordinates of the inner circle point and the center of the circle.

[0103] In practice, the inner circle radius r can be calculated by computer based on the coordinates of the inner circle point and the center of the circle, and the calculation method is the same as that for the outer circle radius.

[0104] Step S206: Calculate the width and height of the image in pixels based on the inner circle radius and the outer circle radius, and obtain a rectangular image after radial linearization based on the width and the height.

[0105] It should be noted that the width and height of the image in pixels can be calculated based on the inner circle radius r and the outer circle radius R.

[0106] Step S207: Obtain the initial rectangular image based on the rectangular image after linearization of the tangent direction and the rectangular image after linearization of the radial direction.

[0107] In practice, after obtaining the width and height, the entire annular region can be unfolded. Let ρ be the maximum radial length, i.e., the radius R of the outer circle, and the radial angle θ be 0. The x, y coordinates of the rectangular image corresponding to this initial position are (0, 0). The radial angle is continuously increased until θ reaches 360°, at which point it returns to the initial position, completing one loop with the same radial length as the center. Then the radius is decreased, and the same process is repeated. Finally, the linearization of the entire image is completed. Figure 12 As shown, Figure 12This diagram illustrates the mapping of a circular image to a rectangular array. After selecting the radial length of the outer ring at one end of a specified quadrant, the radial angle continuously increases until it reaches the other end of the specified area. Then, the radius decreases, and the same process is repeated. After linearization, the number of image columns, determined by the increasing angle, is averaged to match the arc length of the outer ring of the selected quadrant. Following linearization, the width and height of the rectangular image also need adjustment to maintain the maximum scale. First, the width is adjusted; the number of image columns, determined by the increasing angle, is averaged to match the arc length of the outer ring of the selected quadrant. Next, system calibration determines the height of the rectangular image. System calibration can be achieved by covering the inner cavity of the pipe wall with a grid. Since the horizontal and vertical lines of the grid have the same spacing on the pipe wall, they should also have an equal spacing distribution on the linearized rectangular image. Using the spacing of the vertical lines as a reference, the spacing of the horizontal lines in the linearized rectangular image is determined to achieve this equal spacing distribution.

[0108] This embodiment effectively maps the wedge-shaped and radially curved portions of the annular lens into a standard rectangular array through linearization processing in the tangential and radial directions. This reduces geometric distortion of the image, ensuring accurate preservation of image precision and detail, and avoiding image distortion common in traditional processing methods. Mapping the coordinate-transformed annular video data into a rectangular image makes image processing more standardized and uniform. By obtaining precise inner and outer radii, as well as their width and height, it ensures that the image size and proportions meet actual requirements, facilitating subsequent image processing and algorithm applications.

[0109] For example, to help understand the implementation flow of the ring-shaped lens network video data processing method obtained by combining this embodiment with the above embodiment one, please refer to... Figure 13 , Figure 13 A simplified flowchart of a method for processing video data in a ring-shaped lens network is provided. Specifically, the method involves: acquiring a ring-shaped frame image; performing image coordinate transformation on the ring-shaped frame image; linearizing the transformed image, including tangential linearization and radial linearization, to obtain a linearized rectangular image; and performing pixel interpolation, feature extraction and registration, and fusion mosaicking on the rectangular image to obtain the final composite image.

[0110] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the video data processing method of the ring-shaped lens network of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0111] This application also provides a ring-shaped lens network video data processing device, the ring-shaped lens network video data processing device comprising: The acquisition module is used to acquire the collected ring-shaped lens network video data and perform coordinate transformation on the ring-shaped lens network video data to obtain the coordinate-transformed ring-shaped video data. The linearization processing module is used to perform image linearization processing on the coordinate-transformed ring video data to obtain an initial rectangular image. An interpolation processing module is used to perform linear interpolation processing on the initial rectangular image to obtain a target rectangular image; The feature point detection and extraction module is used to detect and extract feature points in the target rectangular image to obtain initial feature point pairs; The filtering module is used to filter the initial feature point pairs using a random sampling consensus algorithm to obtain target feature point pairs; The fusion module is used to perform mean coordinate fusion processing on the target feature point pairs to obtain a fused mosaic panoramic image.

[0112] The annular lens network video data processing device provided in this application, employing the annular lens network video data processing method described in the above embodiments, can solve the technical problems of image distortion and inability to fuse in the current analysis and processing of panoramic video images within the pipeline. Compared with the prior art, the beneficial effects of the annular lens network video data processing device provided in this application are the same as those of the annular lens network video data processing method provided in the above embodiments, and other technical features in the annular lens network video data processing device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0113] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for processing video data in a ring-shaped lens network, characterized in that, The method for processing video data in the ring-shaped lens network includes: Acquire the collected ring-shaped lens network video data, and perform coordinate transformation on the ring-shaped lens network video data to obtain the coordinate-transformed ring-shaped video data; The transformed ring-shaped video data is subjected to image linearization processing to obtain an initial rectangular image; The initial rectangular image is subjected to linear interpolation to obtain the target rectangular image; Feature point detection and extraction are performed on the target rectangular image to obtain initial feature point pairs; The initial feature point pairs are filtered using a random sampling consensus algorithm to obtain target feature point pairs; The target feature point pairs are subjected to mean coordinate fusion processing to obtain a fused mosaic panoramic image; The step of performing image linearization processing on the coordinate-transformed ring-shaped video data to obtain an initial rectangular image includes: Tangential direction image linearization processing is performed on the coordinate-transformed ring video data to map the wedge-shaped part in the coordinate-transformed ring video data to a rectangular array, thereby obtaining a rectangular image after tangential direction linearization processing; The radial direction image linearization processing is performed on the coordinate-transformed ring video data to obtain multiple outer circular points of the coordinate-transformed ring video data; The center coordinates of the ring video data after coordinate transformation are obtained based on the multiple outer circular points, and the outer radius is calculated using the center coordinates and the multiple outer circular points. Determine the inner circle point on the inner circumference of the ring video data after coordinate transformation; The radius of the inner circle is determined based on the coordinates of the inner circle point and the center of the circle. The width and height of the image in pixels are calculated based on the inner circle radius and the outer circle radius, and a rectangular image after radial linearization is obtained based on the width and the height. The initial rectangular image is obtained by linearizing the rectangular image in the tangent direction and the rectangular image in the radial direction. The step of performing mean coordinate fusion processing on the target feature point pairs to obtain the fused mosaic panoramic image includes: The target image is obtained based on the target feature point pairs; The target image is subjected to boundary recognition using an edge detection algorithm to obtain the foreground image boundary and the background image boundary. Based on the foreground image boundary and the background image boundary, the target image is segmented into multiple polygonal regions; The difference between multiple polygonal regions based on their internal pixels is calculated using the mean coordinate method. The difference is added to the original pixel value of each pixel in the foreground image to obtain the fused mosaic panoramic image; The step of detecting and extracting feature points from the target rectangular image to obtain initial feature point pairs includes: The blur level of the target rectangular image is adjusted to construct the scale space of the image; Local extreme points are identified in the scale space, and these local extreme points are used as candidate key points. Calculate the position and scale of the candidate key points, and remove key points whose contrast and edge response do not meet the preset conditions according to the position and scale to obtain the target key points; Feature point matching is performed on the target key points to obtain initial feature point pairs.

2. The method as described in claim 1, characterized in that, The steps of acquiring the collected ring-shaped lens network video data and performing coordinate transformation on the ring-shaped lens network video data to obtain the coordinate-transformed ring-shaped video data include: Acquire the collected video data from the ring-shaped lens network; A rectangular coordinate system is established based on the video data of the ring-shaped lens network. Obtain the coordinates of the center point in the video data of the ring-shaped lens network in the Cartesian coordinate system; Calculate the radial length and radial angle between any point in the ring-shaped lens network video data and the center point; The coordinates of the annular region of any point in the annular lens network video data are obtained based on the radial length and the radial angle. Based on the correspondence between the starting point of the annular region and the corresponding point in the rectangular region, the coordinates of the annular region are transformed to obtain the annular video data after coordinate transformation.

3. The method as described in claim 1, characterized in that, The step of performing linear interpolation on the initial rectangular image to obtain the target rectangular image includes: The initial rectangular image is back-mapping processed to map the initial rectangular image to a preset position on the coordinate-transformed ring video data; Obtain the vertices surrounding the preset position; The vertices are interpolated horizontally to obtain the first interpolation point and the second interpolation point; Vertical interpolation is performed based on the first interpolation point and the second interpolation point to obtain the interpolation point value; The target rectangular image is obtained based on the interpolation point values.

4. The method as described in claim 1, characterized in that, The step of using a random sampling consensus algorithm to filter the initial feature point pairs to obtain target feature point pairs includes: The random sampling consensus algorithm is used to randomly sample multiple data points from the initial feature point pair, and the multiple data points are not collinear. Calculate the transformation matrix parameters for multiple sample data; The transformation matrix is ​​obtained based on the transformation matrix parameters; Calculate the projection error between each feature point pair in the initial feature point pair and the transformation matrix; When the projection error is less than a preset projection error threshold, the feature point pair is retained; When the projection error is greater than or equal to the projection error threshold, the feature point pair is removed from the initial feature point pair to obtain the target feature point pair.

5. A video data processing device for a ring-shaped lens network, characterized in that, The device includes: The acquisition module is used to acquire the collected ring-shaped lens network video data and perform coordinate transformation on the ring-shaped lens network video data to obtain the coordinate-transformed ring-shaped video data. The linearization processing module is used to perform image linearization processing on the coordinate-transformed ring video data to obtain an initial rectangular image. An interpolation processing module is used to perform linear interpolation processing on the initial rectangular image to obtain a target rectangular image; The feature point detection and extraction module is used to detect and extract feature points in the target rectangular image to obtain initial feature point pairs; The filtering module is used to filter the initial feature point pairs using a random sampling consensus algorithm to obtain target feature point pairs; The fusion module is used to perform mean coordinate fusion processing on the target feature point pairs to obtain a fused mosaic panoramic image; The step of performing image linearization processing on the coordinate-transformed ring-shaped video data to obtain an initial rectangular image includes: Tangential direction image linearization processing is performed on the coordinate-transformed ring video data to map the wedge-shaped part in the coordinate-transformed ring video data to a rectangular array, thereby obtaining a rectangular image after tangential direction linearization processing; The radial direction image linearization processing is performed on the coordinate-transformed ring video data to obtain multiple outer circular points of the coordinate-transformed ring video data; The center coordinates of the ring video data after coordinate transformation are obtained based on the multiple outer circular points, and the outer radius is calculated using the center coordinates and the multiple outer circular points. Determine the inner circle point on the inner circumference of the ring video data after coordinate transformation; The radius of the inner circle is determined based on the coordinates of the inner circle point and the center of the circle. The width and height of the image in pixels are calculated based on the inner circle radius and the outer circle radius, and a rectangular image after radial linearization is obtained based on the width and the height. The initial rectangular image is obtained by linearizing the rectangular image in the tangent direction and the rectangular image in the radial direction. The step of performing mean coordinate fusion processing on the target feature point pairs to obtain the fused mosaic panoramic image includes: The target image is obtained based on the target feature point pairs; The target image is subjected to boundary recognition using an edge detection algorithm to obtain the foreground image boundary and the background image boundary. Based on the foreground image boundary and the background image boundary, the target image is segmented into multiple polygonal regions; The difference between multiple polygonal regions based on their internal pixels is calculated using the mean coordinate method. The difference is added to the original pixel value of each pixel in the foreground image to obtain the fused mosaic panoramic image; The step of detecting and extracting feature points from the target rectangular image to obtain initial feature point pairs includes: The blur level of the target rectangular image is adjusted to construct the scale space of the image; Local extreme points are identified in the scale space, and these local extreme points are used as candidate key points. Calculate the position and scale of the candidate key points, and remove key points whose contrast and edge response do not meet the preset conditions according to the position and scale to obtain the target key points; Feature point matching is performed on the target key points to obtain initial feature point pairs.

6. A ring-shaped lens network video data processing device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the ring-type lens network video data processing method as described in any one of claims 1 to 4.

7. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the ring-shaped lens network video data processing method as described in any one of claims 1 to 4.

Images