An imaging method and device based on optical flow and time flow, and an electronic device

Abstract

This invention discloses an imaging method, apparatus, and electronic device based on optical flow and temporal flow. The method includes acquiring a real-time image of a target scene; calculating the optical flow field motion vector of the real-time image based on optical flow; clustering the optical flow field motion vector to obtain a global motion vector; generating a temporal flow domain window based on the global motion vector; continuously generating a temporal image within the temporal flow domain window; acquiring the angle parameters of a virtual field of view; performing a perspective transformation on the temporal image based on the angle parameters; and generating and displaying a real-time imaging texture. This invention utilizes the consistency of optical flow and temporal flow in the production line to stitch together local real-time images into a long scroll image over time. It achieves separate processing and fusion imaging of fixed objects (i.e., production line equipment) and flowing objects (i.e., production line materials) on the production line, resulting in excellent graphic display effects.

Description

Technical Field

[0001] This application relates to the field of image imaging technology, and more specifically, to an imaging method, apparatus, and electronic device based on optical flow and temporal flow. Background Technology

[0002] Currently, there are generally two methods for graphically displaying production lines. One is to use on-site installed cameras to display real-time images. Since the cameras are fixed in place, free-viewpoint changes are not possible. The other method uses 3D modeling and simulation calculations. By inputting initial variables and simulating the workshop scene in real time, arbitrary viewpoint changes can be achieved, with excellent results. However, this method cannot reflect the real-time status of materials on the production line, and the simulation results often differ from the actual situation.

[0003] To achieve optimal graphical representation, the best approach is to combine live video and 3D models. However, due to lens distortion and perspective distortion in live video, it's difficult to achieve a perfect fit with 3D models. Therefore, there is currently no good solution to the challenge of accurately embedding live video into models for seamless display. Summary of the Invention

[0004] To address the aforementioned issues, embodiments of this application provide an imaging method, apparatus, and electronic device based on optical flow and temporal flow.

[0005] In a first aspect, embodiments of this application provide an imaging method based on optical flow and temporal flow, the method comprising: Acquire real-time images of the target scene, calculate the optical flow field motion vectors of the real-time images based on the optical flow method, and cluster the optical flow field motion vectors to obtain global motion vectors; A time-domain window is generated based on the global motion vector, and a time image is continuously generated in the time-domain window; Obtain the angle parameters of the virtual field of view, perform perspective transformation on the time image based on the angle parameters, and generate and display a real-time imaging texture.

[0006] Preferably, acquiring real-time images of the target scene includes: The camera images captured by each camera set at different shooting angles in the target scene are acquired, and the camera images are fused from multiple angles to obtain a real-time image of the target scene with the corresponding roll angle.

[0007] Preferably, before calculating the optical flow field motion vector of the real-time image based on the optical flow method, the method further includes: Gaussian filtering is applied to the real-time images of the scene.

[0008] Preferably, after calculating the optical flow field motion vector of the real-time image based on the optical flow method, the method further includes: Based on the optical flow field motion vector decomposition, the real-time image of the scene is used to determine the boundary of the pipeline region in the real-time image of the scene.

[0009] Preferably, after generating the watershed window based on the global motion vector, the method further includes: The tilt of the time-domain window is corrected based on perspective transformation.

[0010] Preferably, the step of continuously generating time images within the time domain window includes: A time image is generated in the time domain window based on the real-time on-site images, and newly received real-time on-site images are continuously added to the end of the time image.

[0011] Secondly, embodiments of this application provide an imaging device based on optical flow and temporal flow, the device comprising: The acquisition module is used to acquire real-time images of the target scene, calculate the optical flow field motion vector of the real-time image based on the optical flow method, and cluster the optical flow field motion vector to obtain the global motion vector; The generation module is used to generate a time-domain window based on the global motion vector, and continuously generate time images in the time-domain window; The display module is used to obtain the angle parameters of the virtual field of view, perform perspective transformation on the time image based on the angle parameters, and generate and display a real-time imaging texture.

[0012] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method provided as in the first aspect or any possible implementation of the first aspect.

[0013] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method provided as in the first aspect or any possible implementation thereof.

[0014] The beneficial effects of this invention are as follows: 1. By utilizing the consistency of optical flow and temporal flow in the production line, the images of local real-time scenes are stitched together into a long scroll image over time, realizing the separate processing and fusion imaging of fixed objects (i.e. production line equipment) and flowing objects (i.e. production line materials) on the production line, resulting in good graphic display effects.

[0015] 2. Depending on the virtual field of view, the viewing angle can be freely transformed according to the perspective transformation algorithm while displaying real-time images. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A schematic flowchart of an imaging method based on optical flow and temporal flow provided in an embodiment of this application; Figure 2 A schematic diagram of the structure of an imaging device based on optical flow and temporal flow provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0019] In the following description, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The following description provides multiple embodiments of this application, which can be substituted or combined with each other. Therefore, this application can also be considered to include all possible combinations of the same and / or different embodiments described. Thus, if one embodiment includes features A, B, and C, and another embodiment includes features B and D, then this application should also be considered to include embodiments containing one or more other possible combinations of A, B, C, and D, even if such embodiments are not explicitly described in the following text.

[0020] The following description provides examples and does not limit the scope, applicability, or examples set forth in the claims. Changes may be made to the function and arrangement of the described elements without departing from the scope of this application. Various processes or components may be appropriately omitted, substituted, or added to the examples. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with respect to some examples may be combined into other examples.

[0021] See Figure 1 , Figure 1 This is a schematic flowchart of an imaging method based on optical flow and temporal flow provided in an embodiment of this application. In this embodiment, the method includes: S101. Obtain real-time images of the target scene, calculate the optical flow field motion vector of the real-time image based on the optical flow method, and cluster the optical flow field motion vector to obtain the global motion vector.

[0022] The entity executing this application may be a cloud server.

[0023] In this embodiment, real-time images of the target scene captured by a camera are first acquired. Optical flow is then used to calculate the motion vectors of key points in motion within the real-time images, specifically the conveyor belt on the assembly line. Next, by clustering these vectors, a global motion vector is obtained, representing the direction and speed of the conveyor belt's movement.

[0024] Optical flow (or optic flow) is a concept related to motion detection of objects within the field of view. It describes the motion of an observed target, surface, or edge caused by the movement of the observer. When the human eye observes a moving object, the image of the object forms a series of continuously changing images on the retina. This series of continuously changing information constantly "flows" across the retina (i.e., the image plane), like a "flow" of light, hence the name optical flow. Optical flow expresses the changes in the image, and because it contains information about the target's motion, it can be used by the observer to determine the target's motion.

[0025] Specifically, the basic assumptions of the optical flow method are: (1) The brightness remains constant. That is, the brightness of the same target does not change when it moves between different frames. (2) The time is continuous or the motion is "small motion". That is, the change in time will not cause drastic changes in the target position, and the displacement between adjacent frames should be relatively small.

[0026] Consider the light intensity of a pixel I(x,y,t) in the first frame (where t represents its time dimension). It moves a distance of (dx,dy) to the next frame, taking time dt. Since it is the same pixel, we assume that the light intensity of this pixel remains unchanged before and after the movement, i.e.:

[0027] Performing a Taylor expansion on this expression, we obtain:

[0028] Therefore, we can conclude that:

[0029] Let u and v be the optical flow velocity vectors along the X-axis and Y-axis, respectively, then we get:

[0030] In summary, we can conclude that:

[0031] Among them, Ix, Iy, and It can all be obtained from image data, while (u, v) is the motion vector of the optical flow field.

[0032] The specific calculation process of the global motion vector is as follows: In practical calculations, n objects are divided into k clusters, ensuring high similarity within clusters and low similarity between clusters. The process is as follows: First, k objects are randomly selected, each initially representing the average or center of a cluster; for the remaining objects, they are assigned to the nearest cluster based on their distance from the cluster centers; then, the average of each cluster is recalculated. This process is repeated until the criterion function converges. Typically, the squared error criterion is used, defined as follows:

[0033] Where E is the sum of the squared errors of all objects in the database (i.e., the motion vectors of the optical flow field), p is a point in space, and mi is the average value of cluster Ci. This objective function aims to generate clusters that are as compact and independent as possible, using Euclidean distance as the distance metric, although other distance metrics can also be used.

[0034] Input: A database containing n objects and the number of clusters k; Output: k clusters that minimize the squared error criterion.

[0035] In one possible implementation, acquiring real-time images of the target scene includes: The camera images captured by each camera set at different shooting angles in the target scene are acquired, and the camera images are fused from multiple angles to obtain a real-time image of the target scene with the corresponding roll angle.

[0036] In this embodiment, multi-angle fusion imaging processing is performed on images from various cameras at different shooting angles in the target scene to obtain a real-time image of the scene corresponding to the roll angle, which is then used for subsequent optical flow calculation. Multi-angle fusion imaging can be implemented using estimation algorithms, random algorithms, artificial intelligence algorithms, recognition algorithms, etc.

[0037] In one possible implementation, before calculating the optical flow field motion vector of the real-time image based on the optical flow method, the method further includes: Gaussian filtering is applied to the real-time images of the scene.

[0038] In this embodiment, Gaussian filtering is a linear smoothing filter suitable for eliminating Gaussian noise. The filter is a mathematical model that converts the image data into energy, eliminating low-energy data. Noise belongs to the low-energy part.

[0039] Since the Fourier transform of a Gaussian function is still a Gaussian function, a Gaussian function can form a low-pass filter with smoothing performance in the frequency domain. Gaussian filtering can be achieved by multiplying in the frequency domain. Mean filtering performs local averaging on the signal, using the average value to represent the gray value of a pixel. A rectangular box filter performs independent smoothing on each component of this two-dimensional vector. Through calculation and transformation, a unit vector image is obtained. This 512×512 vector image is divided into 8×8 small regions, and then within each small region, the dominant direction within that region is counted, that is, the number of directions of points within that region is counted, and the direction with the most points is taken as the dominant direction of the region. Thus, a new 64×64 vector image is obtained. This new vector image can be further smoothed using a 3×3 template.

[0040] In one possible implementation, after calculating the optical flow field motion vector of the real-time image based on the optical flow method, the method further includes: Based on the optical flow field motion vector decomposition, the real-time image of the scene is used to determine the boundary of the pipeline region in the real-time image of the scene.

[0041] In this embodiment, since there are areas of pipeline (moving) and other background (stationary) in the actual shooting field of the camera, the pipeline area boundary can be detected by decomposing the optical flow field motion vector, so that the area used to generate the time image can be more accurately located in the future.

[0042] The process of determining the boundaries of the pipeline region can use edge detection algorithms. Edge detection algorithms are essentially filtering algorithms; the difference lies in the choice of filter, but the filtering rules are completely consistent. When constructing the edge detection operator, the concept of gradient is introduced. Gradient is a very important concept in artificial intelligence. The basic definition of the first-order differential of a one-dimensional function is:

[0043] Image filtering is generally based on grayscale images, so the image is two-dimensional. Therefore, the derivative of the corresponding two-dimensional function, i.e., the partial differential equation, is:

[0044] As shown in the formula above, the image gradient is the partial derivative of the current pixel with respect to the X and Y axes. Therefore, in image processing, the gradient can also be understood as the rate of change of pixel grayscale value. Due to the special nature of pixels, calculus in image processing is expressed as calculating the difference along the partial derivative direction of the current pixel. Therefore, in practical applications, differentiation is not required; only simple addition and subtraction operations are needed.

[0045] The other concept, the magnitude of the gradient, represents the amount of increase per unit distance in the direction of the maximum rate of change of f(x, y), i.e., the magnitude of the gradient is:

[0046] The direction of the gradient is:

[0047] After introducing the concept of gradient, we need to use several basic edge detection filters: Prewitt, Sobel, and Roberts operators.

[0048] The Prewitt operator combines difference operations with neighborhood averaging. Its convolution template is as follows:

[0049] The Roberts gradient operator uses the difference between two diagonally adjacent pixel values ​​as a metric, and its calculation method is as follows:

[0050] If written in the form of a convolution operation, the convolution kernels are as follows:

[0051] The Sobel operator is similar to the Prewitt operator, but it takes into account the different influence levels of adjacent pixels, so it uses a weighted average. Its convolution template is as follows:

[0052] Other edge detection filters will not be introduced one by one.

[0053] S102. Generate a time-domain window based on the global motion vector, and continuously generate time images in the time-domain window.

[0054] In this embodiment, since the pipeline maintains consistency in temporal and optical flow, the temporal flow domain window corresponding to the image in time can be determined based on the position of the global motion vector generated by the optical flow. In this way, a temporal image can be continuously stitched together in the temporal flow domain window in chronological order, and the temporal image can be used to represent and display the real-time image of the scene that changes over time.

[0055] In one possible implementation, after generating the watershed window based on the global motion vector, the method further includes: The tilt of the time-domain window is corrected based on perspective transformation.

[0056] In this embodiment of the application, since the subsequent process will be implemented entirely based on the time-domain window, tilt correction will also be performed to ensure its correctness.

[0057] Perspective transformation essentially projects an image onto a new view plane, and its general transformation formula is:

[0058]

[0059] In one possible implementation, the continuous generation of time images within the time domain window includes: A time image is generated in the time domain window based on the real-time on-site images, and newly received real-time on-site images are continuously added to the end of the time image.

[0060] In this embodiment, after constructing the time-domain window, the dynamically changing conveyor belt section in the real-time image is displayed as a time image within the time-domain window. To ensure that the time image in the time-domain window changes over time and reflects the actual shooting situation, images are continuously added to the end of the time image to stitch together a long scroll that moves within the time-domain window, reflecting the real-time image.

[0061] S103. Obtain the angle parameters of the virtual field of view, perform perspective transformation on the time image based on the angle parameters, and generate and display a real-time imaging texture.

[0062] In this embodiment, the time image generated by the aforementioned process can only display the real-time image corresponding to a fixed angle and orientation in the virtual scene. Depending on the virtual field of view, when it is necessary to display the actual image at different angles, the image will be subjected to perspective transformation again based on parameters such as distance, roll angle, yaw angle, and pitch angle of the virtual field of view to generate the corresponding real-time imaging texture, which will then be displayed to achieve the function of free viewpoint transformation, thereby realizing both real-time image display and free viewpoint transformation.

[0063] The following will be combined with the appendix Figure 2 This application provides a detailed description of the imaging apparatus based on optical flow and temporal flow provided in its embodiments. It should be noted that the appendix... Figure 2 The imaging apparatus shown is based on optical flow and temporal flow and is used to perform the present application. Figure 1 The methods shown in the embodiments are for illustrative purposes only, illustrating the parts relevant to the embodiments of this application. For specific technical details not disclosed, please refer to this application. Figure 1 The example shown.

[0064] Please see Figure 2 , Figure 2 This is a schematic diagram of the structure of an imaging device based on optical flow and temporal flow provided in an embodiment of this application. Figure 2 As shown, the device includes: The acquisition module 201 is used to acquire real-time images of the target scene, calculate the optical flow field motion vector of the real-time image based on the optical flow method, and cluster the optical flow field motion vector to obtain the global motion vector; The generation module 202 is used to generate a time-domain window based on the global motion vector, and continuously generate a time image in the time-domain window; The display module 203 is used to obtain the angle parameters of the virtual field of view, perform perspective transformation on the time image based on the angle parameters, and generate and display a real-time imaging texture.

[0065] In one possible implementation, the acquisition module 201 includes: The acquisition unit is used to acquire camera images captured by cameras set at different shooting angles in the target scene, perform multi-angle fusion imaging on the camera images, and obtain a real-time image of the target scene corresponding to the roll angle.

[0066] In one possible implementation, the acquisition module 201 further includes: The filtering unit is used to perform Gaussian filtering on the real-time image of the scene.

[0067] In one possible implementation, the acquisition module 201 further includes: The decomposition unit is used to decompose the real-time image based on the optical flow field motion vector and determine the pipeline region boundary of the real-time image.

[0068] In one possible implementation, the generation module 202 includes: The correction unit is used to perform tilt correction on the time-domain window based on perspective transformation.

[0069] In one possible implementation, the generation module 202 further includes: The generation unit is used to generate a time image in the time domain window based on the real-time on-site image, and continuously add newly received real-time on-site images to the end of the time image.

[0070] Those skilled in the art will clearly understand that the technical solutions of the embodiments of this application can be implemented by means of software and / or hardware. In this specification, "unit" and "module" refer to software and / or hardware that can independently complete or cooperate with other components to complete a specific function, wherein the hardware may be, for example, a field-programmable gate array (FPGA), an integrated circuit (IC), etc.

[0071] Each processing unit and / or module in the embodiments of this application can be implemented by an analog circuit that implements the functions described in the embodiments of this application, or by software that executes the functions described in the embodiments of this application.

[0072] See Figure 3 It shows a schematic diagram of the structure of an electronic device according to an embodiment of this application, which can be used to implement... Figure 1 The method in the illustrated embodiment. (As shown) Figure 3 As shown, the electronic device 300 may include: at least one central processing unit 301, at least one network interface 304, user interface 303, memory 305, and at least one communication bus 302.

[0073] The communication bus 302 is used to enable communication between these components.

[0074] The user interface 303 may include a display screen and a camera. Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0075] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0076] The central processing unit 301 may include one or more processing cores. The central processing unit 301 connects to various parts within the electronic device 300 using various interfaces and lines. It executes various functions of the terminal and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 305, and by calling data stored in the memory 305. Optionally, the central processing unit 301 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The central processing unit 301 may integrate one or a combination of several of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display on the screen; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the central processing unit 301 and may be implemented as a separate chip.

[0077] The memory 305 may include random access memory (RAM) or read-only memory. Optionally, the memory 305 may include a non-transitory computer-readable storage medium. The memory 305 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 305 may also be at least one storage device located remotely from the aforementioned central processing unit 301. Figure 3 As shown, the memory 305, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and program instructions.

[0078] exist Figure 3In the illustrated electronic device 300, the user interface 303 is mainly used to provide an input interface for the user and to acquire user input data; while the central processing unit 301 can be used to call the imaging application based on optical flow and temporal flow stored in the memory 305, and specifically perform the following operations: Acquire real-time images of the target scene, calculate the optical flow field motion vectors of the real-time images based on the optical flow method, and cluster the optical flow field motion vectors to obtain global motion vectors; A time-domain window is generated based on the global motion vector, and a time image is continuously generated in the time-domain window; Obtain the angle parameters of the virtual field of view, perform perspective transformation on the time image based on the angle parameters, and generate and display a real-time imaging texture.

[0079] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0080] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0081] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0082] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0083] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0084] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0085] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0086] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0087] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of embodiments of this disclosure upon considering the specification and practicing the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described herein. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. An imaging method based on optical flow and temporal flow, characterized in that, The method includes: Acquire real-time images of the target scene, calculate the optical flow field motion vectors of the real-time images based on the optical flow method, and cluster the optical flow field motion vectors to obtain global motion vectors; A time-domain window is generated based on the global motion vector, and a time image is continuously generated in the time-domain window; Obtain the angle parameters of the virtual field of view, perform perspective transformation on the time image based on the angle parameters, and generate and display a real-time imaging texture. The acquisition of real-time images of the target scene includes: Acquire camera images captured by cameras set at different shooting angles in the target scene, perform multi-angle fusion imaging on the camera images to obtain a real-time image of the target scene corresponding to the roll angle; Following the generation of the watershed window based on the global motion vector, the method further includes: The time-domain window is tilted and corrected based on perspective transformation. The step of continuously generating time images within the time domain window includes: Based on the real-time images of the scene, a time image is generated in the time domain window, and newly received real-time images of the scene are continuously added to the end of the time image to stitch together a long scroll that moves within the time domain window.

2. The method of claim 1, wherein, Before calculating the optical flow field motion vector of the real-time image based on the optical flow method, the method further includes: Gaussian filtering is applied to the real-time images of the scene.

3. The method of claim 1, wherein, After calculating the optical flow field motion vector of the real-time image based on the optical flow method, the method further includes: Based on the optical flow field motion vector decomposition, the real-time image of the scene is used to determine the boundary of the pipeline region in the real-time image of the scene.

4. An imaging apparatus based on optical flow and time flow, characterized by, The device includes: The acquisition module is used to acquire real-time images of the target scene, calculate the optical flow field motion vector of the real-time image based on the optical flow method, and cluster the optical flow field motion vector to obtain the global motion vector; The generation module is used to generate a time-domain window based on the global motion vector, and continuously generate time images in the time-domain window; The display module is used to obtain the angle parameters of the virtual field of view, perform perspective transformation on the time image based on the angle parameters, and generate and display a real-time imaging texture. The acquisition module includes: The acquisition unit is used to acquire camera images captured by each camera set at different shooting angles in the target scene, perform multi-angle fusion imaging on each of the camera images, and obtain a real-time image of the target scene corresponding to the roll angle. The generation module includes: The correction unit is used to perform tilt correction on the time-flow domain window based on perspective transformation; The generation module also includes: The generation unit is used to generate a time image based on the real-time on-site image in the time domain window, and continuously add newly received real-time on-site images to the end of the time image to stitch them together into a long scroll that moves within the time domain window.

5. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1-3.

6. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-3.

Images