Fluid motion characteristic description method, device and medium

Through dense optical flow algorithm and area division technology, the problem of difficult to obtain feature points and insufficient sparse optical flow in the description of fluid motion characteristics is solved, and more accurate description and tracking of fluid motion characteristics is achieved, suitable for the motion characteristics of fluids such as water and air analyze.

CN114862903BActive Publication Date: 2024-11-12CHINA TELECOM CORP LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110075448.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-01-20
Publication Date
2024-11-12
Estimated Expiration
2041-01-20

AI Technical Summary

Technical Problem

When describing fluid motion characteristics, especially fluids such as water and air, there are problems such as difficult to obtain feature points, difficult to continuously track, and sparse light flow is insufficient to accurately fit the fluid force field, resulting in insufficient description of motion characteristics.

Method used

Using a fluid motion feature description method based on dense optical flow, an optical flow field is generated by receiving the time sequence of the fluid, and divided according to the particle size of the specific area, the histogram feature value of the improved optical flow direction is calculated, and whether the area needs to be redied , and finally calculate the fluid motion characteristics, using areas instead of feature points for description and tracking.

Benefits of technology

Improve the accuracy and universality of fluid motion characteristics description, can more accurately fit the fluid force field, track the optical flow trajectory for a longer period of time, provide more comprehensive flow statistics and motion pattern characteristics, suitable for monitoring equipment and computers Vision technology.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114862903B_ABST
    Figure CN114862903B_ABST
Patent Text Reader

Abstract

The present disclosure relates to a method, device and medium for describing fluid motion characteristics. The present disclosure provides a method for describing fluid motion characteristics, including: receiving a time-series image sequence of a fluid; for each pixel of multiple images in the image sequence, using a dense optical flow algorithm to generate multiple optical flow fields corresponding to the image sequence; dividing each optical flow field into multiple regions according to a specific region division granularity; calculating an improved optical flow direction histogram eigenvalue of each region in the multiple regions; determining whether it is necessary to re-divide the region of the optical flow field according to the improved optical flow direction histogram eigenvalue of each region; if it is necessary to re-divide the region of the optical flow field, then changing the specific region division granularity and repeating the above-mentioned steps of dividing the region, calculating the improved optical flow direction histogram eigenvalue and determining whether to re-divide; and calculating the fluid motion characteristics of each region in the multiple regions.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to a method, device and medium for describing fluid motion characteristics, and more specifically, to a method, device and medium for describing fluid motion characteristics by identifying and analyzing a sequence of fluid images and calculating fluid motion characteristics. Background Art

[0002] In recent years, natural disasters such as floods have occurred frequently, drainage and sewage discharge still exist, and behaviors that are harmful to the air, such as burning straw, are also rampant. It is very necessary to prevent, control and manage these phenomena.

[0003] Fluid is a substance that can flow, and is a general term for liquids and gases. Fluids have the following characteristics: they will continuously deform when subjected to any tiny shear force, and have no fixed outline; secondly, fluids are seriously disturbed by the environment, such as reflections and shadows caused by refraction and reflection, and they usually undergo non-rigid motions such as oscillation, diffusion, and deformation when subjected to external forces.

[0004] There are several solutions for describing fluid motion characteristics based on computer vision technology: 1) A solution combined with hardware, where the fluid is sent into the measured pipeline, and the fluid medium parameter measurement instrument and fluid image comprehensive measurement are used; 2) A solution combined with markers, where the markers are placed on the moving fluid, and the fluid motion is estimated by tracking and analyzing the markers in the image; 3) Based on the image sequence of the fluid, computer vision methods are used for identification. Compared with the first two solutions, the third solution does not require hardware conditions and markers, and can be completed based on existing monitoring equipment, which has better universality. Summary of the invention

[0005] A brief overview of the disclosure is given below in order to provide a basic understanding of some aspects of the disclosure. However, it should be understood that this overview is not an exhaustive overview of the disclosure. It is not intended to identify the key or important parts of the disclosure, nor is it intended to limit the scope of the disclosure. Its purpose is simply to give some concepts of the disclosure in a simplified form as a prelude to a more detailed description given later.

[0006] According to one aspect of the present disclosure, a method for describing fluid motion characteristics is provided, comprising: receiving a time-series image sequence of a fluid; for each pixel of a plurality of images in the image sequence, using a dense optical flow algorithm to generate a plurality of optical flow fields corresponding to the image sequence; dividing each optical flow field into a plurality of regions according to a specific region division granularity; calculating an improved optical flow direction histogram eigenvalue of each region in the plurality of regions; determining whether it is necessary to re-divide the region of the optical flow field according to the improved optical flow direction histogram eigenvalue of each region; if it is necessary to re-divide the region of the optical flow field, then changing the specific region division granularity and repeating the above-mentioned steps of dividing the region, calculating the improved optical flow direction histogram eigenvalue and determining whether to re-divide; and calculating the fluid motion characteristics of each region in the plurality of regions.

[0007] According to another aspect of the present disclosure, a fluid motion characteristic description device is provided, comprising: a memory having instructions stored thereon; and a processor configured to execute the instructions stored on the memory to perform the method as described above.

[0008] According to another aspect of the present disclosure, a computer-readable storage medium is provided, which includes computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the one or more processors execute the method according to the above aspects of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0010] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0011] Figure 1 The structure and principle of the SUSAN algorithm of the prior art are shown;

[0012] Figure 2 A flow chart of a method for describing fluid motion characteristics according to an embodiment of the present disclosure is shown;

[0013] Figure 3 is a processing sequence diagram for illustrating a fluid motion characterization method.

[0014] Figure 4 A schematic diagram of a single trajectory tracking process is shown;

[0015] Figure 5 An exemplary configuration of a computing device that can implement an embodiment according to the present disclosure is shown. DETAILED DESCRIPTION

[0016] The following detailed description is made with reference to the accompanying drawings, and the following detailed description is provided to assist in a comprehensive understanding of the various exemplary embodiments of the present disclosure. The following description includes various details to assist in understanding, but these details are considered to be examples only and not to limit the present disclosure, which is defined by the appended claims and their equivalents. The words and phrases used in the following description are only used to enable a clear and consistent understanding of the present disclosure. In addition, for the sake of clarity and brevity, descriptions of well-known structures, functions, and configurations may be omitted. Those of ordinary skill in the art will recognize that various changes and modifications may be made to the examples described herein without departing from the spirit and scope of the present disclosure.

[0017] The present disclosure is based on the third solution described in the background technology, and uses computer vision algorithms to describe the motion characteristics of fluids. Compared with rigid bodies and solids, the identification of fluids is more complicated due to their characteristics.

[0018] Optical flow is the instantaneous speed of the pixel movement of a moving object in space on the observation imaging plane, including direction and size. The method to solve the optical flow is to use the change of pixels in the time domain in the image sequence and the correlation between adjacent frames to find the corresponding relationship between the previous frame and the current frame, and calculate the motion information of the object between adjacent frames. Optical flow has two basic assumptions: 1) The brightness is constant, and the brightness of the same point will not change with the change of time; 2) The change of time will not cause a drastic change in position.

[0019] Among similar technologies, the currently used sparse optical flow solution method. First, the SUSAN algorithm is used to obtain feature points, then the feature point pairs are matched in two adjacent frames, the vector between the two feature point pairs is used as the optical flow vector, and finally the inverse distance interpolation is used to obtain the gridded flow velocity field. Structure and principle Figure 1 shown.

[0020] This method has major problems in practical application. Its essence is to construct an optical flow field around feature points to simulate the motion field of the fluid. Fluids such as water and air are relatively evenly distributed, and feature points are often extreme points and edge point sets of pixels, so the feature points of the fluid are difficult to obtain. If there are interference factors such as time, floating objects, and shores in the image sequence, the feature points will often focus on this part, which is why the marker tracking is combined as mentioned in the second solution above. This method also brings the following problems: 1) The fluid has deformation characteristics, and feature points are difficult to track continuously; 2) The sparse optical flow constructed using feature points is not sufficient to accurately fit the fluid force field; 3) The flow field is gridded by sparse optical flow to obtain local information, lacking an overall description of motion characteristics.

[0021] The present disclosure provides a method for describing fluid motion features based on dense optical flow, including: acquiring an image sequence, constructing a dense optical flow queue, dividing regions, solving features within the region, and solving motion laws and trends.

[0022] Figure 2 is a flow chart illustrating a fluid motion characterization method 100 according to one embodiment of the present disclosure. Figure 3 1 is a process sequence diagram for explaining the method 100. Figure 2 As shown, method 100 includes receiving a time-series image sequence of a fluid (101); for each pixel of a plurality of images in the image sequence, using a dense optical flow algorithm to generate a plurality of optical flow fields corresponding to the image sequence (102); dividing each optical flow field into a plurality of regions according to a specific region division granularity (103); calculating an improved optical flow directional histogram feature value (HOF (Histograms of Oriented Optical Flow) + feature) of each region in the plurality of regions (104); determining whether it is necessary to re-divide the region of the optical flow field according to the improved optical flow directional histogram feature value of each region (105); if it is necessary to re-divide the region of the optical flow field, then changing the specific region division granularity and repeating the above-mentioned steps of dividing the region, calculating the improved optical flow directional histogram feature value and determining whether to re-divide; and calculating the fluid motion feature of each region in the plurality of regions (106).

[0023] In step 102, an optical flow field sequence of length n may be maintained, where the length n is related to the length of the trajectory of the fluid flow to be tracked.

[0024] In step 102, for example, a dense inverse search (Dense Inverse Search-based method) algorithm may be used to solve a dense optical flow field for two adjacent time-series images, and the result is inserted into an optical flow field trajectory queue.

[0025] In step 103, different images may have different granularities due to the form of the fluid and the shooting distance, so it is necessary to select the granularity of division according to the specific situation. As an example, a pyramid model can be used to find a suitable region division granularity. A pyramid model refers to a model that gradually changes the region division granularity. For example, the region size of each layer in the pyramid model can be increased or decreased by a predetermined multiple compared to the region size of the adjacent layer. The initial region division granularity can be the maximum division granularity, the minimum division granularity, or a certain intermediate division granularity.

[0026] In step 104, HOF+ is an improved HOF algorithm, which changes the angle division area from 180 degrees to 360 degrees, so as to be able to distinguish some different movements, for example, in the horizontal direction, movement in one direction and oscillating movement. The direction with the largest weight of the HOF+ feature can be selected to represent the flow direction of the region.

[0027] In step 105, as an example, if the weights of the characteristic values ​​of HOF+ obtained in each direction are not much different (for example, the weight difference is less than a predetermined threshold), or no peak appears (that is, there is no obvious tendency in the flow direction), there may be external force interference in the area, unreasonable area division, etc., and you can consider reselecting the area division granularity.

[0028] If it is determined that the region division granularity needs to be reselected, then the above steps 103, 104 and 105 are repeated according to the reselected region division granularity. This process can be iteratively performed until the region division granularity does not need to be reselected.

[0029] In step 106, the optical flow vector can be calculated for each pixel in each region to obtain the movement direction and displacement distance of the fluid corresponding to each pixel. All pixels combined roughly correspond to the movement of the fluid. More specifically, for example, a threshold can be set for the optical flow vector of each pixel in each region, and the optical flow vectors with a length less than the threshold are removed, and the number of optical flow vectors greater than the threshold in the region is calculated as the flow value, and the average length of the optical flow vector is calculated as the flow velocity.

[0030] Step 106 may also include, for the multiple optical flow fields, performing single-trace tracking on the flow of each region according to the maximum weight of the weights of the improved optical flow direction histogram feature values ​​of each region in various directions, so as to obtain the fluid motion trajectory.

[0031] Alternatively, step 106 may further include, for the multiple optical flow fields, performing multi-trajectory tracking on the flow of each region according to a predetermined plurality of higher weights in the weights of the improved optical flow direction histogram feature values ​​of each region in various directions, so as to obtain the fluid motion trajectory.

[0032] More specifically, in step 106, for a plurality of continuous dense optical flow fields, the motion curve can be fitted according to the total flow rate, total flow velocity, flow velocity and flow direction of the HOF+ features in different dimensions (direction division) in the same area, and the motion trajectory can be constructed according to the regional direction. By comparing the weights of the HOF+ features in each direction, the largest weight can be selected for single-track tracking, or TOP K (i.e., the largest K) weights (i.e., multiple higher weights are predetermined) can be selected for multi-track tracking. When performing multi-track tracking, algorithms related to the maximum flow problem (i.e., based on the maximum flow principle) can be used. At the same time, during the tracking process, attention should be paid to weight / flow changes. If the fluctuation is large, or the change is greater than / less than the threshold, the tracking can be terminated.

[0033] Figure 4 A schematic diagram of a single trajectory tracking process is shown. Figure 4 For region A, it is calculated that the flow direction of the region points to region B, so a path A->B is established. Similarly, the flow direction of region B points to region C, so B->C is added to the path. And so on, a fluid motion trajectory of A->B->C->D is finally obtained. Since the present disclosure uses regions instead of feature points to calculate the feature values ​​of the regions instead of the feature values ​​of the feature points, there is no need to worry about deformation, drift, etc. of the feature points causing the inability to track during the trajectory tracking process.

[0034] According to the method for describing fluid motion characteristics disclosed in the present invention, the motion characteristics of the fluid can be described using monitoring equipment and computer vision technology, which is universal compared to the specific equipment detection in the prior art. In addition, by using dense optical flow for solution, the fluid force field is more accurately fitted, and the generalization of the model is improved. In addition, according to the fluid characteristics, the local motion characteristics are calculated by regional division, and the motion tracking of feature points is converted into regional motion tracking, which can track optical flow trajectories for longer periods of time. In addition, the present disclosure proposes characteristic calculation methods such as flow statistics and motion laws, so that the description of the motion characteristics of the fluid is more comprehensive and accurate.

[0035] This disclosure proposes for the first time to use dense optical flow field to fit the fluid motion field. Fluid motion meets the definition and assumptions of optical flow, and the global smoothness assumption introduced by dense optical flow also conforms to the characteristics of fluid motion. By calculating the instantaneous velocity vector of each pixel point, the motion direction and displacement distance of the fluid corresponding to the pixel point are obtained, and the entire pixel point set is combined to roughly correspond to the motion of the fluid.

[0036] The present disclosure also proposes for the first time to use region division to calculate feature values ​​instead of feature points for description and tracking. Due to the characteristics of fluid media such as uniformity, easy deformation, and easy interference, the detection and tracking of feature points are relatively difficult and will introduce large errors. Therefore, the present disclosure uses regions instead of feature points and uses HOF+ to calculate regional feature values ​​to describe and track the characteristics of the fluid.

[0037] The present invention proposes a method for describing fluid motion characteristics based on the dense optical flow algorithm. By collecting continuous fluid image data, the force field formed by the fluid motion is approximately fitted using the dense optical flow field to obtain the instantaneous speed and direction of each pixel of the fluid; at the same time, according to the fluid characteristics, the HOF feature is improved, the fluid image is divided into regions, and the fluid motion feature descriptor is designed based on the region. The descriptor finally obtained can better characterize the fluid motion characteristics, and can be used to identify the fluid's motion direction, flow rate, motion speed, trend state, motion law, etc., and provide an analytical basis for judging scenes such as water flow, dam breach, sewage discharge, and smoke.

[0038] Figure 5 An exemplary configuration of a computing device 1200 capable of implementing embodiments according to the present disclosure is shown.

[0039] The computing device 1200 is an example of a hardware device to which the above aspects of the present disclosure can be applied. The computing device 1200 can be any machine configured to perform processing and / or computing. The computing device 1200 can be, but is not limited to, a workstation, a server, a desktop computer, a laptop computer, a tablet computer, a personal data assistant (PDA), a smart phone, a vehicle-mounted computer, or a combination thereof.

[0040] like Figure 5As shown, the computing device 1200 may include one or more elements that can be connected or communicated with the bus 1202 via one or more interfaces. The bus 1202 may include, but is not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus, etc. The computing device 1200 may include, for example, one or more processors 1204, one or more input devices 1206, and one or more output devices 1208. The one or more processors 1204 may be any kind of processor, and may include, but is not limited to, one or more general-purpose processors or special-purpose processors (such as special-purpose processing chips). The processor 1204 may be configured to implement the above-mentioned fluid motion feature description method, for example. The input device 1206 may be any type of input device capable of inputting information to the computing device, and may include, but is not limited to, a mouse, a keyboard, a touch screen, a microphone, and / or a remote controller. The output device 1208 may be any type of device capable of presenting information, and may include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer.

[0041] The computing device 1200 may also include or be connected to a non-transitory storage device 1214, which may be any non-transitory storage device that can implement data storage, and may include, but is not limited to, a disk drive, an optical storage device, a solid-state memory, a floppy disk, a flexible disk, a hard disk, a tape, or any other magnetic medium, a compressed disk or any other optical medium, a cache memory, and / or any other storage chip or module, and / or any other medium from which a computer can read data, instructions, and / or code. The computing device 1200 may also include a random access memory (RAM) 1210 and a read-only memory (ROM) 1212. The ROM 1212 may store programs, utilities, or processes to be executed in a non-volatile manner. The RAM 1210 may provide volatile data storage and store instructions related to the operation of the computing device 1200. The computing device 1200 may also include a network / bus interface 1216 coupled to a data link 1218. The network / bus interface 1216 may be any type of device or system capable of enabling communication with an external device and / or a network, and may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication device, and / or a chipset (such as a Bluetooth TM equipment, 802.11 equipment, WiFi equipment, WiMax equipment, cellular communication facilities, etc.).

[0042] The present disclosure may be implemented as any combination of an apparatus, a system, an integrated circuit, and a computer program on a non-transitory computer-readable medium. One or more processors may be implemented as an integrated circuit (IC), an application specific integrated circuit (ASIC), or a large scale integrated circuit (LSI), a system LSI, a super LSI, or a super LSI component that performs some or all of the functions described in the present disclosure.

[0043] The present disclosure includes the use of software, applications, computer programs, or algorithms. The software, applications, computer programs, or algorithms may be stored on a non-transitory computer-readable medium to enable a computer such as one or more processors to perform the above steps and the steps described in the accompanying drawings. For example, one or more memories store the software or algorithm in executable instructions, and one or more processors may associate a set of instructions to execute the software or algorithm to provide various functions according to the embodiments described in the present disclosure.

[0044] Software and computer programs (which may also be referred to as programs, software applications, applications, components or code) include machine instructions for a programmable processor and may be implemented in a high-level procedural language, an object-oriented programming language, a functional programming language, a logic programming language, or an assembly language or a machine language. The term "computer-readable medium" refers to any computer program product, apparatus or device for providing machine instructions or data to a programmable data processor, such as magnetic disks, optical disks, solid-state storage devices, memories and programmable logic devices (PLDs), including computer-readable media that receive machine instructions as computer-readable signals.

[0045] For example, computer readable media may include dynamic random access memory (DRAM), random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), compact disk read-only memory (CD-ROM) or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to carry or store the desired computer readable program code in the form of instructions or data structures and can be accessed by a general or special purpose computer or a general or special purpose processor. As used herein, disk or disc includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks usually reproduce data magnetically, while discs reproduce data optically by laser. Combinations of the above are also included in the scope of computer readable media.

[0046] The subject matter of the present disclosure is provided as an example of devices, systems, methods and programs for performing the features described in the present disclosure. However, in addition to the above-mentioned features, other features or variations may also be expected. It is expected that the implementation of the components and functions of the present disclosure may be completed with any emerging technology that may replace any of the above-mentioned implementation technologies.

[0047] In addition, the above description provides examples without limiting the scope, applicability or configuration set forth in the claims. Without departing from the spirit and scope of the present disclosure, the functions and arrangements of the elements discussed may be changed. Various embodiments may appropriately omit, replace or add various processes or components. For example, features described with respect to certain embodiments may be combined in other embodiments.

[0048] In addition, in the description of the present disclosure, the terms “first”, “second”, “third”, etc. are used for descriptive purposes only and cannot be understood as indicating or implying relative importance and order.

[0049] Similarly, while operations are depicted in a particular order in the drawings, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed to achieve desired results. In certain circumstances, multitasking and parallel processing can be advantageous.

Claims

1. A method for describing fluid motion characteristics, comprising: receiving a time-sequential image sequence of the fluid; For each pixel of a plurality of images in the image sequence, a dense optical flow algorithm is used to generate a plurality of optical flow fields corresponding to the image sequence; Divide each optical flow field into multiple regions according to the specific region division granularity; Calculating a feature value of an improved optical flow direction histogram of each of the multiple regions, wherein the improved optical flow direction histogram changes the angle division area from 180 degrees to 360 degrees; Determining whether the region of the optical flow field needs to be re-divided according to the improved optical flow direction histogram feature value of each region, including: determining that the region of the optical flow field needs to be re-divided when the difference of the weight values ​​of the improved optical flow direction histogram feature values ​​of each region of each optical flow field in each direction is less than a preset threshold or no peak appears; If it is necessary to re-divide the area of ​​the optical flow field, then change the specific area division granularity and repeat the above steps of dividing the area, calculating the improved optical flow direction histogram eigenvalues ​​and determining whether to re-divide; and Calculating the fluid motion characteristics of each area of ​​the multiple areas, including: calculating the optical flow vector of each pixel in each area as the flow velocity vector; calculating the average length of the optical flow vectors greater than a predetermined threshold in each area as the flow velocity of the area; and calculating the number of optical flow vectors greater than a predetermined threshold in each area as the flow value in the area.

2. The method according to claim 1, wherein: The specific area division granularity is determined according to the pyramid model.

3. The method according to claim 1, wherein: Calculating the fluid motion characteristic of each of the plurality of regions further comprises: For the multiple optical flow fields, single-trace tracking is performed on the flow of each region according to the maximum weight of the improved optical flow direction histogram feature values ​​of each region in various directions, so as to obtain the fluid motion trajectory.

4. The method according to claim 1, wherein: Calculating the fluid motion characteristic of each of the plurality of regions further comprises: For the multiple optical flow fields, multi-trajectory tracking is performed on the flow of each region according to a plurality of predetermined higher weights in the weights of the improved optical flow direction histogram feature values ​​of each region in various directions, so as to obtain the fluid motion trajectory.

5. The method according to claim 3 or 4, wherein: In the process of single-track tracking or multi-track tracking, whether to end the tracking is determined according to the fluctuation or change of the weight of the feature value of the flow rate in a specific area on the tracking path or the improved optical flow direction histogram.

6. The method according to claim 4, wherein: Multi-trajectory tracking based on the maximum flow principle.

7. A fluid motion characteristic description device, comprising: a memory having instructions stored therein; as well as A processor configured to execute instructions stored on the memory to perform the method according to any one of claims 1 to 6.

8. A computer-readable storage medium comprising computer-executable instructions, which, when executed by one or more processors, cause the one or more processors to perform the method according to any one of claims 1 to 6.