A large scene multi-view dynamic point cloud space-time synchronization method and system

Abstract

The application discloses a large-scene multi-view dynamic point cloud space-time synchronization method and system, and the method comprises the following steps: acquiring point cloud sequences at different scanning positions; wherein the point cloud sequences comprise source point clouds and target point clouds; fusing the space and time domain features extracted from the point cloud sequences and projecting the features into a feature map of a plane grid; segmenting the feature map in the grid to obtain static and dynamic part point clouds; performing integration operation on the static part point clouds, and performing space registration on the source point clouds and the target point clouds after the integration operation to obtain a space alignment result; calculating the distance between the target point clouds and a reference frame based on the source point clouds based on the dynamic part point clouds and the space alignment result, and selecting the time point with the shortest distance as the relative time difference between the source point clouds and the target point clouds. The application can effectively utilize the space-time information of the point clouds, obtain accurate synchronization results without relying on external time synchronization equipment, and has good robustness in large scenes and sparse point clouds.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of 3D vision, in particular to a large scene multi-view dynamic point cloud space-time synchronization method and system. BACKGROUND

[0002] The perception and understanding of real three-dimensional scenes has been a hot topic in the field of 3D vision in recent years, and how to use 3D imaging systems (such as laser radars) to effectively model the scene is the basis of these researches. In many practical applications, three-dimensional scenes are often accompanied by dynamic changes in the time dimension, such as traffic data at crossroads, human data in sports competitions, etc. These time-series point cloud data combined with accurate three-dimensional positioning and tracking algorithms and human pose detection algorithms can play a huge value in actual traffic management and competition analysis. With the increasing demand for laser radars, many portable and low-cost laser radar products have appeared on the market, which can record spatial point cloud data within a certain spatial range and a period of time, providing a hardware basis for building large-scale dynamic three-dimensional scenes.

[0003] When using a single laser radar to scan a scene, the view angle is limited and the point cloud is sparse. In order to make the scene modeling more complete and detailed, multiple laser radars need to be used for joint scanning. However, the data scanned by multiple laser radars need to be accurately spliced in space; on the other hand, due to the need to manually trigger different laser radars during data acquisition, the point cloud data sequences from different sources have a fixed interval in the time dimension, which need to be manually synchronized and corrected. Therefore, the space-time synchronization of point cloud data is a necessary step to ensure the completeness and accuracy of the scene.

[0004] At present, there are many algorithms for multi-point cloud spatial registration, such as ICP and feature descriptor-based methods. These methods can achieve good results in solving the registration problem of static scenes or 3D models, but in the registration of large-scale dynamic scenes, the following challenges exist:

[0005] Without synchronizing the point cloud time, the dynamic objects in the scene will bring many mismatches to the search for corresponding points in the spatial registration algorithm due to their different spatial positions, affecting the registration effect.

[0006] The single-frame scene point cloud density is too sparse, which is not conducive to spatial registration. Although the time-integrated method can increase the density of points, it will cause ghosting of dynamic objects in the scene. SUMMARY

[0007] The present application aims to at least solve one of the technical problems in the related art to some extent.

[0008] Based on the fact that when a dynamic large scene is scanned by using a laser radar, due to the angle of view and distance limitation of a single laser radar and the sparsity of a single frame of point cloud, it is often necessary to use multiple laser radars to scan different parts of the scene. In this regard, the present application proposes a large scene multi-view dynamic point cloud space-time synchronization method, which inputs point cloud sequences at different scanning positions of the same scene and outputs the relative poses (spatial synchronization) of the two scanning positions and the relative time difference (time synchronization) of the point cloud sequences. The method proposed in the present application has three stages, the first stage is to divide the dynamic and static points of each frame of point cloud from different sources, the second stage is to extract local feature descriptors of the static part of the point cloud and complete spatial matching, and the third stage is to calculate the distance between the dynamic part of a point cloud sequence and a reference frame in another point cloud sequence under the premise of spatial alignment, and select the minimum value as the relative time difference of the two point cloud sequences. By dividing the dynamic and static objects in the scene, the local scene point clouds from different laser radars can be aligned in time and space.

[0009] Another object of the present application is to propose a large scene multi-view dynamic point cloud space-time synchronization system.

[0010] To achieve the above object, the present application proposes a large scene multi-view dynamic point cloud space-time synchronization method, which comprises:

[0011] Obtaining point cloud sequences at different scanning positions; wherein the point cloud sequences include source point clouds and target point clouds;

[0012] Fusing the spatial and time domain features extracted from the point cloud sequences and projecting them into a feature map of a plane grid, and dividing the feature map in the grid to obtain static and dynamic part point clouds;

[0013] Integrating the static part of the point cloud, and spatially registering the source point cloud and the target point cloud after the integration operation to obtain a spatial alignment result;

[0014] Based on the dynamic part of the point cloud and the spatial alignment result, the distance between the target point cloud and the reference frame based on the source point cloud is calculated, and the time with the shortest distance is selected as the relative time difference between the source point cloud and the target point cloud.

[0015] In addition, the large scene multi-view dynamic point cloud space-time synchronization method according to the above embodiments of the present application can also have the following additional technical features:

[0016] Further, in an embodiment of the present application, the fusion of the spatial and temporal features extracted from the point cloud sequence and the projection into the feature map of the planar grid are performed, and the static and dynamic part point clouds are obtained by segmenting the feature map in the grid, including: frame preprocessing is performed on each frame of point cloud of the point cloud sequence; spatial feature extraction is performed on each frame of point cloud after frame preprocessing; feature fusion is performed on adjacent frames based on the extracted spatial features; a point cloud state classification result is obtained by performing a back projection operation based on the voxel label obtained by segmenting and fusing the features, and a static and dynamic part point cloud is obtained based on the point cloud state classification result.

[0017] Further, in an embodiment of the present application, the integral operation is performed on the static part point cloud, and the spatial registration is performed on the source point cloud and the target point cloud after the integral operation to obtain the spatial alignment result, including: the integral operation is performed on the static part point cloud based on the source point cloud and the target point cloud respectively for a preset time; the distance threshold of the static part point cloud after the integral operation is obtained, and the spatial structure feature is obtained by performing local feature extraction based on the distance threshold; the spatial transfer matrix is calculated based on the matching point pair of the target point cloud obtained based on the spatial structure feature, and the spatial alignment of the source point cloud and the target point cloud is performed based on the spatial transfer matrix to obtain the spatial alignment result.

[0018] Further, in an embodiment of the present application, the distance between the target point cloud and the reference frame based on the source point cloud is calculated based on the dynamic part point cloud and the spatial alignment result, including: the dynamic part point cloud of the source point cloud is transformed based on the spatial transfer matrix to obtain position information in the same spatial coordinate system as the target point cloud; the dynamic part point cloud traversal result of the target point cloud is obtained based on the position information, and the distance between the reference frame of the source point cloud and the dynamic part point cloud of the target point cloud is calculated based on the dynamic part point cloud traversal result and a preset distance calculation formula.

[0019] Further, in an embodiment of the present application, the preset distance calculation formula includes:

[0020] The expression of the chamfer distance is:

[0021]

[0022] The expression of the feature-based structural similarity is:

[0023]

[0024] Where SSIM is the structural similarity index, and are the global features of the point cloud P src , P dst .

[0025] The frame with the minimum distance to the reference frame in the target point cloud is taken as a matching frame in time, and a relative time difference between the source point cloud and the target point cloud is obtained based on the matching frame, and the expression of the matching frame is:

[0026]

[0027] To achieve the above object, another aspect of the present application provides a large-scene multi-view dynamic point cloud space-time synchronization system, comprising:

[0028] A data acquisition module is configured to acquire a point cloud sequence at different scanning positions; wherein the point cloud sequence comprises a source point cloud and a target point cloud;

[0029] A point cloud segmentation module is configured to fuse spatial and time domain features extracted from the point cloud sequence and project the features into a feature map of a plane grid, segment the feature map in the grid to obtain static and dynamic part point clouds;

[0030] A space registration module is configured to perform an integration operation on the static part point cloud, and perform space registration on the source point cloud and the target point cloud after the integration operation to obtain a space alignment result;

[0031] A time alignment module is configured to calculate distances between the target point cloud and a reference frame based on the source point cloud based on the dynamic part point cloud and the space alignment result, and select a time with the shortest distance as a relative time difference between the source point cloud and the target point cloud.

[0032] The large-scene multi-view dynamic point cloud space-time synchronization method and system can simultaneously solve the space-time synchronization problem of multi-view point clouds and overcome the challenge of point cloud sparseness caused by large scenes.

[0033] Additional aspects and advantages of the present application will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0034] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description, taken in conjunction with the accompanying drawings, in which:

[0035] Figure 1 is a flowchart of a large-scene multi-view dynamic point cloud space-time synchronization method according to an embodiment of the present application;

[0036] Figure 2 is a framework diagram of a large-scene multi-view dynamic point cloud space-time synchronization method according to an embodiment of the present application;

[0037] Figure 3 is a point cloud dynamic / static point segmentation schematic diagram according to an embodiment of the present application;

[0038] Figure 4 is a static point cloud based spatial registration diagram according to an embodiment of the present application;

[0039] Figure 5 is a dynamic point cloud distance based time alignment diagram according to an embodiment of the present application;

[0040] Figure 6 is a large scene multi-view dynamic point cloud space-time synchronization system structure diagram according to an embodiment of the present application. DETAILED DESCRIPTION

[0041] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0042] In order for those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings of the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0043] The large scene multi-view dynamic point cloud space-time synchronization method and system according to the embodiments of the present application will be described below with reference to the accompanying drawings.

[0044] Figure 1 is a flowchart of the large scene multi-view dynamic point cloud space-time synchronization method according to an embodiment of the present application.

[0045] As shown in Figure 1 , the method includes but is not limited to the following steps:

[0046] S1, acquiring point cloud sequences at different scanning positions; wherein the point cloud sequences include source point clouds and target point clouds;

[0047] S2, fusing the spatial and time domain features extracted from the point cloud sequences and projecting them into a feature map of a planar grid, segmenting the feature map in the grid to obtain static and dynamic part point clouds;

[0048] S3, performing integration operation on the static part point clouds, and performing spatial registration on the source point clouds and the target point clouds after the integration operation to obtain a spatial alignment result;

[0049] S4, calculating the distance between the target point clouds and the reference frame based on the source point clouds based on the dynamic part point clouds and the spatial alignment result, and selecting the time with the shortest distance as the relative time difference between the source point clouds and the target point clouds.

[0050] The large-scene multi-view dynamic point cloud space-time synchronization method provided by the embodiment of the application can effectively utilize the space-time information of the point cloud, obtain accurate synchronization results without relying on external time synchronization equipment, and has good robustness in large scenes and sparse point clouds.

[0051] The large-scene multi-view dynamic point cloud space-time synchronization method of the embodiment of the application will be described in detail below with reference to the accompanying drawings.

[0052] First, the symbols appearing below are described: t: the current time of algorithm processing, point cloud frames collected by the two laser radars at time t, static part in the point cloud frames collected by the two laser radars at time t, dynamic part in the point cloud frames collected by the two laser radars at time t, T: space transfer matrix of the two laser radars in the world coordinate system, Δ t : time difference of starting of the two laser radars.

[0053] Specifically, as shown in the figure, Figure 2 the large-scene multi-view dynamic point cloud space-time synchronization method provided by the application is divided into three stages, namely point cloud dynamic / static point segmentation, space registration based on static point cloud, and time alignment based on dynamic point cloud distance. In the point cloud static / dynamic point segmentation stage, the spatial and time domain features of each input frame of point cloud are extracted, fused, and projected into the feature map of the ground grid, and then the feature map is classified by using a segmentation network; in the space registration based on static point cloud stage, after classification, the static part of the point cloud is integrated for a certain time to obtain relatively dense source point cloud and target point cloud, and space registration is completed by using the space descriptor matching method; in the time alignment based on dynamic point cloud distance stage, for the dynamic part of the point cloud, a frame is selected as a reference frame in the source point cloud, and then the distance between the target point cloud sequence and the reference frame is calculated, and the time difference between the two point clouds is taken as the time difference between the two point clouds.

[0054] The first stage is point cloud dynamic / static point segmentation. The point cloud dynamic / static point segmentation provided by the application includes four parts, namely frame preprocessing, frame feature extraction, time sequence feature fusion, and point-by-point dynamic / static classification, as shown in the figure. Figure 3

[0055] In the frame preprocessing stage, it is assumed that the ground in each frame of point cloud is an ideal plane, so the plane is first fitted, and the points within a certain distance threshold range above and below the ground are removed, and then the point cloud is cropped, mainly to retain the points in the interested part (overlapping area) and remove the points far away, so as to facilitate the processing in the next step;

[0056] ​In the frame feature extraction stage, spatial features need to be extracted from each frame of point cloud from the source point cloud sequence and the target point cloud sequence. For example, the point cloud can be voxelized in the BEV perspective, and features are extracted for the points of each voxel, so that a two-dimensional feature map is obtained;

[0057] In the temporal feature fusion stage, features from adjacent frames need to be fused to obtain temporal information, which is beneficial to the judgment of dynamic points;

[0058] In the point-by-point dynamic and static classification module, a two-dimensional segmentation network can be used to segment the fused features to obtain a voxel-by-voxel label, and the label is back-projected to each point inside to obtain the dynamic / static classification result of each point in the point cloud. According to the classification result, each frame of point cloud can be divided into static and dynamic parts.

[0059] In the actual scene, a period of point cloud data needs to be collected first, and dynamic points are manually labeled as ground truth for training of various network parameters.

[0060] The second stage is spatial registration based on static point clouds, as shown in Figure 4 The static part of the point cloud in the source point cloud sequence is integrated for a certain period of time to obtain a relatively dense scene part of the point cloud, and the same operation is performed on the target point cloud sequence.

[0061] After integration, for each point in the two static point clouds, a certain distance threshold is set, and structural features are extracted for the points within the threshold range. Traditional manual descriptors or network-based learning methods can be used for local feature extraction; then the nearest neighbor points in the target point cloud are searched in the feature space. According to the matched point pairs searched, a rigid transformation matrix T is calculated. Through the transformation matrix, the source point cloud and the target point cloud can be aligned in space.

[0062] The third stage is time alignment based on dynamic point cloud distance, as shown in Figure 5 According to the transfer matrix calculated in the second stage based on the spatial registration of static point clouds, the dynamic point cloud part of the source point cloud sequence is first transformed so that it is in the same spatial coordinate system as the target point cloud. Then, a dynamic reference frame is selected in the source point cloud sequence, and the distance dist(P src , P dst ) between each frame of the target dynamic point cloud is calculated. The distance index should gradually decrease as the two point clouds approach in real time. Two feasible distance calculation methods are listed in the present application:

[0063] ① Chamfer distance, the calculation formula is:

[0064]

[0065] ② Feature-based structural similarity:

[0066]

[0067] where SSIM is the structural similarity index, and is the global feature of the point cloud P src , P dst is the global feature of the point cloud P

[0068] In a certain time range, the frame with the minimum distance to the reference frame in the target point cloud is searched as the matching frame in time, which can be expressed as:

[0069]

[0070] In actual calculation, multiple reference frames can be selected for matching respectively, and the average value of is calculated as the time difference of the two point cloud sequences. t

[0071] According to the large-scene multi-view dynamic point cloud space-time synchronization method, the space-time information of the point cloud can be effectively utilized to obtain accurate synchronization results without relying on external time synchronization equipment, and the method has good robustness in large scenes and sparse point clouds. The space-time synchronization problem of multi-view point clouds can be solved at the same time, and the challenge of point cloud sparseness brought by large scenes can be overcome.

[0072] In order to realize the above-mentioned embodiments, as shown in Figure 6 The embodiment also provides a large-scene multi-view dynamic point cloud space-time synchronization system 10, which comprises a data acquisition module 100, a point cloud segmentation module 200, a spatial registration module 300 and a time alignment module 400.

[0073] The data acquisition module 100 is used for acquiring point cloud sequences at different scanning positions; wherein the point cloud sequences comprise source point clouds and target point clouds;

[0074] The point cloud segmentation module 200 is used for fusing the spatial and time domain features extracted from the point cloud sequences and projecting the features into a feature map of a plane grid, and segmenting the feature map in the grid to obtain static and dynamic part point clouds;

[0075] The spatial registration module 300 is used for performing integration operation on the static part point clouds, and performing spatial registration on the source point clouds and the target point clouds after the integration operation to obtain a spatial alignment result;

[0076] The time alignment module 400 is used for calculating the distances between the target point clouds and the reference frames based on the source point clouds based on the dynamic part point clouds and the spatial alignment result, and selecting the time with the shortest distance as the relative time difference between the source point clouds and the target point clouds.​

[0077] Further, the point cloud segmentation module 200 is further configured to:

[0078] perform frame preprocessing on each frame of point cloud of the point cloud sequence;

[0079] perform spatial feature extraction on each frame of point cloud after frame preprocessing;

[0080] perform feature fusion of adjacent frames based on the extracted spatial features;

[0081] perform back projection operation based on the voxel-by-voxel label obtained from the fused features to obtain point cloud state classification results, and obtain static and dynamic part point clouds based on the point cloud state classification results.

[0082] Further, the spatial registration module 300 is further configured to:

[0083] perform integral operation on the static part point clouds based on the source point cloud and the target point cloud respectively for a preset time;

[0084] obtain a distance threshold of the static part point clouds after the integral operation, and perform local feature extraction based on the distance threshold to obtain spatial structure features;

[0085] perform spatial transformation matrix calculation based on the matching point pairs of the target point cloud obtained based on the spatial structure features, and perform spatial alignment of the source point cloud and the target point cloud based on the spatial transformation matrix to obtain a spatial alignment result.

[0086] Further, the time alignment module 300 is further configured to:

[0087] transform the dynamic part point clouds of the source point cloud based on the spatial transformation matrix to obtain position information in the same spatial coordinate system as the target point cloud;

[0088] obtain a dynamic part point cloud traversal result of the target point cloud based on the position information, and calculate the distance between the reference frame of the source point cloud and the dynamic part point clouds of the target point cloud based on a preset distance calculation formula.

[0089] Further, the preset distance calculation formula includes:

[0090] The expression of the chamfer distance is:

[0091]

[0092] The expression of the feature-based structural similarity is:

[0093]

[0094] where SSIM is a structural similarity index, and For the point cloud P src , P dst Global features of the point cloud P

[0095] The frame with the minimum distance to the reference frame in the target point cloud is taken as the matching frame in time, and the relative time difference between the source point cloud and the target point cloud is obtained based on the matching frame, and the expression of the matching frame is:

[0096]

[0097] The large-scene multi-view dynamic point cloud space-time synchronization system according to the embodiments of the present application can effectively utilize the space-time information of the point cloud, obtain accurate synchronization results without relying on external time synchronization devices, and has good robustness in large scenes and sparse point clouds. The space-time synchronization problem of multi-view point cloud can be solved at the same time, and the challenge of point cloud sparseness brought by large scenes can be overcome.

[0098] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples without contradiction..

[0099] In addition, the terms "first", "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality of" is at least two, for example, two, three, etc., unless otherwise specifically limited.

Claims

1. A multi-view dynamic point cloud spatio-temporal synchronization method, characterized in that, The method comprises the following steps: obtaining a point cloud sequence at different scanning positions; wherein the point cloud sequence comprises source point clouds and target point clouds; fusing spatial and time domain features extracted from the point cloud sequence and projecting the features into a feature map of a planar grid to segment the feature map in the grid to obtain static and dynamic part point clouds; performing integral operation on the static part point clouds based on the source point clouds and the target point clouds respectively for a preset time; obtaining a distance threshold of the static part point clouds after the integral operation, and extracting local features based on the distance threshold to obtain spatial structure features; performing spatial transfer matrix calculation based on matching point pairs of the target point clouds obtained based on the spatial structure features, and performing spatial alignment of the source point clouds and the target point clouds based on the spatial transfer matrix to obtain a spatial alignment result; transforming the dynamic part point clouds of the source point clouds based on the spatial transfer matrix to obtain position information in the same spatial coordinate system as the target point clouds; obtaining a traversal result of the dynamic part point clouds of the target point clouds based on the position information, calculating the distance between the reference frame of the source point clouds and the dynamic part point clouds of the target point clouds based on a preset distance calculation formula, and selecting a time point with the shortest distance as the relative time difference between the source point clouds and the target point clouds.

2. The method of claim 1, wherein, The step of fusing the spatial and time domain features extracted from the point cloud sequence and projecting the features into a feature map of a planar grid to segment the feature map in the grid to obtain static and dynamic part point clouds comprises: performing frame preprocessing on each frame of point clouds of the point cloud sequence; extracting spatial features from each frame of point clouds after the frame preprocessing; performing feature fusion of adjacent frames based on the extracted spatial features; performing back projection operation based on the voxel-by-voxel label obtained based on the fused features to obtain a point cloud state classification result, and obtaining the static and dynamic part point clouds based on the point cloud state classification result.

3. The method of claim 1, wherein, The preset distance calculation formula comprises: the expression of the chamfer distance is: the expression of the feature-based structural similarity is: wherein SSIM is the structural similarity index, and is a global feature of the point cloud P src , P dst . the expression of the matching frame with the minimum distance to the reference frame in the target point clouds is:

4. A multi-view dynamic point cloud spatio-temporal synchronization system, characterized in that, comprises: a data acquisition module configured to obtain a point cloud sequence at different scanning positions; wherein the point cloud sequence comprises source point clouds and target point clouds; a point cloud segmentation module configured to fuse spatial and time domain features extracted from the point cloud sequence and project the features into a feature map of a planar grid to segment the feature map in the grid to obtain static and dynamic part point clouds; a spatial registration module configured to perform integral operation on the static part point clouds based on the source point clouds and the target point clouds respectively for a preset time, obtain a distance threshold of the static part point clouds after the integral operation, extract local features based on the distance threshold to obtain spatial structure features, perform spatial transfer matrix calculation based on matching point pairs of the target point clouds obtained based on the spatial structure features, and perform spatial alignment of the source point clouds and the target point clouds based on the spatial transfer matrix to obtain a spatial alignment result. The time alignment module is configured to transform the dynamic part of the source point cloud according to the spatial transfer matrix to obtain position information of the dynamic part of the source point cloud in the same spatial coordinate system as the target point cloud; obtain a traversal result of the dynamic part of the target point cloud based on the position information, calculate a distance between a reference frame of the source point cloud and the dynamic part of the target point cloud according to a preset distance calculation formula based on the traversal result of the dynamic part of the target point cloud, and select a time point with the shortest distance as a relative time difference between the source point cloud and the target point cloud.

5. The system of claim 4, wherein, The point cloud segmentation module is further configured to: perform frame preprocessing on each frame of point cloud of the point cloud sequence; perform spatial feature extraction on each frame of point cloud after frame preprocessing; perform feature fusion of adjacent frames based on the extracted spatial features; perform back projection operation on a voxel-by-voxel label obtained based on the fused features to obtain a point cloud state classification result, and obtain static and dynamic part point clouds based on the point cloud state classification result.

6. The system of claim 4, wherein, The preset distance calculation formula includes: an expression of a chamfer distance is: an expression of a feature-based structural similarity is: wherein SSIM is the structural similarity index, and is a global feature of the point cloud P src , P dst . a frame with the minimum distance to the reference frame in the target point cloud is taken as a matching frame in time, a relative time difference between the source point cloud and the target point cloud is obtained based on the matching frame, and an expression of the matching frame is:

Images