A method and system for rotational speed measurement based on event camera and point cloud registration

Abstract

The application provides a rotating speed measurement method and system based on an event camera and point cloud registration, and the method comprises the following steps: acquiring rotating vane data; performing point cloud data preprocessing on the acquired rotating vane data; and obtaining a vane rotating speed result by using a point cloud registration method according to the preprocessed point cloud data, wherein the point cloud registration method obtains an optimal solution by iteratively calculating a rotation matrix of point cloud data. An event camera is used to shoot a rotating vane of a UAV, and aedat4 data format is recorded. The rotating speed measurement method based on the event camera is convenient to use, does not need to directly contact a measured object, and does not need to install a signal feedback device on the measured object. Meanwhile, the application also has strong anti-interference performance, does not need to keep a measuring instrument and a measured object relatively static, and can obtain the rotating speed of the object by using a user to hold the event camera to shoot the measured object.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of event camera data processing, and particularly relates to a rotating speed measurement method and system based on event camera and point cloud registration. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute the prior art.

[0003] In many industrial production fields, the rotating speed value of an object is an important index. When the quality of some products is detected, the rotating speed of mechanical movement can be measured to determine the current operation condition of equipment. Current rotating speed measurement methods mainly include the following: a contact type mechanical rotating speed meter, which allows a rigid body to move to drive a rotating speed meter probe to obtain a reading, but this method may cause personal safety problems for users in some cases, such as measuring a fan with fan blades, a high-speed rotating electric saw, and some large rotating equipment. In addition, some commonly used non-contact rotating speed measurement methods, such as an optical code disc rotating speed measurement method, a Hall element rotating speed measurement method, and a laser rotating speed measurement method, need to install specific signal feedback devices, such as optical code discs, magnets, and reflective stickers, on the measured object, and the measurement instrument and the measured object need to be kept relatively static during the measurement, and these methods are also easily affected by temperature changes and electromagnetic interference.

[0004] Currently, some video-based rotating speed measurement methods have strong anti-interference performance, but these traditional CMOS and CCD cameras have low frame rates, and need to accumulate a certain number of photons on the photosensitive element before imaging, so they cannot capture objects with high rotating speed. Although a high-speed camera can capture high-speed rotating objects, it has high cost and high power consumption, and cannot be widely used. Unlike frame-based cameras, an event camera has no concept of exposure and frame rate, each pixel point of the event camera independently performs operation, and an event is output only when the brightness change of a certain pixel point accumulates to a certain threshold value. Each event e=(timestamp, x, y, p) includes a time stamp (timestamp), a pixel coordinate (x, y), and an event polarity (p=0, 1, 1 represents brightness enhancement, and 0 represents brightness reduction). The event camera has low delay, high dynamic range, and extremely low power consumption, which enables it to play a role in monitoring, environmental perception, and high-speed object tracking in many special scenarios. SUMMARY

[0005] The present application proposes a rotational speed measurement method and system based on an event camera and point cloud registration to solve the above problems.

[0006] According to some embodiments, the present application adopts the following technical solutions:

[0007] A rotational speed measurement method based on an event camera and point cloud registration comprises:

[0008] Obtaining rotating blade data;

[0009] Performing point cloud data preprocessing on the obtained rotating blade data,

[0010] According to the preprocessed point cloud data, a point cloud registration method is used to obtain a blade rotational speed result,

[0011] The point cloud registration method obtains an optimal solution through iterative calculation of a rotation matrix of point cloud data.

[0012] Further, an event camera is used to shoot a rotating blade of a UAV, and aedat4 data format is recorded.

[0013] Further, the point cloud data preprocessing comprises converting the recorded data into a data form of e=(timestamp, x, y, p) as a candidate event.

[0014] Further, two event segments with the same length adjacent on a time axis are cut from the candidate event, and are respectively used as a source point cloud and a target point cloud.

[0015] Further, the point cloud registration method comprises using a point cloud registration method to calculate a rotation matrix between the source point cloud and the target point cloud.

[0016] Further, the calculation of the rotation matrix between the source point cloud and the target point cloud comprises finding a nearest point and calculating an optimal solution of the rotation matrix.

[0017] Further, the point cloud registration method further comprises converting the rotation matrix into Euler angles through iterative calculation and accumulating to obtain a final rotation angle, and obtaining a rotational speed result through a rotational speed formula.

[0018] A rotational speed measurement system based on an event camera and point cloud registration comprises:

[0019] A data acquisition module configured to obtain rotating blade data;

[0020] A preprocessing module configured to perform point cloud data preprocessing on the obtained rotating blade data,

[0021] The registration module is configured to obtain a fan speed result by using a point cloud registration method according to the preprocessed point cloud data,

[0022] The point cloud registration method obtains an optimal solution by iteratively calculating a rotation matrix of the point cloud data.

[0023] A computer readable storage medium, wherein a plurality of instructions are stored, the instructions are suitable for being loaded and executed by a processor of a terminal device, and the instructions are suitable for implementing the speed measurement method based on an event camera and point cloud registration.

[0024] A terminal device, comprising a processor and a computer readable storage medium, the processor is used to implement instructions, and the computer readable storage medium is used to store a plurality of instructions, the instructions are suitable for being loaded and executed by the processor, and the instructions are suitable for implementing the speed measurement method based on an event camera and point cloud registration.

[0025] Compared with the prior art, the beneficial effects of the present application are:

[0026] The speed measurement method based on the event camera is convenient to use, does not need to directly contact the measured object, and does not need to install a signal feedback device on the measured object. Meanwhile, the present application also has strong anti-interference performance, and the user can obtain the object speed by holding the event camera to shoot the measured object without keeping the measuring instrument and the measured object relatively static;

[0027] The data information of the high-speed rotating object is collected by using the event camera, the motion blur phenomenon existing in the traditional low-frame-rate camera is eliminated, the iterative closest point method is used, and the current rotating speed of the object can be quickly and accurately calculated. In addition, compared with the traditional speed measurement method, the present application is very convenient to use, does not need to install any signal feedback device on the measured object, and the user can obtain the reading by holding the event camera with relatively light mass to align the measured object, and there is no use burden. BRIEF DESCRIPTION OF DRAWINGS

[0028] The drawings accompanying the specification of the present application form a part of the present application, the illustrative embodiments of the present application and the description thereof are used to explain the present application, and do not constitute an improper limitation on the present application.

[0029] Figure 1 is a real object diagram of the event camera with a model number of DAVIS346 used in the present application;

[0030] Figure 2 is a comparison diagram of a rotating diagram of a wing of a quadrotor unmanned aerial vehicle captured by a traditional camera (a) and a rotating diagram of a wing of an unmanned aerial vehicle captured by an event camera (b);

[0031] Figure 3is the event stream with negative polarity (p=0, representing the wing) captured by an event camera in a single wing range for a period of time, Fig. (a) is the specific data form, Fig. (b) is the representation of three-dimensional space;

[0032] Figure 4 is an example diagram of two adjacent point cloud data collected by an event camera before (a) and after (b) point cloud registration;

[0033] Figure 5 is a rotation angle diagram calculated in the iteration process of point cloud registration;

[0034] Figure 6 is a flowchart of the object rotation speed measurement method using an event camera according to the present application. DETAILED DESCRIPTION

[0035] The present application will be further described below in conjunction with the accompanying drawings and examples.

[0036] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0037] It should be noted that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise, and it should be understood that when the terms "comprise" and / or "include" are used in the specification, there is a presence of the features, steps, operations, devices, components and / or combinations thereof.

[0038] Example 1.

[0039] As shown in Figure 1 A rotation speed measurement method based on an event camera and point cloud registration, comprising:

[0040] acquiring rotating fan data;

[0041] preprocessing the acquired rotating fan data,

[0042] According to the preprocessed point cloud data, the point cloud registration method is used to obtain the fan speed result,

[0043] wherein the point cloud registration method obtains the optimal solution by iteratively calculating the rotation matrix of the point cloud data.

[0044] An event camera is used to shoot the rotating fan of a drone, and aedat4 data format is recorded.

[0045] The point cloud data preprocessing includes converting the recorded data into the data form of e=(timestamp, x, y, p) as a candidate event.

[0046] Two event fragments with the same length adjacent on the time axis are intercepted from the candidate event as a source point cloud and a target point cloud, respectively.

[0047] The point cloud registration method includes calculating the rotation matrix between the source point cloud and the target point cloud by using the point cloud registration method.

[0048] The calculation of the rotation matrix between the source point cloud and the target point cloud includes searching for the nearest point and calculating the optimal solution of the rotation matrix.

[0049] The point cloud registration method further includes converting the rotation matrix into Euler angles and accumulating to obtain the final rotation angle by iterative calculation, and obtaining the rotation speed result by the rotation speed formula.

[0050] Specifically,

[0051] The event camera is used to record the event stream data of the high-speed rotating object, Figure 1 An actual application scenario of the present application is shown, and a user uses a general camera and an event camera to shoot the rotating wing of a drone, respectively. It can be seen that due to the high speed of the wing, the image (a) shot by the traditional camera has a serious motion blur phenomenon, and the event camera can capture the high-speed motion of the wing. The light intensity of the light gray part in the figure (b) is enhanced, and the light intensity of the dark gray part is reduced, which is caused by the wing shielding the background.

[0052] The results of the test are as shown in the steps of the present application Figure 6 The main steps are as follows:

[0053] Step 1: Rotating fan data acquisition

[0054] The event camera of the model DAVIS346 as shown in the figure is used to shoot the fan of the drone, the aedat4 data format is recorded, and a decoding program is used to convert it into the data form of e=(timestamp, x, y, p). The data example is as shown in the figure. Figure 1 Figure 2 The figure selects all events with negative polarity (p=-1) in a 50ms time period (i.e. events caused by the fan shielding the background), and the coordinate axis t represents the time information of the event point, and x and y represent the position information of the event point.

[0055] Step 2: Point cloud data preprocessing

[0056] Two event fragments with the same length adjacent on the time axis are intercepted from the candidate event as a source point cloud (source point cloud, P​s ) and target point cloud (P t ). In order to provide a better initial value for subsequent point cloud registration, their time axis index is converted to 0 start timing. An example is shown in Fig. Figure 3 (a), the time length of the selected source point cloud and target point cloud is 5ms (time length = 5ms), and the light gray event points in the figure represent source point cloud data, and the black event points represent target point cloud data.

[0057] Step 3: Point cloud registration

[0058] The present application uses a point cloud registration method to calculate the rotation matrix R between the source point cloud and the target point cloud. Taking the most widely used Iterative Closest Point (ICP) algorithm as an example, the calculation method of the rotation matrix R has the following steps:

[0059] Step 3.1: Find the nearest point

[0060] For each point in the source point cloud P s , find the nearest point in the target point cloud Pt, where the Euclidean distance in three-dimensional space is represented as The present application uses KD-tree algorithm to speed up the search, and the calculation complexity is reduced from O(mn) to O(mlog(n)).

[0061] Step 3.2: Calculate the optimal solution of the rotation matrix R

[0062] First, calculate the center of mass of the source point cloud and the target point cloud Subtract the center of mass coordinates from each point in P t and P s to get Then calculate the three-dimensional matrix H, SVD decompose H = U∑V T to get V and U, and then calculate the optimal rotation matrix R * = VU T and the optimal translation matrix,

[0063] Step 3.3: Iteration

[0064] In each iteration process, first, perform step 3.1 nearest point search and step 3.2 to calculate the optimal solution of the rotation matrix to get the optimal parameter transformation R * , t * of the current loop. Then, the current source point cloud can be calculated as Finally, the current source point cloud is brought into the next iteration calculation process and is executed in a loop until the final iteration termination condition is met (the maximum number of iterations is reached, or the average distance of the corresponding points of the source point cloud and the target point cloud is less than a specific threshold). An example of the final point cloud matching is shown in FIG. 8. Figure 3 (b) As shown, the current source point cloud has a high degree of coincidence with the target point cloud.

[0065] Step 3.4: Rotational speed calculation

[0066] In each iteration process, the rotation matrix R * of the current iteration is recorded x , θ y , θ z The Euler angles obtained in each iteration are continuously accumulated to obtain sum_θ x , sum_θ y , sum_θ z These values are the cumulative transformation angles of the initial source point cloud P s transformed to the current source point cloud . According to the calculation results, sum_θ x , sum_θ y tend to 0, which is due to the rotation occurring in the xy plane. While sum_θ z tends to a stable value in multiple iterations, an example of a single registration is shown in FIG. 9. Figure 4 Therefore, the final sum_θ z is the rotation angle when the point cloud registration is completed. The formula for calculating the rotational speed value per minute is 60*sum_θ z / time length (rpm).

[0067] Example 2.

[0068] A rotational speed measurement system based on an event camera and point cloud registration, comprising:

[0069] A data acquisition module configured to acquire rotating blade data;

[0070] A preprocessing module configured to perform point cloud data preprocessing on the acquired rotating blade data,

[0071] A registration module configured to obtain a blade rotational speed result using a point cloud registration method based on the preprocessed point cloud data,

[0072] wherein the point cloud registration method obtains an optimal solution by iteratively calculating the rotation matrix of the point cloud data.

[0073] Example 3.

[0074] A computer readable storage medium, wherein a plurality of instructions are stored, the instructions being suitable for being loaded by a processor of a terminal device and performing a rotational speed measurement method based on an event camera and point cloud registration provided by the embodiments.

[0075] Embodiment 4.

[0076] A terminal device, comprising a processor and a computer readable storage medium, the processor being configured to implement instructions; and the computer readable storage medium being configured to store a plurality of instructions, the instructions being suitable for being loaded by the processor and performing a rotational speed measurement method based on an event camera and point cloud registration provided by the embodiments.

[0077] Those skilled in the art will understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer usable program code.

[0078] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions that are executed by the processor of the computer or other programmable data processing apparatus generate an apparatus that implements the flow Figure 1 The function specified in one or more flows and / or blocks. Figure 1 The function specified in one or more flows and / or blocks.

[0079] These computer program instructions can also be stored in a computer readable storage medium that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer readable storage medium produce a manufactured product including instruction apparatus, which implements the flow Figure 1 The function specified in one or more flows and / or blocks. Figure 1 The function specified in one or more flows and / or blocks.

[0080] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable data processing apparatus to produce a computer implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide a process for implementing the flow Figure 1one or more processes and / or blocks Figure 1 steps of the functions specified in the one or more blocks.

[0081] The above only is the preferred embodiment of the present application, and is not used to limit the present application, for the person skilled in the art, the present application can have various changes and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0082] Although the specific embodiments of the present application are described above in combination with the drawings, it is not a limitation on the protection scope of the present application, and the person skilled in the art should understand that various modifications or changes made on the basis of the technical scheme of the present application without creative labor are still within the protection scope of the present application.

Claims

1. A method for rotational speed measurement based on event camera and point cloud registration, characterized in that, The method comprises the following steps: acquiring rotating fan blade data; performing point cloud data preprocessing on the acquired rotating fan blade data; cutting two adjacent event segments with the same length on the time axis from the candidate events as a source point cloud and a target point cloud, respectively; obtaining a fan rotating speed result by using a point cloud registration method according to the preprocessed point cloud data; the point cloud registration method comprises the following steps: calculating a rotation matrix between the source point cloud and the target point cloud by using a point cloud registration method; the calculation of the rotation matrix between the source point cloud and the target point cloud comprises the following steps: finding the nearest point, and calculating the optimal solution of the rotation matrix; 2. A method of rotational speed measurement based on event camera and point cloud registration according to claim 1, characterized in that, the point cloud registration method further comprises the following steps:

3. A method of rotational speed measurement based on event camera and point cloud registration according to claim 2, characterized in that, The point cloud data preprocessing includes converting the recorded data into a data form of e=( as a candidate event.

4. A rotational speed measurement system based on an event camera and point cloud registration, characterized in that by iterative calculation, converting the rotation matrix into Euler angles and accumulating to obtain a final rotation angle, and obtaining a rotating speed result by using a rotating speed formula. The point cloud registration method obtains the rotation matrix by iterative calculation of the point cloud data to obtain the optimal solution. An event camera is used to shoot a rotating fan blade of a drone, and aedat4 data format is recorded. The method comprises the following steps: a data acquisition module is configured to acquire rotating fan blade data; a preprocessing module is configured to perform point cloud data preprocessing on the acquired rotating fan blade data, and cut two adjacent event segments with the same length on the time axis from the candidate events as a source point cloud and a target point cloud, respectively; a registration module is configured to obtain a fan rotating speed result by using a point cloud registration method according to the preprocessed point cloud data, and the point cloud registration method comprises the following steps:

5. A computer readable storage medium, characterized in that, calculating a rotation matrix between the source point cloud and the target point cloud by using a point cloud registration method; 6. A terminal device, characterized by comprising: the calculation of the rotation matrix between the source point cloud and the target point cloud comprises the following steps: finding the nearest point, and calculating the optimal solution of the rotation matrix; the point cloud registration method further comprises the following steps: by iterative calculation, converting the rotation matrix into Euler angles and accumulating to obtain a final rotation angle, and obtaining a rotating speed result by using a rotating speed formula. The point cloud registration method obtains the rotation matrix by iterative calculation of the point cloud data to obtain the optimal solution. A terminal device has a plurality of instructions stored therein, and the instructions are adapted to be loaded and executed by a processor of the terminal device to implement the rotating speed measurement method based on the event camera and the point cloud registration. The terminal device comprises a processor and a computer readable storage medium, the processor is used to implement instructions, and the computer readable storage medium is used to store a plurality of instructions, the instructions are adapted to be loaded and executed by the processor to implement the rotating speed measurement method based on the event camera and the point cloud registration.

Images