A method and device for event camera motion compensation based on motion model

Abstract

The present invention discloses a method and device for motion compensation of an event camera based on a motion model. The method sequentially reads the event sequence output by the event camera according to a preset time window, converts the event sequence in the time window into an average time plane, obtains the centroid of the average time plane, performs a convolution operation on the average time plane using a Sobel operator, calculates the horizontal and vertical gradients of the average time plane, updates the parameters of the motion model using a gradient descent method, and updates the event position using the parameters of the updated motion model. The present invention achieves the effect of motion compensation by updating the event position through establishing a motion model, and can effectively compensate for the motion blur of a high-speed moving target caused by event accumulation.

Description

Technical Field

[0001] The present application belongs to the technical field of target detection and tracking, and in particular, to a method and device for event camera motion compensation based on a motion model. Background Art

[0002] When ordinary cameras capture images, they show the scene within the exposure time. When an object in the scene moves, the image of the scene will show the combination of all the positions of the object during the exposure time. Due to the limitation of the frame rate of ordinary cameras, objects moving at high speed will appear blurred, which is called motion blur. Motion blur is the movement effect in the scene image, which is more obvious in the case of long exposure or high-speed movement of the target in the scene. In the process of target detection, motion blur will confuse the exact position of the target.

[0003] Traditional frame-based cameras have limitations such as information redundancy, high-speed blur, and low dynamic range. Therefore, in order to obtain higher sensitivity, faster sensors and lower latency processing time are required. Event cameras are a promising solution that is expected to overcome some problems based on traditional camera frameworks. It senses and encodes the world by imitating the retina and uses it as a neural signal to obtain visual information. Event cameras have no concept of exposure and frame rate, no idle time, and each pixel can perform independent calculations, and read data only when the light intensity changes at this pixel. The event camera outputs a series of asynchronous event streams based on the change in light intensity, where each event contains pixel coordinates, time, and polarity information (+1 indicates increased brightness, -1 indicates decreased brightness).

[0004] Although event cameras do not have the concept of frame rate, they output continuous events in time and space. However, in the process of applying them to traditional technologies, it is necessary to first accumulate a period of time or a certain number of events into a time plane similar to an image frame. Therefore, in the case of fast scene changes, due to the compression of too many events, the generated time plane will be somewhat blurred, affecting the characteristics of the target and the judgment of the target position, and further affecting target tracking, obstacle avoidance, etc. Specifically, the target edge in the time plane has a tail, which causes certain troubles for subsequent feature extraction and position detection. Summary of the invention

[0005] The purpose of this application is to provide an event camera motion compensation method and device based on a motion model, which performs motion compensation on the motion blur of high-speed moving targets caused by event accumulation based on the motion model, assists the target detection algorithm to accurately locate the target position, and thus improves the accuracy of target tracking and obstacle avoidance.

[0006] In order to achieve the above purpose, the technical solution of this application is as follows:

[0007] An event camera motion compensation method based on a motion model is used to compensate for motion blur of a high-speed moving target caused by event accumulation, comprising:

[0008] Step 1: Initialize the motion model parameters and learning rate, and read the event sequence output by the event camera according to the preset time window;

[0009] Step 2: Convert the event sequence in the time window into an average time surface and obtain the centroid of the average time surface;

[0010] Step 3: Use the Sobel operator to perform a convolution operation on the average time plane, calculate the horizontal and vertical gradients of the average time plane, and use the gradient descent method to update the parameters of the motion model;

[0011] Step 4: Update the event position using the updated motion model parameters;

[0012] Step 5: Determine whether the iteration termination condition is met. If so, end the motion compensation of the current time window. Otherwise, return to step 2 for the next iteration.

[0013] Furthermore, the updating of the event position by using the updated motion model parameters includes:

[0014] The event position is updated using the following formula:

[0015]

[0016] Where (x, y) is the event position before the update, (x′, y′) is the event position after the update, (mx, my) is the centroid of the average time surface, t0 is the initial time in the time window, t is the time of the current event, and h x ,h y ,h z , θ is the motion model (h x ,h y ,h z , θ) parameters.

[0017] Furthermore, the updating of the parameters of the motion model by using the gradient descent method includes:

[0018] Step 3.1, calculate the gradient of the motion model parameter variable;

[0019] Define the loss function Thus, the motion model (h x ,h y ,h z ,θ) are the gradients of the four parameters:

[0020]

[0021]

[0022]

[0023]

[0024] Where [i, j] refers to the horizontal and vertical coordinates of each event, ct refers to the number of events, G x represents the horizontal gradient of the average time surface, G y Represents the vertical gradient;

[0025] Step 3.2: Update the parameter variables of the motion model using the gradient descent method:

[0026] h x =h x -lrx*dx,h y =h y -lry*dy;

[0027] h z =h z -lrz*dz,θ=θ-lrθ*dθ.

[0028] Among them, lrx, lry, lrz, and lrθ are the learning rates corresponding to each parameter.

[0029] Furthermore, the determining whether the iteration termination condition is met includes:

[0030] When the motion model (h x ,h y ,h z When the gradients dx, dt, dz, and dθ of the four parameters of θ ( , θ) are all less than the preset convergence threshold, it is judged that the iteration termination condition is met.

[0031] Furthermore, the event camera motion compensation method based on the motion model further includes:

[0032] When the gradient of any parameter of the current motion model has an opposite sign to the gradient of the previous iteration, the learning rate of the corresponding parameter is reduced and the process returns to step 2 to continue the iteration.

[0033] The present application also proposes an event camera motion compensation device based on a motion model, which is used to compensate for the motion blur of a high-speed moving target caused by event accumulation, including:

[0034] The initialization module is used to initialize the motion model parameters and learning rate, and read the event sequence output by the event camera according to the preset time window;

[0035] An iterative starting module is used to convert the event sequence in the time window into an average time surface and obtain the centroid of the average time surface;

[0036] A parameter updating module is used to perform a convolution operation on the average time plane using a Sobel operator, calculate the horizontal and vertical gradients of the average time plane, and update the parameters of the motion model using a gradient descent method;

[0037] A position update module, used to update the event position using the parameters of the updated motion model;

[0038] The judgment module is used to judge whether the iteration termination condition is met. If so, the motion compensation of the current time window is terminated, otherwise, the module returns to the iteration start module for the next iteration.

[0039] Furthermore, the position update module uses the parameters of the updated motion model to update the event position and performs the following operations:

[0040] The event position is updated using the following formula:

[0041]

[0042] Where (x, y) is the event position before the update, (x′, y′) is the event position after the update, (mx, my) is the centroid of the average time surface, t0 is the initial time in the time window, t is the time of the current event, and h x ,h y ,h z , θ is the motion model (h x ,h y ,h z , θ) parameters.

[0043] Furthermore, the parameter updating module uses a gradient descent method to update the parameters of the motion model and performs the following operations:

[0044] Step 3.1, calculate the gradient of the motion model parameter variable;

[0045] Define the loss function Thus, the motion model (h x ,h y ,h z ,θ) are the gradients of the four parameters:

[0046]

[0047]

[0048]

[0049]

[0050] Where [i, j] refers to the horizontal and vertical coordinates of each event, ct refers to the number of events, G x represents the horizontal gradient of the average time surface, G y Represents the vertical gradient;

[0051] Step 3.2: Update the parameter variables of the motion model using the gradient descent method:

[0052] h x =h x -lrx*dx,h y =h y -lry*dy;

[0053] h z =h z -lrz*dz,θ=θ-lrθ*dθ.

[0054] Among them, lrx, lry, lrz, and lrθ are the learning rates corresponding to each parameter.

[0055] Furthermore, the determination module whether the iteration termination condition is reached includes:

[0056] When the motion model (h x ,h y ,h z When the gradients dx, dt, dz, and dθ of the four parameters of θ ( , θ) are all less than the preset convergence threshold, it is judged that the iteration termination condition is met.

[0057] Furthermore, the judgment module further performs the following operations:

[0058] When the gradient of any parameter of the current motion model has an opposite sign to the gradient of the previous iteration, the learning rate of the corresponding parameter is reduced and the iteration is returned to the starting module to continue the iteration.

[0059] The present application proposes a method and device for event camera motion compensation based on a motion model, which achieves the effect of motion compensation by establishing a motion model, updating the parameters of the motion model, and using the parameters of the updated motion model to update the event position. It can effectively compensate for the motion blur of high-speed moving targets caused by event accumulation. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is a flow chart of the event camera motion compensation method based on the motion model of this application;

[0061] Figure 2 This is a schematic diagram of the structure of an event camera motion compensation device based on a motion model of the present application;

[0062] Figure 3 This is the effect image before motion compensation;

[0063] Figure 4 This is the effect image after motion compensation. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0065] Event cameras, also known as event cameras, are asynchronous sensors that sample light based on scene dynamics. Standard cameras acquire complete images at a specified rate (such as 30fps), while event cameras, such as dynamic vision sensors (DVS), respond asynchronously and independently to brightness changes in the scene for each pixel. The output of an event camera is a sequence of "events" or "pulses" at a variable rate. Each event represents a change in light brightness. When the light intensity changes from the previous moment by more than a certain threshold, a pulse is generated and transmitted by the chip. The information contained includes position, positive and negative polarity (light becomes stronger or weaker), and current time. It is based on the pulse mechanism in biological vision. The time surface is the representation of the position and time information of an event in the form of a two-dimensional graph, and the time information is the "pixel value" of each position.

[0066] This application discloses an event camera motion compensation method based on a motion model, such as Figure 1 As shown, including:

[0067] Step S1, initializing motion model parameters and learning rate, and reading the event sequence output by the event camera according to a preset time window.

[0068] The technical solution of the present application is aimed at static cameras or scenarios without IMU functions. A motion model is established for moving targets, and compensation is made by iteratively optimizing the motion model, thereby solving the motion blur problem caused by high-speed moving targets.

[0069] The established motion model is expressed as (h x ,h y ,h z ,θ), where h x ,h y ,h z They correspond to the movements in the x-axis, y-axis and z-axis directions respectively, and θ corresponds to the rotation angle. Initially, these parameters are set to 0.

[0070] At the same time, the initial learning rate lr=1 for subsequent iterative operations is also set.

[0071] This application sets a time window and performs motion compensation on the event sequence within the time window. In a time window, the accumulated events will form motion blur. The reason for this blur is that the event points are displaced due to motion during this period, and the superposition will appear blurry.

[0072] The application pre-sets a time window, for example, the time window is 10ms, that is, each time a 10ms event sequence is obtained and then subsequent iterative processing is performed. The application does not limit the specific size of the time window. The initial time of the time window is represented as t0.

[0073] Step S2: convert the event sequence in the time window into an average time surface, and obtain the centroid of the average time surface.

[0074] After reading the event sequence in a time window, the event sequence in the time window is converted into an average time plane. Specifically, the time information of the event at position (i, j) in the time window is t, and the cumulative number of events at position (i, j) is I i，j , then the average time surface is:

[0075]

[0076] Next, calculate the centroid of the mean time surface:

[0077] The x-coordinate of all events in the cumulative time window is cx, the y-coordinate of all events in the cumulative time window is cy, and the number of statistical events is ct.

[0078] Calculate the centroid as:

[0079]

[0080] Step S3: Use the Sobel operator to perform a convolution operation on the average time plane, calculate the horizontal and vertical gradients of the average time plane, and use the gradient descent method to update the parameters of the motion model.

[0081] The Sobel operator is one of the operators in image processing, mainly used for edge detection. Technically, it is a discrete difference operator used to calculate the approximate value of the gradient of the image brightness function. Using this operator at any point in the image will generate a corresponding gradient vector.

[0082] In this embodiment, the Sobel operator in the horizontal direction is set to sobel_x=[3, 0, -3; 10, 0, -10; 3, 0, -3], and the Sobel operator in the vertical direction is set to sobel_y=[-3, -10, -3; 0, 0, 0; 3, 10, 3].

[0083] It should be noted that the above-mentioned Sobel operator is only a preferred embodiment, and those skilled in the art may use weight values ​​of different rows and columns to generate different Sobel operators, which will not be described in detail here.

[0084] Use sobel_x and sobel_y to convolve the average time plane respectively. If there is no event at the position corresponding to the convolution kernel, it will be skipped. The horizontal gradient G of the average time plane is obtained. x and the vertical gradient G y .

[0085] Next, the gradient descent method is used to update the parameters of the motion model, which includes the following steps:

[0086] Step 3.1: Calculate the gradient of the motion model parameter variables.

[0087] Define the loss function Thus, the motion model (h x ,h y ,h z ,θ) are the gradients of the four parameters:

[0088]

[0089]

[0090]

[0091]

[0092] Where [i, j] refers to the horizontal and vertical coordinates of each event, and ct refers to the number of events.

[0093] Step 3.2: Update the parameter variables of the motion model using the gradient descent method:

[0094] h x =h x -lrx*dx,h y =h y -lry*dy;

[0095] h z =h z -lrz*dz,θ=θ-lrθ*dθ.

[0096] Among them, lrx, lry, lrz, and lrθ are the learning rates corresponding to each parameter.

[0097] Step S4: Update the event position using the updated motion model parameters.

[0098] After updating the parameters of the motion model, this embodiment can update the event position according to the new motion model, thereby performing motion compensation for the motion blur of the high-speed moving target caused by event accumulation.

[0099] Update the event position. The specific update formula is as follows:

[0100]

[0101] Where (x, y) is the event position before the update, (x′, y′) is the event position after the update, (mx, my) is the centroid, t0 is the initial time in the time window, and t is the time of the current event.

[0102] Step S5: determine whether the iteration termination condition is met. If so, end the motion compensation of the current time window. Otherwise, return to step S2 for the next iteration.

[0103] The application may set a maximum number of iterations for the termination condition of the iteration, and stop the iteration when the maximum number of iterations is reached. x ,h y ,h z The iteration termination condition is that the gradients dx, dy, dz, and dθ of the four parameters of θ are all less than the preset convergence threshold.

[0104] In a specific embodiment, determining whether an iteration termination condition is met includes:

[0105] When the motion model (h x ,h y ,h z When the gradients dx, dy, dz, and dθ of the four parameters of θ are all less than the preset convergence threshold, it is judged that the iteration termination condition is met.

[0106] Specifically, a corresponding convergence threshold is set for the gradient of each parameter of the motion model. x ,h y ,h z When the gradients of the four parameters dx, dt, dz, and dθ of the four time window (θ, dx) are all less than the preset convergence threshold, it is considered that the iteration termination condition is met and the event position is no longer updated. The motion compensation of the current time window ends and the final compensation result is output, thus avoiding some unnecessary compensation operations. For example, the convergence threshold of dx and dy is 10 -5 , the convergence threshold of dz is 10 -4 , the convergence threshold of dθ is 0.1, and the above convergence threshold can be adjusted according to actual needs.

[0107] After the event sequence processing in one time window is completed, the present application obtains the event sequence in the next time window from the event camera again, and processes the event sequence in the obtained time window through the above steps, which will not be repeated here.

[0108] In a specific embodiment, the motion model-based event camera motion compensation method, after updating the event position using the updated motion model parameters, further includes:

[0109] When the gradient of any parameter of the current motion model has an opposite sign to the gradient of the previous iteration, the learning rate of the corresponding parameter is reduced and the process returns to step 2 to continue the iteration.

[0110] In this embodiment, considering the current motion model (h x ,h y ,h z When the gradient of any of the four parameters of the iteration (θ, θ) has an opposite sign to the gradient of the previous iteration, it indicates that the gradient is too large each time and the learning rate needs to be reduced. For example, the new learning rate can be half of the initial learning rate, or reduced by a preset percentage, and there is no specific restriction here.

[0111] For example, if the sign of dx in the current iteration is opposite to that of dx in the previous iteration, then reduce lrx to half of the original value, and return to step 2 for the next iteration. If the sign of dx and dy in the current iteration is opposite to that of dx and dy in the previous iteration, then reduce lrx and lry to half of the original value, and return to step 2 for the next iteration, and so on.

[0112] In one embodiment, Figure 2 As shown, the present application also provides an event camera motion compensation device based on a motion model, which is used to compensate for the motion blur of a high-speed moving target caused by event accumulation, including:

[0113] The initialization module is used to initialize the motion model parameters and learning rate, and read the event sequence output by the event camera according to the preset time window;

[0114] An iterative starting module is used to convert the event sequence in the time window into an average time surface and obtain the centroid of the average time surface;

[0115] A parameter updating module is used to perform a convolution operation on the average time plane using a Sobel operator, calculate the horizontal and vertical gradients of the average time plane, and update the parameters of the motion model using a gradient descent method;

[0116] A position update module, used to update the event position using the parameters of the updated motion model;

[0117] The judgment module is used to judge whether the iteration termination condition is met. If so, the motion compensation of the current time window is terminated, otherwise, the module returns to the iteration start module for the next iteration.

[0118] For the specific definition of the event camera motion compensation device based on the motion model, please refer to the definition of the event camera motion compensation method based on the motion model above, which will not be repeated here. Each module in the above-mentioned event camera motion compensation device based on the motion model can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0119] The memory and the processor are electrically connected directly or indirectly to achieve data transmission or interaction. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines. The memory stores a computer program that can be run on the processor, and the processor implements the network topology layout method in the embodiment of the present invention by running the computer program stored in the memory.

[0120] The memory may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc. The memory is used to store a program, and the processor executes the program after receiving an execution instruction.

[0121] The processor may be an integrated circuit chip with data processing capabilities. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0122] In a specific embodiment, the position updating module uses the parameters of the updated motion model to update the event position and performs the following operations:

[0123] The event position is updated using the following formula:

[0124]

[0125] Where (x, y) is the event position before the update, (x′, y′) is the event position after the update, (mx, my) is the centroid of the average time surface, t0 is the initial time in the time window, t is the time of the current event, and h x ,h y ,h z , θ is the motion model (h x ,h y ,h z , θ) parameters.

[0126] In a specific embodiment, the parameter updating module uses a gradient descent method to update the parameters of the motion model and performs the following operations:

[0127] Step 3.1, calculate the gradient of the motion model parameter variable;

[0128] Define the loss function Thus, the motion model (h x ,h y ,h z ,θ) are the gradients of the four parameters:

[0129]

[0130]

[0131]

[0132]

[0133] Where [i, j] refers to the horizontal and vertical coordinates of each event, ct refers to the number of events, G x represents the horizontal gradient of the average time surface, G y Represents the vertical gradient;

[0134] Step 3.2: Update the parameter variables of the motion model using the gradient descent method:

[0135] h x =h x -lrx*dx,h y =h y -lry*dy;

[0136] hz =h z -lrz*dz,θ=θ-lrθ*dθ.

[0137] Among them, lrx, lry, lrz, and lrθ are the learning rates corresponding to each parameter.

[0138] In a specific embodiment, the determining whether the iteration termination condition is met includes:

[0139] When the motion model (h x ,h y ,h z When the gradients dx, dt, dz, and dθ of the four parameters of θ ( , θ) are all less than the preset convergence threshold, it is judged that the iteration termination condition is met.

[0140] In a specific embodiment, the judgment module further performs the following operations:

[0141] When the gradient of any parameter of the current motion model has an opposite sign to the gradient of the previous iteration, the learning rate of the corresponding parameter is reduced and the iteration is returned to the starting module to continue the iteration.

[0142] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. An event camera motion compensation method based on a motion model, used to compensate for motion blur of a high-speed moving target caused by event accumulation, characterized in that: The event camera motion compensation method based on the motion model comprises: Step 1: Initialize the motion model parameters and learning rate, and read the event sequence output by the event camera according to the preset time window; Step 2: Convert the event sequence in the time window into an average time surface and obtain the centroid of the average time surface; Step 3: Use the Sobel operator to perform a convolution operation on the average time plane, calculate the horizontal and vertical gradients of the average time plane, and use the gradient descent method to update the parameters of the motion model; Step 4: Update the event position using the updated motion model parameters; Step 5: Determine whether the iteration termination condition is met. If so, end the motion compensation of the current time window. Otherwise, return to step 2 for the next iteration. Among them, the event sequence in the time window is converted into an average time surface, including: The time information of the event at position (i, j) in the time window is t, and the cumulative number of events at position (i, j) is I i，j , then the average time surface is:

2. The event camera motion compensation method based on motion model according to claim 1, characterized in that: The updating of the event position by using the updated motion model parameters includes: The event position is updated using the following formula: Where (x, y) is the event position before the update, (x′, y′) is the event position after the update, (mx, my) is the centroid of the average time surface, t0 is the initial time in the time window, t is the time of the current event, and h x ,h y ,h z , θ is the motion model (h x ,h y ,h z ,θ) parameters, where h x 、h y 、h z They correspond to the movements in the x-axis, y-axis and z-axis directions respectively, and θ corresponds to the rotation angle.

3. The event camera motion compensation method based on motion model according to claim 1, characterized in that: The method of updating the parameters of the motion model by using the gradient descent method includes: Step 3.1, calculate the gradient of the motion model parameter variable; Define the loss function Thus, the motion model (h x ,h y ,h z ,θ) are the gradients of the four parameters: Where [i, j] refers to the horizontal and vertical coordinates of each event, ct refers to the number of events, G x represents the horizontal gradient of the average time surface, G y Represents the vertical gradient; Step 3.2: Update the parameter variables of the motion model using the gradient descent method: h x =h x -lrx*dx,h y =h y -lry*dy; h z =h z -lrz*dz,θ=θ-lrθ*dθ; Among them, lrx, lry, lrz, and lrθ are the learning rates corresponding to each parameter.

4. The event camera motion compensation method based on motion model as claimed in claim 3, characterized in that: The determining whether the iteration termination condition is reached includes: When the motion model (h x ,h y ,h z When the gradients dx, dy, dz, and dθ of the four parameters of θ are all less than the preset convergence threshold, it is judged that the iteration termination condition is met.

5. The event camera motion compensation method based on motion model as claimed in claim 1, characterized in that: The event camera motion compensation method based on the motion model further includes: When the gradient of any parameter of the current motion model has an opposite sign to the gradient of the previous iteration, the learning rate of the corresponding parameter is reduced and the process returns to step 2 to continue the iteration.

6. An event camera motion compensation device based on a motion model, used to compensate for motion blur of a high-speed moving target caused by event accumulation, characterized in that: The event camera motion compensation device based on the motion model comprises: The initialization module is used to initialize the motion model parameters and learning rate, and read the event sequence output by the event camera according to the preset time window; An iterative start module is used to convert the event sequence in the time window into an average time surface and obtain the centroid of the average time surface; A parameter updating module is used to perform a convolution operation on the average time plane using a Sobel operator, calculate the horizontal and vertical gradients of the average time plane, and update the parameters of the motion model using a gradient descent method; A position update module, used to update the event position using the parameters of the updated motion model; A judgment module is used to judge whether the iteration termination condition is met. If so, the motion compensation of the current time window is terminated, otherwise, the iteration start module is returned to perform the next iteration; Among them, the event sequence in the time window is converted into an average time surface, including: The time information of the event at position (i, j) in the time window is t, and the cumulative number of events at position (i, j) is I i，j , then the average time surface is:

7. The event camera motion compensation device based on motion model according to claim 6, characterized in that: The position update module uses the parameters of the updated motion model to update the event position and performs the following operations: The event position is updated using the following formula: Where (x, y) is the event position before the update, (x′, y′) is the event position after the update, (mx, my) is the centroid of the average time surface, t0 is the initial time in the time window, t is the time of the current event, and h x ,h y ,h z , θ is the motion model (h x ,h y ,h z ,θ) parameters, where h x 、h y 、h z They correspond to the movements in the x-axis, y-axis and z-axis directions respectively, and θ corresponds to the rotation angle.

8. The event camera motion compensation device based on motion model according to claim 6, characterized in that: The parameter updating module uses the gradient descent method to update the parameters of the motion model and performs the following operations: Step 3.1, calculate the gradient of the motion model parameter variable; Define the loss function Thus, the motion model (h x ,h y ,h z ,θ) are the gradients of the four parameters: Where [i, j] refers to the horizontal and vertical coordinates of each event, ct refers to the number of events, G x represents the horizontal gradient of the average time surface, G y Represents the vertical gradient; Step 3.2: Update the parameter variables of the motion model using the gradient descent method: h x =h x -lrx*dx,h y =h y -lry*dy; h z =h z -lrz*dz,θ=θ-lrθ*dθ; Among them, lrx, lry, lrz, and lrθ are the learning rates corresponding to each parameter.

9. The event camera motion compensation device based on motion model according to claim 8, characterized in that: The judging module judges whether the iteration termination condition is reached, including: When the motion model (h x ,h y ,h z When the gradients dx, dy, dz, and dθ of the four parameters of θ are all less than the preset convergence threshold, it is judged that the iteration termination condition is met.

10. The event camera motion compensation device based on motion model according to claim 6, characterized in that: The judging module further performs the following operations: When the gradient of any parameter of the current motion model has an opposite sign to the gradient of the previous iteration, the learning rate of the corresponding parameter is reduced and the iteration starts and the module is returned to continue the iteration.

Images