Motion estimation method and device, equipment, storage medium and chip

Abstract

The invention relates to a motion estimation method and device, equipment, a storage medium and a chip, and the method comprises the steps: obtaining a first event accumulation diagram and a second event accumulation diagram, the first event accumulation diagram is obtained through the accumulation processing of event data outputted by an event camera for shooting a target object in a preset first time period, and the second event accumulation diagram is obtained through the accumulation processing of the event data; the second event accumulation diagram is obtained by performing accumulation processing on event data output by the event camera for shooting the target object within a preset second time period; and performing motion estimation based on the first event accumulation diagram and the second event accumulation diagram to obtain the motion intensity of the target object. Therefore, the problems of inaccurate motion intensity estimation, influence on estimation precision and the like in the prior art can be solved.

Description

Technical Field

[0001] The present disclosure relates to the field of terminal technology, and in particular to a motion estimation method, device, equipment, storage medium and chip. Background Art

[0002] In recent years, with the rapid development of image processing technology and the emergence of various visual sensors, motion estimation technology based on machine vision has also been widely used. For example, in motion capture scenarios, terminal devices (such as mobile phones) are currently used to capture moving objects and obtain multiple frames of continuous contact image sensor (CIS) images; then the motion intensity of the moving object is estimated based on the CSI image. However, in practice, it is found that the captured CIS image may be blurred and unclear, which leads to inaccurate estimation of the motion intensity of the moving object, affecting the accuracy of motion intensity estimation. Summary of the invention

[0003] In order to overcome the problems existing in the related art, the present disclosure provides a motion estimation method, device, equipment, storage medium and chip to solve the technical problems existing in the above-mentioned related art such as inaccurate motion intensity estimation and affecting estimation accuracy.

[0004] According to a first aspect of an embodiment of the present disclosure, a motion estimation method is provided, including:

[0005] Acquire a first event accumulation graph and a second event accumulation graph, wherein the first event accumulation graph is obtained by accumulating event data output by an event camera when photographing a target object within a preset first time period, and the second event accumulation graph is obtained by accumulating event data output by the event camera when photographing the target object within a preset second time period;

[0006] Motion estimation is performed based on the first event accumulation graph and the second event accumulation graph to obtain the motion intensity of the target object.

[0007] In some embodiments, performing motion estimation based on the first event accumulation graph and the second event accumulation graph to obtain the motion intensity of the target object includes:

[0008] Performing feature matching processing on the first event accumulation map and the second event accumulation map to obtain difference pixel pair information, wherein the difference pixel pair information includes position difference information between pixel points having a position mapping relationship between the first event accumulation map and the second event accumulation map;

[0009] Motion estimation is performed based on the difference pixel pair information, the preset first time period and the preset second time period to obtain the motion intensity of the target object.

[0010] In some embodiments, performing motion estimation based on the difference pixel pair information, the preset first time period and the preset second time period to obtain the motion intensity of the target object includes:

[0011] Determining an event time difference based on the preset first time period and the preset second time period;

[0012] Estimating the motion speed based on the difference pixel pair information and the event time difference to obtain the motion speed of the target object;

[0013] The motion intensity of the target object is estimated based on the motion speed of the target object to obtain the motion intensity of the target object, where the motion intensity is used to reflect the motion amplitude of each pixel point of the target object in the shooting field of view of the event camera.

[0014] In some embodiments, the method further comprises:

[0015] Obtaining the motion intensity of each pixel in a preset area, where the preset area is a sub-area in the shooting field of view;

[0016] The motion intensity of each pixel in the preset area is averaged to obtain the average motion intensity of the target object in the preset area.

[0017] In some embodiments, the event data is exposure value EVS data output by the event camera.

[0018] In some embodiments, the preset first time period and the preset second time period are two adjacent time periods.

[0019] In some embodiments, the method further comprises:

[0020] Acquire contact image sensor CIS data, where the CIS data is data outputted by photographing the target object based on a photographing field of view of the event camera;

[0021] Estimating the ambient brightness based on the CIS data to obtain the ambient brightness of the target object;

[0022] Based on the motion intensity of the target object and the ambient brightness, an exposure parameter of the event camera is determined.

[0023] In some embodiments, the method further comprises:

[0024] Acquiring CIS data in the preset area;

[0025] Estimating the ambient brightness based on the CIS data in the preset area to obtain the ambient brightness corresponding to the preset area;

[0026] An exposure parameter of the event camera is determined based on an average motion intensity of the target object in the preset area and the ambient brightness.

[0027] According to a second aspect of an embodiment of the present disclosure, a motion estimation device is provided, including:

[0028] an acquisition module, configured to acquire a first event accumulation graph and a second event accumulation graph, wherein the first event accumulation graph is obtained by accumulating event data output by an event camera when photographing a target object within a preset first time period, and the second event accumulation graph is obtained by accumulating event data output by the event camera when photographing the target object within a preset second time period;

[0029] The processing module is configured to perform motion estimation based on the first event accumulation graph and the second event accumulation graph to obtain the motion intensity of the target object.

[0030] For the contents not introduced or described in the embodiments of the present disclosure, please refer to the relevant introduction in the aforementioned method embodiments, and the embodiments of the present disclosure are not limited thereto.

[0031] According to a third aspect of an embodiment of the present disclosure, a terminal device is provided, comprising: a processor; and a memory for storing processor executable instructions; wherein the processor is configured to execute the executable instructions to implement the steps of the above-mentioned motion estimation method.

[0032] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the motion estimation method provided in the first aspect of the present disclosure are implemented.

[0033] According to a fifth aspect of an embodiment of the present disclosure, a chip is provided, including: a processor and an interface; the processor is used to read instructions to execute the steps of the above-mentioned motion estimation method.

[0034] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: the terminal device obtains a first event accumulation graph and a second event accumulation graph, wherein the first event accumulation graph is obtained by accumulating and processing the event data output by the event camera shooting the target object within a preset first time period, and the second event accumulation graph is obtained by accumulating and processing the event data output by the event camera shooting the target object within a preset second time period; based on the first event accumulation graph and the second event accumulation graph, motion estimation is performed to obtain the motion intensity of the target object. It can be seen that the terminal device can perform motion estimation of the target object based on the first event accumulation graph and the second event accumulation graph, thereby estimating the motion intensity of the target object. Since the event camera has the characteristics of high temporal resolution, which can usually reach the microsecond level, it can accurately record the motion information of the target object in the event data, so that the motion estimation by using the event camera is more accurate, and the technical problems such as inaccurate motion intensity estimation and affecting the estimation accuracy existing in the related art are solved. This is conducive to improving the accuracy or precision of motion estimation.

[0035] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0037] Figure 1 It is a schematic diagram of a framework according to an exemplary embodiment.

[0038] Figure 2 The figure is a flowchart of a motion estimation method according to an exemplary embodiment.

[0039] Figure 3 It is a schematic diagram of an event accumulation graph according to an exemplary embodiment.

[0040] Figure 4 It is a schematic diagram showing a feature matching process according to an exemplary embodiment.

[0041] Figure 5 The diagram is a schematic structural diagram of a motion estimation device according to an exemplary embodiment.

[0042] Figure 6 The diagram is a schematic structural diagram of a terminal device according to an exemplary embodiment.

[0043] Figure 7 The figure is a schematic diagram showing the structure of a chip according to an exemplary embodiment. DETAILED DESCRIPTION

[0044] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0045] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.

[0046] See also Figure 1 FIG. 1 is a schematic diagram of a framework according to an exemplary embodiment. Figure 1 The framework schematic diagram shown is applied to a terminal device, in which a dual-camera module or a single hybrid module is provided. The dual-camera module may include a CIS camera module and an exposure value (Exposure Values, EVS) module, and the CIS camera module is used to shoot and obtain CIS data, which may also be referred to as CIS image data. The CIS image data may include the CIS data of each pixel in the shooting field of view corresponding to the above-mentioned CIS camera module. The EVS camera module is used to shoot and obtain EVS data, which may also be referred to as event data. The EVS data may include the event data of each pixel in the shooting field of view corresponding to the above-mentioned EVS camera module, and the event data may specifically include the horizontal and vertical coordinates, timestamps and event polarity of each pixel. The event polarity indicates the polarity of the event, such as the pixel becoming brighter or the pixel becoming darker, which may specifically refer to an increase or decrease in the brightness value of the pixel. The hybrid module is used to capture image data, which includes both CIS data and EVS data; that is, the image captured by the hybrid module includes both CIS pixels and EVS pixels. In practical applications, the image captured by the hybrid module usually has CIS pixels at designated (pixel) positions and EVS pixels at other positions. Accordingly, the terminal device can extract the corresponding EVS data and CIS data from the image data captured by the hybrid module according to the pixel position.

[0047] For the convenience of description, the present disclosure is Figure 1 The relevant contents are described in the example of a terminal device equipped with a dual camera module including a CIS and an EVS camera module, but this does not constitute a limitation. Figure 1The framework diagram shown may include: a CIS camera module 100, an EVS camera module 200, an ambient brightness estimation module 300, a motion intensity estimation module 400 and an exposure parameter determination module 500. Among them:

[0048] The CIS camera module 100 is used to capture CIS data, for example, to capture a target object to obtain corresponding CIS data, etc. The target object may be a moving object or a stationary object, including but not limited to pedestrians, trees, puppies, kittens or other custom objects.

[0049] The EVS camera module 200 is used to obtain EVS data by shooting, for example, the target object can be photographed to obtain corresponding EVS data, etc. The EVS data can also be called event data, which may include (x, y, p, t), where x and y represent the pixel coordinates of the event in two-dimensional space, such as the coordinates of each pixel constituting the target object, etc.; p represents the polarity of the event, such as the pixel becoming brighter or darker, etc.; t represents the timestamp of the event, which may specifically represent the timestamp corresponding to the shooting of the target object, etc.

[0050] The above-mentioned environment brightness estimation module 300 is mainly responsible for estimating the environment brightness based on the CIS data obtained by shooting, so as to estimate the corresponding environment brightness. The specific implementation of the above-mentioned environment brightness estimation will be introduced below in this disclosure, and will not be described in detail here.

[0051] The motion intensity estimation module 400 is mainly responsible for performing motion intensity estimation based on the EVS data obtained by shooting, so as to estimate the corresponding motion intensity. The specific implementation of the motion intensity estimation will be introduced below in this disclosure and will not be described in detail here.

[0052] The above-mentioned exposure parameter determination module 500 is mainly responsible for determining the corresponding exposure parameters based on the ambient brightness and motion intensity obtained by the above-mentioned estimation. It is convenient for the above-mentioned CIS camera module 100 and / or the above-mentioned EVS camera module 200 to adjust their own exposure parameters, and then the determined exposure parameters can be used for subsequent image capture, etc. The specific implementation method of the above-mentioned exposure parameter determination will also be introduced below in this disclosure and will not be described in detail here. The exposure parameters include, but are not limited to, for example, exposure time, sensitivity (ISO) or other parameters used to affect the shooting of the camera module.

[0053] Based on the above examples, see Figure 2 FIG. 1 is a flow chart showing a motion estimation method according to an exemplary embodiment. Figure 2 The method shown is applied to a terminal device, and the method may include the following implementation steps:

[0054] S201. Obtain a first event accumulation graph and a second event accumulation graph, wherein the first event accumulation graph is obtained by accumulating event data output by an event camera when photographing a target object within a preset first time period, and the second event accumulation graph is obtained by accumulating event data output by the event camera when photographing the target object within a preset second time period.

[0055] The above-mentioned first event accumulation graph of the present disclosure may be obtained by accumulating and processing the event data output by the event camera when photographing the target object within a preset first time period. In practical applications, the above-mentioned event data may refer to the EVS data output by the event camera when photographing the target object at different times. That is, the present disclosure may accumulate and process the EVS data output by the event camera when photographing the target object within a preset first time period, so as to obtain the above-mentioned first event accumulation graph. The present disclosure does not limit the specific implementation method of the above-mentioned accumulation processing. For example, the EVS data obtained by photographing at each moment within the preset first time period may be stacked together according to the pixel position, so as to form the above-mentioned first event accumulation graph. For example, please refer to Figure 3 FIG. 1 is a schematic diagram showing an event accumulation graph according to an exemplary embodiment. Figure 3 The event accumulation graph shown may include information of at least one pixel (also referred to as a pixel position), and the information includes but is not limited to, for example, the coordinates of the pixel and event polarity, etc., which are not excessively limited or detailed in the present disclosure.

[0056] Similarly, the second event accumulation graph may be obtained by accumulating the event data output by the event camera when shooting the target object within the preset second time period. The introduction of the second event accumulation graph may refer to the relevant introduction of the first event accumulation graph, which will not be repeated here. The preset first time period and the preset second time period may refer to two different time periods customized by the terminal device or the user. Usually, the first preset time period and the preset second time period refer to two adjacent time periods. For example, the first preset time period may be the time period [t-τ, t], and the preset second time period may be the time period [t, t+τ], etc.

[0057] S202: Perform motion estimation based on the first event accumulation graph and the second event accumulation graph to obtain motion intensity of the target object.

[0058] The present disclosure does not limit the specific implementation methods of the above-mentioned motion estimation. For example, in one implementation method, the present disclosure may perform feature matching processing on the above-mentioned first event accumulation map and the above-mentioned second event accumulation map, so as to obtain corresponding difference pixel pair information, and the difference pixel pair information may include position difference information between pixel points that have a position mapping relationship in the above-mentioned first event accumulation map and the above-mentioned second event accumulation map. Among them, the present disclosure does not limit the specific implementation methods of the above-mentioned feature matching processing. For example, a preset image processing algorithm may be used to perform feature point matching on the above-mentioned first event accumulation map and the above-mentioned second event accumulation map. The image processing algorithm is a pre-customized processing algorithm for image feature matching, such as a feature point matching algorithm, etc. The present disclosure does not make too many limitations and details on this. For example, please refer to Figure 4 FIG. 1 is a schematic diagram showing a feature matching process according to an exemplary embodiment. Figure 4 As shown, the terminal device may perform feature matching on the pixel points having a position mapping relationship in the first event accumulation graph and the second event accumulation graph, thereby obtaining position difference information between the pixel points.

[0059] Furthermore, the present disclosure may perform motion estimation based on the above-mentioned difference pixel pair information, the above-mentioned preset first time period and the above-mentioned preset second time period, so as to estimate the motion intensity of the above-mentioned target object, which may also be referred to as the motion amplitude of the above-mentioned target object. Among them, the present disclosure does not limit the implementation method of the motion estimation. For example, the present disclosure may first determine the corresponding event occurrence time difference based on the above-mentioned preset first time period and the preset second time period, which may be referred to as the event time difference. The event time difference is the difference between the end time of the above-mentioned preset second time period and the start time of the above-mentioned preset first time period. For example, the above-mentioned preset first time period and the preset second time period are the time periods [t-τ, t] and [t, t+τ] respectively, and the event time difference between them is 2τ. Next, the present disclosure may perform motion speed estimation based on the above-mentioned difference pixel pair information and the above-mentioned event time difference, so as to estimate the motion speed of the above-mentioned target object. The specific estimation process is shown in the following formula (1):

[0060]

[0061] Among them, (x, y) represents the pixel position, represents the movement speed of the pixel point (x, y), Δx represents the position difference information of the above difference pixel pair information in the horizontal coordinate, Δy represents the position difference information of the above difference pixel pair information in the vertical coordinate, and Δt represents the time difference of the above event.

[0062] Finally, the present disclosure can estimate the motion intensity based on the motion speed of the target object, thereby estimating the motion intensity of the target object. The motion intensity may include the motion intensity or motion amplitude of each pixel of the target object in the shooting field of view of the event camera. The specific estimation process can be shown in the following formula (2):

[0063]

[0064] in, Indicates the motion intensity of the pixel (x, y).

[0065] In some optional embodiments, the present disclosure can obtain the motion intensity of each pixel in a preset area, and the preset area can be any sub-area in the shooting field of view of the above-mentioned event camera; that is, the present disclosure can obtain the motion intensity of the preset area from the motion intensity of the target object corresponding to the above-mentioned shooting field of view. The present disclosure does not limit the size and position of the above-mentioned preset area, which can be determined according to actual needs. For example, the size of the preset area can be a K×K window size, etc., where K is a user-defined value. In practical applications, the preset area can be a user-defined area of ​​interest, or it can be a focus area of ​​the image, or it can be a customized image center area, etc., and the present disclosure does not make too many restrictions and details on this. Furthermore, the present disclosure can average (average) the motion intensity of each pixel in the above-mentioned preset area, so as to obtain the average motion intensity of the above-mentioned target object in the preset area.

[0066] In some optional embodiments, the present disclosure further obtains the CIS data in the preset area output by the event camera shooting the target object. For example, the CIS camera module in the event camera can be used to shoot in its shooting field of view to obtain CIS data, and then the CIS data of the preset area can be obtained from the CIS data of the shooting field of view. Further, the present disclosure can estimate the ambient brightness based on the CIS data in the preset area, so as to estimate the ambient brightness corresponding to the preset area. Among them, the present disclosure does not limit the implementation method of the ambient brightness estimation. For example, the present disclosure can perform brightness average calculation on the CIS data of each pixel in the preset area, so as to use the average brightness value of each pixel in the preset area as the ambient brightness corresponding to the preset area. Finally, the present disclosure can determine the exposure parameters of the event camera based on the average motion intensity in the preset area and the ambient brightness corresponding to the preset area. The present disclosure does not limit the implementation method of determining the exposure parameters. For example, the present disclosure can query the exposure parameters corresponding to the average motion intensity and ambient brightness of the preset area from a preset list by looking up a table. The exposure parameters include, but are not limited to, exposure time, sensitivity (ISO) or other customized exposure parameters.

[0067] In some optional embodiments, the present disclosure may also obtain CIS data, which may be CIS data output by photographing the target object in the shooting field of view of the event camera, such as data output by photographing the target object in its shooting field of view using the CIS camera module of the event camera, etc.; that is, the CIS data in the entire shooting field of view of the event camera may be obtained. Further, the present disclosure may estimate the ambient brightness based on the CIS data, thereby estimating the ambient brightness of the target object. The implementation method of the ambient brightness estimation is not limited. For example, the present disclosure may perform brightness average calculation on the CIS data of each pixel in the shooting field of view, thereby obtaining the average brightness value of each pixel as the ambient brightness of the target object. Finally, the present disclosure may determine the exposure parameters of the event camera based on the motion intensity of the target object in the entire shooting field of view and the ambient brightness; the implementation method of determining the exposure parameters is not limited. For example, the present disclosure may query the exposure parameters matching the motion intensity and ambient brightness of the target object from a preset list by looking up a table.

[0068] By implementing the disclosed embodiment, the terminal device obtains a first event accumulation graph and a second event accumulation graph, wherein the first event accumulation graph is obtained by accumulating and processing the event data output by the event camera shooting the target object within a preset first period, and the second event accumulation graph is obtained by accumulating and processing the event data output by the event camera shooting the target object within a preset second period; motion estimation is performed based on the first event accumulation graph and the second event accumulation graph to obtain the motion intensity of the target object. It can be seen that the terminal device can perform motion estimation of the target object based on the first event accumulation graph and the second event accumulation graph, thereby estimating the motion intensity of the target object. Since the event camera has the characteristics of high temporal resolution, which can usually reach the microsecond level, it can accurately record the motion information of the target object in the event data, so that the motion estimation is more accurate by using the event camera, and the technical problems such as inaccurate motion intensity estimation and affecting the estimation accuracy existing in the related art are solved. This is conducive to improving the accuracy or precision of motion estimation. Further, more accurate or precise exposure parameters can be obtained, so that the next motion capture is clearer.

[0069] Based on the above examples, see Figure 5 FIG. 1 is a schematic diagram showing the structure of a motion estimation device according to an exemplary embodiment. Figure 5 The device shown can be applied to a terminal device, and the device may include an acquisition module 501 and a processing module 502. Among them:

[0070] The acquisition module 501 is configured to acquire a first event accumulation graph and a second event accumulation graph, wherein the first event accumulation graph is obtained by accumulating event data output by an event camera when photographing a target object within a preset first period of time, and the second event accumulation graph is obtained by accumulating event data output by the event camera when photographing the target object within a preset second period of time;

[0071] The processing module 502 is configured to perform motion estimation based on the first event accumulation graph and the second event accumulation graph to obtain the motion intensity of the target object.

[0072] In some embodiments, the processing module 502 is configured to:

[0073] Performing feature matching processing on the first event accumulation map and the second event accumulation map to obtain difference pixel pair information, wherein the difference pixel pair information includes position difference information between pixel points having a position mapping relationship between the first event accumulation map and the second event accumulation map;

[0074] Motion estimation is performed based on the difference pixel pair information, the preset first time period and the preset second time period to obtain the motion intensity of the target object.

[0075] In some embodiments, the processing module 502 is configured to:

[0076] Determining an event time difference based on the preset first time period and the preset second time period;

[0077] Estimating the motion speed based on the difference pixel pair information and the event time difference to obtain the motion speed of the target object;

[0078] The motion intensity of the target object is estimated based on the motion speed of the target object to obtain the motion intensity of the target object, where the motion intensity is used to reflect the motion amplitude of each pixel point of the target object in the shooting field of view of the event camera.

[0079] In some embodiments, the processing module 502 is further configured to:

[0080] Obtaining the motion intensity of each pixel in a preset area, where the preset area is a sub-area in the shooting field of view;

[0081] The motion intensity of each pixel in the preset area is averaged to obtain the average motion intensity of the target object in the preset area.

[0082] In some embodiments, the event data is exposure value EVS data output by the event camera.

[0083] In some embodiments, the preset first time period and the preset second time period are two adjacent time periods.

[0084] In some embodiments, the processing module 502 is further configured to:

[0085] Acquire contact image sensor CIS data, where the CIS data is data outputted by photographing the target object based on a photographing field of view of the event camera;

[0086] Estimating the ambient brightness based on the CIS data to obtain the ambient brightness of the target object;

[0087] Based on the motion intensity of the target object and the ambient brightness, an exposure parameter of the event camera is determined.

[0088] In some embodiments, the processing module 502 is further configured to:

[0089] Acquiring CIS data in the preset area;

[0090] Estimating the ambient brightness based on the CIS data in the preset area to obtain the ambient brightness corresponding to the preset area;

[0091] An exposure parameter of the event camera is determined based on an average motion intensity of the target object in the preset area and the ambient brightness.

[0092] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0093] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon. When the program instructions are executed by a processor, the steps of the motion estimation method provided by the present disclosure are implemented.

[0094] Figure 6 6 is a schematic diagram of a terminal device according to an exemplary embodiment. For example, the terminal device 600 may be a mobile phone, a computer, a digital broadcast terminal, a message transceiver, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

[0095] Reference Figure 6 The terminal device 600 may include one or more of the following components: a processing component 602 , a memory 604 , a power component 606 , a multimedia component 608 , an audio component 610 , an input / output interface 612 , a sensor component 614 , and a communication component 616 .

[0096] The processing component 602 generally controls the overall operation of the device 600, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above-mentioned motion estimation method. In addition, the processing component 602 may include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.

[0097] The memory 604 is configured to store various types of data to support operations on the device 600. Examples of such data include instructions for any application or method operating on the device 600, contact data, phone book data, messages, pictures, videos, etc. The memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0098] The power supply component 606 provides power to the various components of the device 600. The power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 600.

[0099] The multimedia component 608 includes a screen that provides an output interface between the device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the terminal device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0100] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC), and when the device 600 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 604 or sent via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.

[0101] The input / output interface 612 provides an interface between the processing component 602 and the peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0102] The sensor assembly 614 includes one or more sensors for providing various aspects of status assessment for the device 600. For example, the sensor assembly 614 can detect the open / closed state of the device 600, the relative positioning of components, such as the display and keypad of the device 600, and the sensor assembly 614 can also detect the position change of the device 600 or a component of the device 600, the presence or absence of user contact with the device 600, the orientation or acceleration / deceleration of the device 600, and the temperature change of the device 600. The sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 614 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0103] The communication component 616 is configured to facilitate wired or wireless communication between the device 600 and other devices. The device 600 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0104] In an exemplary embodiment, the device 600 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-mentioned motion estimation method.

[0105] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, and the instructions can be executed by the processor 620 of the device 600 to complete the above-mentioned superordinate motion estimation method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0106] In addition to being an independent electronic device, the above-mentioned device can also be a part of an independent electronic device. For example, in one embodiment, the device can be an integrated circuit (IC) or a chip, wherein the integrated circuit can be an IC or a collection of multiple ICs; the chip can include but is not limited to the following types: GPU (Graphics Processing Unit), CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), SOC (System on Chip, SoC), etc. The above-mentioned integrated circuit or chip can be used to execute executable instructions (or codes) to implement the above-mentioned motion estimation method. The executable instructions can be stored in the integrated circuit or chip, or can be obtained from other devices or equipment, such as the integrated circuit or chip including a processor, a memory, and an interface for communicating with other devices. The executable instruction may be stored in the memory, and when the executable instruction is executed by the processor, the above-mentioned motion estimation method may be implemented; alternatively, the integrated circuit or chip may receive the executable instruction through the interface and transmit it to the processor for execution, so as to implement the above-mentioned motion estimation method.

[0107] In another exemplary embodiment, a computer program product is also provided. The computer program product includes a computer program executable by a programmable device, and the computer program has a code portion for executing the above motion estimation method when executed by the programmable device.

[0108] See also Figure 7 FIG. 1 is a schematic diagram showing the structure of a chip according to an exemplary embodiment. Figure 7 The chip 700 shown includes a processor 701 and an interface 702. Optionally, it may also include a memory 703. The number of the processor 701 may be one or more, and the number of the interface 702 may be multiple.

[0109] In one embodiment, for a case where a chip is used to implement the method embodiment of the present disclosure:

[0110] The interface 702 is used to receive or output signals;

[0111] The processor 701 is configured to execute part or all of the contents in the motion estimation method embodiment.

[0112] Understandably, the processor in the embodiment of the present disclosure may be an integrated circuit chip having signal processing capabilities. In the implementation process, each step of the above method embodiment may be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0113] It is understandable that the memory in the embodiments of the present disclosure may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0114] It should be noted here that the description of the above storage medium, device and chip embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium, storage medium and device embodiments of the present disclosure, please refer to the description of the method embodiments of the present disclosure for understanding.

[0115] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0116] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A motion estimation method, characterized in that: include: Acquire a first event accumulation graph and a second event accumulation graph, wherein the first event accumulation graph is obtained by accumulating event data output by an event camera when photographing a target object within a preset first time period, and the second event accumulation graph is obtained by accumulating event data output by the event camera when photographing the target object within a preset second time period; Motion estimation is performed based on the first event accumulation graph and the second event accumulation graph to obtain the motion intensity of the target object.

2. The method according to claim 1, characterized in that The performing motion estimation based on the first event accumulation graph and the second event accumulation graph to obtain the motion intensity of the target object comprises: Performing feature matching processing on the first event accumulation map and the second event accumulation map to obtain difference pixel pair information, wherein the difference pixel pair information includes position difference information between pixel points having a position mapping relationship between the first event accumulation map and the second event accumulation map; Motion estimation is performed based on the difference pixel pair information, the preset first time period and the preset second time period to obtain the motion intensity of the target object.

3. The method according to claim 2, characterized in that The performing motion estimation based on the difference pixel pair information, the preset first time period and the preset second time period to obtain the motion intensity of the target object comprises: Determining an event time difference based on the preset first time period and the preset second time period; Estimating the motion speed based on the difference pixel pair information and the event time difference to obtain the motion speed of the target object; The motion intensity of the target object is estimated based on the motion speed of the target object to obtain the motion intensity of the target object, where the motion intensity is used to reflect the motion amplitude of each pixel point of the target object in the shooting field of view of the event camera.

4. The method according to claim 3, characterized in that The method further comprises: Obtaining the motion intensity of each pixel in a preset area, where the preset area is a sub-area in the shooting field of view; The motion intensity of each pixel in the preset area is averaged to obtain the average motion intensity of the target object in the preset area.

5. The method according to any one of claims 1 to 4, characterized in that The event data is exposure value EVS data output by the event camera.

6. The method according to any one of claims 1 to 4, characterized in that The preset first time period and the preset second time period are two adjacent time periods.

7. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Acquire contact image sensor CIS data, where the CIS data is data outputted by photographing the target object based on a photographing field of view of the event camera; Estimating the ambient brightness based on the CIS data to obtain the ambient brightness of the target object; Based on the motion intensity of the target object and the ambient brightness, an exposure parameter of the event camera is determined.

8. The method according to claim 4, characterized in that The method further comprises: Acquiring CIS data in the preset area; Estimating the ambient brightness based on the CIS data in the preset area to obtain the ambient brightness corresponding to the preset area; An exposure parameter of the event camera is determined based on an average motion intensity of the target object in the preset area and the ambient brightness.

9. A motion estimation device, characterized in that: include: an acquisition module, configured to acquire a first event accumulation graph and a second event accumulation graph, wherein the first event accumulation graph is obtained by accumulating event data output by an event camera when photographing a target object within a preset first time period, and the second event accumulation graph is obtained by accumulating event data output by the event camera when photographing the target object within a preset second time period; The processing module is configured to perform motion estimation based on the first event accumulation graph and the second event accumulation graph to obtain the motion intensity of the target object.

10. A terminal device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the executable instructions to implement the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A chip, characterized in that: The method comprises a processor and an interface; the processor is used to read instructions to execute the method according to any one of claims 1 to 8.

Images