Event data set construction method and system

Abstract

The invention relates to the technical field of data processing, in particular to an event data set construction method and system, event data of a target area are acquired from output data of one event camera, and event data of other event cameras in the target area in a target scene are acquired by utilizing pixel size multiples among multiple event cameras. According to the method, the data set which can truly reflect a scene and contains event data with different resolutions is constructed by utilizing an optical light splitting method, and the accuracy and the simplicity of data set acquisition are improved.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, and in particular relates to a method and system for constructing an event dataset. Background Technology

[0002] Image super-resolution is the process of reconstructing a high-resolution image from a low-resolution image to recover details, textures, and edge information lost in the low-resolution image due to undersampling. Super-resolution of event data requires super-resolution not only in the spatial domain but also in the temporal domain.

[0003] An event camera is an image acquisition device that uses a dynamic vision sensor (also known as an event-triggered detector, neuromorphic detector, or bionic vision sensor) as its core imaging component to acquire information about changes in light intensity in a scene. Event cameras have been widely used in machine vision fields such as target classification and recognition due to their advantages of large dynamic range, high temporal resolution, and low data redundancy.

[0004] To improve the recognition accuracy of event cameras in target classification, some researchers have conducted super-resolution studies on event camera output data and constructed datasets. However, due to the difficulty in creating datasets, there are currently few datasets available for verifying the application of event camera super-resolution algorithms. Existing methods involve downsampling high-resolution event data to obtain low-resolution event data. However, the datasets obtained by this method fail to truly reflect the sampling results of the event camera at different resolutions. Moreover, downsampling methods often have certain regularities and cannot guarantee that the downsampling model can be reverse-engineered and applied to super-resolution algorithms. Summary of the Invention

[0005] In view of this, the present invention aims to provide a method and system for constructing an event dataset, which utilizes optical spectroscopy to provide a method for constructing a dataset for an event camera super-resolution algorithm that can truly reflect the sampling results of the event camera.

[0006] To achieve the above objectives, the technical solution created by this invention is implemented as follows: A method for constructing an event dataset, comprising: S1: Using a semi-reflective mirror, the optical information of the target scene is projected onto multiple event cameras with different pixel sizes; S2: Based on the pixel size multiples among multiple event cameras, establish the coordinate space mapping relationship between one event camera and other event cameras; S3: Using coordinate space mapping relationships, determine the imaging area of ​​one event camera corresponding to the imaging area of ​​other event cameras; collect event data in the imaging areas of all event cameras to obtain an event dataset including different resolutions.

[0007] Furthermore, each event camera in step S1 is equipped with a focusable optical lens.

[0008] Furthermore, if multiple event cameras have the same pixel size, the focal length of each event camera's lens is adjusted so that the resolution of the images acquired by the multiple event cameras is different.

[0009] An event dataset construction system includes: multiple event cameras with different pixel sizes; a semi-reflective mirror for projecting optical information of a target scene onto the multiple event cameras; a control center for controlling the multiple event cameras to simultaneously capture images of the target scene; establishing a coordinate space mapping relationship between one event camera and other event cameras based on the pixel size ratios among the multiple event cameras; using the coordinate space mapping relationship to determine the corresponding imaging areas of the imaging area of ​​one event camera in other event cameras; and collecting event data from the imaging areas of all event cameras to obtain an event dataset including different resolutions.

[0010] Furthermore, each event camera is equipped with a focusable optical lens.

[0011] Furthermore, the control center connects to and controls each event camera via cables.

[0012] Compared with the prior art, the present invention can achieve the following beneficial effects: This invention creates an event dataset construction method and system that utilizes a semi-reflective mirror to construct an optical common-path structure, enabling multiple event cameras with different pixel sizes to receive optical information from the same physical scene, thus ensuring spatial alignment at the physical level. Based on this, the pixel size ratio is used as a spatial scale mapping coefficient to establish a deterministic mapping relationship between the coordinate systems of multiple event cameras. This fundamentally avoids the need for post-processing registration. Instead of simple center region cropping, this invention precisely maps a selected pixel region in one event camera to a corresponding spatial region in other event cameras through coordinate mapping relationships, ensuring that the two regions correspond to the same field of view in physical space. This mapping relationship achieves precise alignment for any region and at any scale. Attached Figure Description

[0013] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments and descriptions of the invention are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 A flowchart illustrating the event dataset construction method described in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of the event dataset construction system described in an embodiment of the present invention.

[0014] Explanation of reference numerals in the attached figures: 1. Semi-reflective mirror; 2. Event camera; 3. Control center; 4. Adjustable focus lens; 5. Cables. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not constitute a limitation thereof.

[0016] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0017] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0018] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0019] The invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0020] like Figure 1 As shown, the event dataset construction method described in this embodiment of the invention includes: S1: Using a semi-reflective mirror, the optical information of the target scene is projected onto multiple event cameras with different pixel sizes. Since each event camera has a different pixel size, the field of view of the scene information acquired by each event camera is also different.

[0021] In some embodiments, multiple event cameras with the same pixel size may be selected. However, in order to ensure that event data with different resolutions can be acquired in the end, each event camera is equipped with an adjustable focus optical lens. In this case, by adjusting the focal length of the lens of each event camera, the resolution of the images acquired by the multiple event cameras is different.

[0022] In this embodiment of the invention, two event cameras are specifically used. Correspondingly, a semi-reflective lens is used to project the optical information of the target scene into the first event camera and the second event camera.

[0023] S2: Based on the pixel size multiples between multiple event cameras, establish the coordinate space mapping relationship between one event camera and other event cameras.

[0024] It is understood that, corresponding to the two event cameras in this embodiment of the invention, step S2 can be further explained as follows: Calculate the pixel size multiple k = m / n between the pixel size m of one event camera and the pixel size n of the other event camera; Let the coordinates of any pixel in the first event camera be (u1, v1). Then the center coordinates of the corresponding physical space region in the second event camera are (u2, v2). (u2,v2)=(k×u1,k×v1); The above formula represents the coordinate space mapping relationship between the first event camera and the second event camera.

[0025] S3: Using coordinate space mapping relationships, determine the imaging area of ​​one event camera corresponding to the imaging area of ​​other event cameras; collect event data in the imaging areas of all event cameras to obtain an event dataset including different resolutions.

[0026] It is understood that, corresponding to the two event cameras in this embodiment of the invention, step S3 can be further explained as follows: The imaging region R2 in the second event camera is obtained by the following formula: ; The event data generated in the imaging region R1 with a size of P×P pixels is obtained from the output data of the first event camera, and the event data generated in the imaging region R2 of the second event camera is obtained by corresponding to the above formula. Event data from the imaging regions of two event cameras are collected to obtain an event dataset including two resolutions.

[0027] Understandably, if the pixel size of the two event cameras is the same, then by adjusting the focal length of the adjustable lenses of the two event cameras to m and n respectively, and repeating the above step S2, event data with different resolutions can also be obtained.

[0028] This invention also provides an event dataset construction system, combined with Figure 1 and Figure 2 The system includes a semi-reflective mirror 1, multiple event cameras 2, and a control center 3. The semi-reflective mirror 1 projects the optical information of the target scene onto the multiple event cameras 2. The control center 3 controls the multiple event cameras 2 to simultaneously capture images of the target scene and establishes a coordinate space mapping relationship between one event camera 2 and other event cameras 2 based on the pixel size ratio between the multiple event cameras 2. Using the coordinate space mapping relationship, the imaging area of ​​one event camera 2 is determined to correspond to the imaging area of ​​other event cameras 2. Event data from the imaging areas of all event cameras 2 are collected to obtain an event dataset including different resolutions.

[0029] In some embodiments, each event camera 2 is equipped with an adjustable focus optical lens 3, and the control center 3 is connected to and controls each event camera 2 via a cable 5.

[0030] In this embodiment of the invention, two event cameras 2 are specifically used. Correspondingly, the control center 3 connects to and controls the two event cameras 2 to simultaneously capture the target scene via cable 5, and obtains the event data of the target area from the output data of one of the event cameras 2. Using the pixel size multiple between the two event cameras 2, the event data of the target area of ​​the other event camera 2 in the target scene is obtained.

[0031] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0032] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for constructing an event dataset, characterized in that, include: S1: Using a semi-reflective mirror, the optical information of the target scene is projected onto multiple event cameras with different pixel sizes; S2: Based on the pixel size multiples among multiple event cameras, establish the coordinate space mapping relationship between one event camera and other event cameras; S3: Using coordinate space mapping relationships, determine the imaging area of ​​one event camera corresponding to the imaging area of ​​other event cameras; collect event data in the imaging areas of all event cameras to obtain an event dataset including different resolutions.

2. The event dataset construction method according to claim 1, characterized in that, Each event camera in step S1 is equipped with a focusable optical lens.

3. The event dataset construction method according to claim 2, characterized in that, If multiple event cameras have the same pixel size, adjust the lens focal length of each event camera so that the resolution of the images acquired by the multiple event cameras is different.

4. An event dataset construction system, characterized in that, include: Multiple event cameras with different pixel sizes; A semi-reflective translucent mirror is used to project optical information of a target scene onto multiple event cameras; The control center controls multiple event cameras to simultaneously capture images of the target scene, and establishes a coordinate space mapping relationship between one event camera and other event cameras based on the pixel size multiples between the multiple event cameras. By using coordinate space mapping relationships, the imaging area of ​​one event camera is determined to correspond to the imaging area of ​​other event cameras; event data in the imaging areas of all event cameras are collected to obtain an event dataset including different resolutions.

5. The event dataset construction system according to claim 4, characterized in that, Each event camera is equipped with a focusable optical lens.

6. The event dataset construction system according to claim 4, characterized in that, The control center connects to and controls each event camera via cables.

Images