Event data coding method and device, electronic equipment and storage medium

Abstract

The invention discloses an event data coding method and device, electronic equipment and a storage medium, and the method comprises the steps: determining a first coding value of each pixel according to the event occurrence state of each pixel in an event pixel array; if the pixel is the target pixel of the event, determining a second coding value of the target pixel according to the event type of the event of the target pixel; and outputting target event data according to the first coded value of each pixel and the second coded value of the target pixel. According to the invention, event data with low data volume can be stably output.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically to an event data encoding method, apparatus, electronic device, and storage medium. Background Technology

[0002] Event cameras output event data based on changes in light intensity at each pixel. Currently, the encoding methods for outputting event data are frame mode and event mode. Frame mode has the advantage of lower backend decoding difficulty, but it often transmits a larger amount of data. Compared to frame mode, event mode significantly reduces the amount of data when there are few events, but its upper limit is not fixed. In some motion scenes where a large number of events occur, the amount of data can be very large. Therefore, in some motion scenes, the amount of data output based on event mode may even exceed the amount of data output based on frame mode in the same scene. This results in a large amount of encoded event data, making it difficult to stably output low-volume event data, which is detrimental to the storage and transmission of event data. Summary of the Invention

[0003] This invention provides an event data encoding method, apparatus, electronic device, and storage medium, which aim to stably output event data with low data volume.

[0004] In a first aspect, embodiments of the present invention provide an event data encoding method, the event data encoding method comprising:

[0005] Determine the first encoded value of each pixel based on the event occurrence state of each pixel in the event pixel array;

[0006] If the pixel is the target pixel where the event occurred, then the second encoding value of the target pixel is determined according to the event type of the event that occurred to the target pixel;

[0007] Based on the first encoded value of each pixel and the second encoded value of the target pixel, the target event data is output.

[0008] Secondly, embodiments of the present invention provide an event data encoding device, the event data encoding device comprising:

[0009] The first encoding module is used to determine the first encoding value of each pixel based on the event occurrence state of each pixel in the event pixel array;

[0010] The second encoding module is used to determine the second encoding value of the target pixel based on the event type of the event that occurred in the target pixel if the pixel is the target pixel in which the event occurred.

[0011] The output module is used to output target event data based on the first encoding value of each pixel and the second encoding value of the target pixel.

[0012] Thirdly, embodiments of the present invention provide an electronic device, which includes a processor and a memory. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of any of the event data encoding methods provided in the embodiments of the present invention.

[0013] Fourthly, embodiments of the present invention also provide a computer-readable storage medium, which includes a computer program. When the computer program is run on an electronic device, the computer program is used to cause the electronic device to perform the steps of any of the event data encoding methods provided in the embodiments of the present invention.

[0014] This invention determines a first encoded value for each pixel based on the event occurrence state of each pixel in the event pixel array; if the pixel is a target pixel where an event has occurred, a second encoded value for the target pixel is determined based on the event type of the event occurring at the target pixel; and target event data is output based on the first encoded values ​​of each pixel and the second encoded value of the target pixel. This significantly reduces the amount of target event data output compared to frame mode when the number of pixels experiencing events is small. Furthermore, in motion scenarios with a large number of pixels experiencing events, it ensures that the amount of target event data output will not exceed the amount of events encoded and output by frame mode, thus enabling stable output of low-volume event data, facilitating the transmission and storage of event data. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating one embodiment of the event data encoding method provided in this invention.

[0017] Figure 2 This is a flowchart illustrating another embodiment of the event data encoding method provided in this invention.

[0018] Figure 3 This is a schematic diagram of the event pixel array applied to the event data encoding method provided in this embodiment of the invention;

[0019] Figure 4This is a schematic diagram of the structure of the event data encoding device provided in the embodiment of the present invention;

[0020] Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Furthermore, in the description of the embodiments of the present invention, the terms "first," "second," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance. Thus, features defined with "first" or "second" may explicitly or implicitly include one or more features. In the description of the embodiments of the present invention, "multiple" means two or more, unless otherwise explicitly specified.

[0022] This invention provides an event data encoding method, apparatus, electronic device, and storage medium.

[0023] Specifically, this embodiment will be described from the perspective of an event data encoding device, which can be integrated into an electronic device. The electronic device can be an image sensor chip, an image sensor, a terminal device, etc. That is, the event data encoding method of this embodiment can be executed by an electronic device.

[0024] The following detailed description is provided in conjunction with the accompanying drawings. In this embodiment, an electronic device is used as the execution subject. It should be noted that the order of description in the following embodiments is not intended to limit the preferred order of the embodiments. Although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be performed in a different order than that shown in the accompanying drawings.

[0025] According to the background art description of this invention, currently, event cameras mainly output event data in frame mode and event mode. In frame mode, event data is output regardless of whether an event has occurred. Frame mode typically defines 2 bits of data to represent the state of a pixel. When the 2 bits are 00, it indicates no event has occurred; 01 indicates a positive event; and 10 indicates a negative event. Although frame mode is easy to decode at the back end, the amount of data it outputs is often large, which is not conducive to the storage and transmission of event data.

[0026] In other related solutions, event cameras can also output data in event mode. Event mode compresses the event data for each event point, and the decoding end needs to reconstruct the location information of the event point based on the compressed information. Compared to frame mode, event mode produces much less data when there are few events, but its backend decoding is more complex and computationally intensive. In some motion scenes that generate a large number of events, the amount of data generated is very large. Therefore, in some motion scenes, the amount of data output based on event mode may even exceed the amount of data output based on frame mode in the same scene, resulting in a large amount of encoded event data. Currently, event encoding methods cannot maintain the advantage of low data volume in both non-motion and motion scenes, and cannot stably output low-volume event data.

[0027] The amount of data output in event mode not only exceeds that of frame mode in motion scenarios, but its upper limit is also uncertain. This indeterminate upper limit increases the system's transmission bandwidth (such as MIPI bandwidth). Furthermore, because the upper limit of event mode data is not fixed, a very large amount of memory needs to be pre-allocated for storing and receiving data when debugging the driver on the backend SOC, resulting in extremely low memory utilization in most scenarios.

[0028] To address the above problems, this invention discloses an event data encoding method, please refer to... Figure 1 The specific process of this event data encoding method can be summarized in steps S10 to S30, where:

[0029] Step S10: Determine the first encoding value of each pixel based on the event occurrence state of each pixel in the event pixel array.

[0030] In this embodiment, the event pixel array is a device formed by arranging multiple pixel arrays. These pixels can be event pixels, and the event pixel array is generally set in an image sensor, such as an event camera or a fusion camera. Event data can be output based on the event state of each pixel in the event pixel array. The event state of each event pixel in the event pixel array can be determined through the event data. The event state can be divided into positive events (UP events), negative events (DOWN events), and no events (NO events). A positive event indicates that a light intensity enhancement event has occurred on the pixel, a negative event indicates that a light intensity decrease event has occurred on the pixel, and no event indicates that no light intensity change event has occurred on the pixel. Based on the event data, the recording of the event states of these pixels can be used for event imaging, thereby reconstructing the scene.

[0031] In this embodiment, since both positive and negative events indicate that an event has occurred on a pixel, while no event indicates that no event has occurred on a pixel, the event occurrence state can be distinguished for pixels in the event pixel array. The event occurrence state includes an event occurrence state and an event non-occurrence state. Specifically, the event occurrence state of event pixels with positive or negative event states is called an event occurrence state, indicating that a light intensity change event has occurred on that pixel, and this pixel can be referred to as an event-affected pixel. Conversely, the event occurrence state of pixels with no event states is called an event non-occurrence state, indicating that no light intensity change event has occurred on that pixel, and this pixel can be referred to as an event non-occurrence pixel.

[0032] In this embodiment, the event state of each pixel in the event pixel array can be obtained first. The event occurrence state of each pixel in the event pixel array can be determined based on the event state of each pixel, thereby distinguishing between event-affected pixels and event-free pixels in the event pixel array. The event occurrence states of these pixels are encoded separately to distinguish between event-affected pixels and event-free pixels, thereby determining the first encoding value of each pixel.

[0033] Step S20: If the pixel is the target pixel where the event occurred, then determine the second encoding value of the target pixel according to the event type of the event that occurred in the target pixel.

[0034] In this embodiment, if a pixel is a target pixel where an event has occurred, that is, if a pixel is a pixel where a light intensity change event has occurred, then it is determined to be a target pixel, i.e., an event-affected pixel. It can be understood that the event-affected state of a target pixel is the state where an event has occurred. If a pixel is a target pixel where no event has occurred, that is, if a pixel is a pixel where a light intensity change event has not occurred, then it is determined not to be a target pixel, i.e., an event-free pixel.

[0035] Optionally, the target pixel of the event is determined based on the event occurrence state of each pixel. If the event occurrence state is "event occurrence", the pixel can be determined as the target pixel of the event. If the event occurrence state is "no event occurrence", the pixel can be determined as not being the target pixel of the event.

[0036] Then, the event type of the event occurring at the target pixel is obtained. The event type represents the type of light intensity change event occurring at the target pixel. The event type can also be determined based on the event state of the target pixel. If the event state of the target pixel is a positive event, then the event type of the target pixel is a light intensity enhancement event, and the target pixel can be referred to as a positive event pixel. If the event state of the target pixel is a negative event, then the event type of the target pixel is a light intensity reduction event, and the target pixel can be referred to as a positive event pixel. It can be understood that the above-mentioned event pixels include both positive and negative event pixels.

[0037] The event types of the events occurring in these target pixels are encoded separately to distinguish between positive and negative event pixels, thereby determining the second encoded value of each pixel.

[0038] Step S30: Output target event data based on the first encoding value of each pixel and the second encoding value of the target pixel.

[0039] In this embodiment, each pixel in the event pixel array has a first encoding value, but may not necessarily have a second encoding value. Only the target pixel has both a first encoding value and a second encoding value. The target event data is output by summarizing the first encoding value of each pixel and the second pixel value of the target pixel.

[0040] For example, if both the first and second encoded values ​​are 1-bit encoded data (values), then for a target pixel, the corresponding data size is 2 bits, while for a pixel without an event, the corresponding data size is 1 bit. In frame mode, each pixel occupies 2 bits.

[0041] Therefore, in the technical solution disclosed in this embodiment, the first pixel value of the pixel is encoded by the event occurrence state of each pixel, and the second pixel value is encoded for the target pixel of the event. The resulting target event data will be lower than the encoding amount output by frame mode in the same scene. Moreover, the smaller the target pixel of the event, the more obvious this advantage is. Compared with frame mode, the event encoding method provided in this embodiment can further reduce the amount of data of the encoded output event data.

[0042] Furthermore, the event encoding method provided in this embodiment can output target event data in any scenario (including events) with a data volume that will not exceed the encoding volume of frame mode output in the same scenario. Compared with event mode, it can stably output event data with a low data volume.

[0043] Furthermore, the upper limit of the output data volume of the event encoding method provided in this embodiment is fixed. For example, when the first encoding value and the second encoding value are both defined as 1 bit, it will never exceed 2N bits, where N is the total number of pixels in the event pixel array. This will not lead to an increase in the upper limit of the system transmission bandwidth (such as MIPI bandwidth), and there is no need to pre-allocate a very large amount of memory for storing and receiving data when debugging the backend SOC driver, thus avoiding low memory utilization in most scenarios.

[0044] Optionally, step S10, "determining the first encoding value of each pixel based on the event occurrence state of each pixel in the event pixel array," may include:

[0045] If the event occurrence state is an event occurrence state, then the first preset value is set to the first encoding value of the pixel;

[0046] If the event occurrence state is "no event occurrence", then the second preset value is set to the first encoding value of the pixel.

[0047] In this embodiment, if the event occurrence state of a pixel is an event occurrence state, then the first preset value associated with the event occurrence state is set as the first encoding value of the pixel; if the event occurrence state of a pixel is an event non-occurrence state, then the second preset value associated with the event occurrence state is set as the first encoding value of the pixel.

[0048] The first preset value and the second preset value can be binary encoded values, such as "1" and "0". The first preset value and the second preset value are different, and the event occurrence state corresponding to each pixel can be distinguished based on the first encoded value of each pixel.

[0049] Optionally, step S20, "determining the first encoding value of each pixel based on the event occurrence state of each pixel in the event pixel array," may include:

[0050] If the event type is a light intensity enhancement event, then the third preset value is set to the second encoding value of the target pixel;

[0051] If the event type is a light intensity reduction event, then the fourth preset value is set as the second encoding value of the target pixel.

[0052] In this embodiment, if a pixel is the target pixel of an event, then the corresponding event type needs to be distinguished by encoding.

[0053] If the event type of the target pixel is a light intensity enhancement event, then the third preset value is set to the second encoding value of the target pixel; if the event type of the target pixel is a light intensity reduction event, then the fourth preset value is set to the second encoding value of the target pixel.

[0054] In some embodiments, if the state of the target pixel is a positive event state, the light intensity enhancement event of the target pixel can be determined, and the third preset value can be set as the second encoding value of the target pixel. If the state of the target pixel is a negative event state, the light intensity reduction event of the target pixel can be determined, and the fourth preset value can be set as the second encoding value of the target pixel.

[0055] The third and fourth preset values ​​can also be binary encoded values, such as "1" and "0". The third and fourth preset values ​​are different; the third preset value is associated with light intensity enhancement events, and the fourth preset value is associated with light intensity reduction events. The event occurrence state corresponding to each pixel can be distinguished based on the second encoded value of each pixel.

[0056] It should be noted that the sum of the data volume occupied by either the first or second preset value, and the data volume occupied by either the third or fourth preset value, must not exceed a preset data volume. In this embodiment, in order to reduce the data volume compared to frame mode, the preset data volume can be the data volume defined by frame mode for each pixel. For example, if frame mode defines the data volume for each pixel as 2 bits, then the sum of the above data volumes cannot exceed 2 bits.

[0057] Optionally, the third preset value can be the same as the first preset value or the second preset value, and the fourth preset value can be the same as the first preset value or the second preset value. However, the first preset value and the second preset value are different, and the third preset value and the fourth preset value are different.

[0058] For example, the first preset value is 1, the second preset value is 0, the third preset value is 1, and the fourth preset value is 0. Thus, when the first encoding value is 1, it indicates that there is an event pixel (target pixel); when the first encoding value is 0, it indicates that there is no event pixel; when the second encoding value is 1, it indicates that there is a positive event pixel; and when the second encoding value is 0, it indicates that there is a negative event pixel.

[0059] In this way, the first and second preset values ​​distinguish between the event occurrence and non-event states of a pixel, while the third and fourth preset values ​​distinguish between the event types of the target pixel in the event occurrence state. The event state of each pixel in the event pixel array can be reconstructed from the target event data, thereby improving the encoding and decoding efficiency of event data.

[0060] Optionally, refer to Figure 2 Based on any of the above embodiments, in another embodiment of the event encoding method of the present invention, step 30, "outputting target event data according to the first encoding value corresponding to each pixel and the second encoding value of the target pixel," may include:

[0061] Step 31: Based on the first preset encoding rule, summarize the first encoding values ​​corresponding to each pixel to obtain the first encoding data.

[0062] Step 32: Based on the second preset encoding rule, summarize the second encoding values ​​corresponding to the target pixels to obtain the second encoding data.

[0063] Step 33: Output the target event data based on the first encoded data and the second encoded data.

[0064] In this embodiment, the first preset encoding rule refers to a pre-defined arrangement order of pixels in an event pixel array. Based on this rule, the first encoded values ​​of each pixel can be arranged. During decoding, each first encoded value can be mapped to a pixel, thereby determining the location of pixels with events and the location of events without pixels. Similarly, the second preset encoding rule also refers to a pre-defined arrangement order of pixels in an event pixel array. Based on this rule, the second encoded values ​​of each target pixel can be arranged. During decoding, each second encoded value can be mapped to a target pixel, thereby determining the location of positive event pixels and the location of negative event pixels among the pixels with events.

[0065] Optionally, the first preset encoding rule and the second preset encoding rule can be the same or different. The first preset encoding rule and the second preset encoding rule can be set according to the row and column relationship or arrangement order of each pixel in the event pixel array, so as to more clearly correspond to the event pixel array.

[0066] The arrangement order of pixels can be determined according to the first preset encoding rule. The first encoded values ​​of each pixel are then arranged according to this arrangement order, and the results are summarized to obtain the first encoded data. The arrangement order of pixels can be determined according to the second preset encoding rule, which in turn determines the arrangement order of each target pixel. The second encoded values ​​of each target pixel are then arranged according to this arrangement order, and the results are summarized to obtain the second encoded data. Finally, the first encoded data and the second encoded data are combined to obtain the target event data.

[0067] For example, 1 is used to represent a pixel with an event (event occurred), and 0 is used to represent a pixel without an event (event occurred). For a 2*2 event pixel array, if the second pixel in the first row and the first pixel in the second row are event pixels, and the other pixels are non-event pixels, and they are arranged in order from left to right in each row, then the first encoded data is 0110 (or 01, 10), and the data size is 4 bits. It can be understood that, for the above scenario, if arranged in order from top to bottom in each column, the second encoded data is 0110 (or 01, 10).

[0068] For example, 1 is used to represent a positive event pixel (a pixel that experiences a light intensity enhancement event), and 0 is used to represent a negative event pixel (a pixel that experiences a light intensity reduction event). For the aforementioned 2x2 event pixel array, if the second pixel in the first row is a positive event pixel and the first pixel in the second row is a negative event pixel, arranged from left to right in each row, then the second encoded data is 10, and the data size is 2 bits. It can be understood that for the above scenario, if arranged from top to bottom in each column, the second encoded data is 01.

[0069] For example, for the above 2*2 event pixel array, the first encoded data is 0110 and the second encoded data is 10. Then the target event data is the combination of 0110 and 10, and its data size is 6 bits.

[0070] In the current frame mode, the event state of a pixel is typically represented by 2 bits of data: 00 indicates no event, 01 indicates an up event, and 10 indicates a down event. Each pixel requires a fixed 2 bits of data per pixel. For the same 2x2 event pixel array mentioned above, frame mode requires 8 bits of data to output the event pixels of that event pixel array.

[0071] In this embodiment, each pixel occupies 1-2 bits of data. For the same event pixel array, in the same scene under non-extreme conditions (with the same number of pixels experiencing events), the larger the event pixel array, the closer the data volume of the target event data is to 1 bit / pixel. The advantage of low data volume is more obvious compared to frame mode. In practical applications, in most scenarios, less than 10% of the pixels in the event pixel array experience events, meaning the data volume of the target event data is often very close to 1.1 bits / pixel. Compared to frame mode, this reduces bandwidth by nearly half, making it more conducive to data transmission and storage.

[0072] Understandably, when the backend SOC decodes, it only needs to combine the received target event data with the first preset rule, the second preset rule, and the specific pixels represented by the first, second, third, and fourth encoding values ​​to quickly and efficiently decode the event data and obtain the event output by each pixel in the event pixel array.

[0073] Specifically, after acquiring the event data to be decoded, a first data to be decoded and a second data to be decoded are determined from the event data. The first data to be decoded represents the event occurrence state corresponding to each pixel. Then, according to a first preset encoding rule, the pixels to be decoded corresponding to each value to be decoded in the first data to be decoded are determined sequentially. Based on the value to be decoded in the first data to be decoded corresponding to the pixel, the event occurrence state of the pixel to be decoded is determined. For pixels with an event occurrence state of "event occurrence," according to a second preset encoding rule, the pixels with event occurrence corresponding to each value to be decoded in the second data to be decoded are determined sequentially. Based on the value to be decoded in the second data to be decoded corresponding to the pixel with event occurrence, the event type of the pixel with event occurrence is determined, thereby determining the event type of the light intensity change event occurring on this pixel with event occurrence, thus completing the decoding.

[0074] For better understanding, refer to Figure 3 This diagram illustrates an event pixel array for an application scenario. The array is an 8x8 pixel array, where red cells represent positive events, green cells represent negative events, and blank cells represent no events. Eight pixels generate events. In the first encoded data, 1 represents a pixel with an event, and 0 represents a pixel without an event; in the second encoded data, 1 represents a positive event pixel, and 0 represents a negative event pixel. Both the first and second preset rules are arranged from left to right in each row.

[0075] Therefore, the first encoded data is 00001000, 00010000, 00100000, 00010000, 0000100, 00000100, 00000010, 00000010, 00000001, which can also be represented as 0x08, 0x10, 0x20, 0x10, 0x08, 0x04, 0x02, 0x01. The second encoded data is 11110000, which can also be represented as 0xf0. The target event data is a combination of the first and second encoded data. The first encoded data has a data size of 64 bits, the second encoded data has a data size of 8 bits, and the target event data has a data size of 72 bits, or 1.125 bits / pixel.

[0076] Optionally, step S30, "outputting target event data based on the first encoding value corresponding to each pixel and the second encoding value of the target pixel," may include:

[0077] If the pixel is the target pixel, then the target encoded data of the pixel is obtained based on the first encoded value and the second encoded value corresponding to the pixel;

[0078] If the pixel is not the target pixel, then the target encoded data of the pixel is obtained based on the first encoded value of the pixel;

[0079] The target event data is output based on the target encoding data of each pixel.

[0080] In this embodiment, target event data can be determined on a per-pixel basis. Specifically, if a pixel is a target pixel, its first and second encoding values ​​are combined in a fixed order to obtain the target encoded data for that pixel, with the first encoding value first and the second encoding value last. If a pixel is not a target pixel, its first encoding value is used as the target encoded data. By summarizing the target encoded data for each pixel according to a third preset encoding rule, target event data can be output. The third preset encoding rule refers to a pre-defined arrangement order of pixels in an event pixel array.

[0081] During decoding, each encoded value in the target event data needs to be read one by one. For the read target encoded value (the target encoded value is the first encoded value of the target event data during initial reading), the event occurrence state of the pixel to be decoded corresponding to the target encoded value is determined based on the target encoded value (the pixel to be decoded is the first pixel in the event pixel array determined based on the third preset encoding rule during initial reading). It can be understood that during initial reading, the target pixel value (the first encoded value) represents the event occurrence state corresponding to the first pixel.

[0082] If the event occurrence state of the pixel to be decoded is no event occurrence state, that is, the pixel to be decoded is a pixel without event, then the next encoding value corresponding to the target encoding value in the target event data is updated to the new target encoding value, and the next pixel corresponding to the pixel to be decoded in the third preset encoding rule is updated to the new pixel to be decoded, and the process returns to the step of "determining the event occurrence state of the pixel to be decoded based on the target encoding value for the read target encoding value".

[0083] If the event occurrence state of the pixel to be decoded is no event occurrence state, that is, the pixel to be decoded is a pixel without event, then the next encoding value corresponding to the target encoding value in the target event data is updated to the new target encoding value, and the next pixel corresponding to the pixel to be decoded in the third preset encoding rule is updated to the new pixel to be decoded, and the process returns to the step of "determining the event occurrence state of the pixel to be decoded based on the target encoding value for the read target encoding value".

[0084] If the event occurrence state of the pixel to be decoded is an event occurrence state, that is, the pixel to be decoded is an event pixel, then based on the next encoded value corresponding to the target encoded value in the target event data, the event type of the pixel to be decoded is determined, it is determined whether the event pixel is a positive event pixel or a negative event pixel, and the next encoded value corresponding to the target encoded value (the encoded value after the next encoded value) is updated to the new target encoded value, and the next pixel corresponding to the pixel to be decoded in the third preset encoding rule is updated to the new pixel to be decoded, and the process returns to the step of "for the read target encoded value, determine the event occurrence state of the pixel to be decoded corresponding to the target encoded value based on the target encoded value".

[0085] To better understand, we use 1 to represent a pixel with an event and 0 to represent a pixel without an event. 1 represents a positive event pixel and 0 represents a negative event pixel. For a 2x2 event pixel array, if the second pixel in the first row is a positive event pixel, the first pixel in the second row is a negative event pixel, and the other pixels are pixels without events, then the target code data for the first pixel in the first row can be "0", the target code data for the second pixel in the first row can be "11", the target code data for the first pixel in the second row can be "10", and the target code data for the first pixel in the first row can be "0". Arranging these in left-to-right order within each row, we obtain the target event data "011100".

[0086] When decoding "011100", the first encoded value 0 is read to determine that the first pixel is a pixel with no event. The second encoded value is read to determine that the second pixel is a pixel with an event. The third encoded value is read to determine that the second pixel is a positive event pixel. The fourth encoded value is read to determine that the third pixel is a pixel with an event. The fifth encoded value is read to determine that the third pixel is a negative event pixel. The sixth encoded value is read to determine that the fourth pixel is a pixel with no event, thus completing the decoding.

[0087] Optionally, the method further includes:

[0088] If the event pixel array contains the target pixel, then the process of outputting target event data based on the first encoding value of each pixel and the second encoding value of the target pixel is performed.

[0089] If the event pixel array does not contain the target pixel, then the target event data is output according to the first encoding value of each pixel.

[0090] In this embodiment, the event pixel array may contain both event pixels (i.e., target pixels) and pixels without events, or contain only event pixels, or contain only pixels without events. Determining whether the event pixel array contains a target pixel can be based on the event occurrence state of each pixel. If all pixels in the event pixel array are in a no-event state, it can be determined that the event pixel array does not contain a target pixel; otherwise, it can be determined that it contains a target pixel. This determination can be performed before or simultaneously with step S20. It can also be determined in other ways, such as if the current shooting scene is a static scene, indicating that the event pixel array does not contain a target pixel.

[0091] If the event pixel array contains a target pixel, then target event data needs to be output based on the first encoded value of each pixel and the second encoded value of the target pixel. If the event pixel array does not contain a target pixel, then the step of determining the second encoded value of the target pixel based on the event type of the event occurring at the target pixel is unnecessary. Only the first encoded value of each pixel needs to be output. During decoding, it can be directly determined from the target event data that all pixels in the event pixel array are event-free pixels, thus reducing execution steps and improving encoding efficiency.

[0092] Optionally, the method further includes:

[0093] If the number of target pixels in the event pixel array is greater than a preset threshold, then the second encoding value of the target pixel is determined based on the event type of the event that occurred to the target pixel.

[0094] If the number of target pixels in the event pixel array is less than or equal to a preset threshold, then target event data is output according to the first encoding value of each pixel.

[0095] In this embodiment, based on the above, it is known that when the number of events in the event pixel array is larger and there are more event pixels, the first pixel value of the pixel is encoded by the event occurrence state of each pixel, and then the second pixel value is encoded for the target pixel where the event occurred. The resulting encoding method for the target event data will result in a data volume that is closer to or lower than the data volume output by frame mode in the same scene.

[0096] Based on this, in this embodiment, a preset threshold corresponding to the number of event pixels can be set to distinguish between high event volume scenarios and low event volume scenarios. For high event volume scenarios where the number of target pixels in the event pixel array that have occurred events is greater than or equal to the preset threshold, the traditional frame mode can be used for encoding. That is, the target encoding value of each pixel is determined according to the event state of each pixel in the event pixel array, and the target event data is output according to the target encoding value of each pixel. If the number of target pixels in the event pixel array is less than the preset threshold, then step S10 is executed, which encodes the first pixel value of the pixel based on the event occurrence state of each pixel, and then encodes the second pixel value for the target pixel that has occurred, thus obtaining the encoding method of the target event data.

[0097] This allows for the encoding of individual pixels and the synchronous output of target event data in high-event scenarios, thereby improving the independence between the event data corresponding to each pixel. In the event of transmission interruption, it can ensure the accurate output of some event data. In low-event scenarios, where the amount of data is small and encoding errors and transmission interruptions are less likely to occur, the amount of data can be further reduced to transmit and store event data more efficiently.

[0098] This embodiment also provides an event data encoding device, which can be specifically integrated into an electronic device. For example... Figure 4 As shown, the event data encoding device may include:

[0099] The first encoding module 1001 is used to determine the first encoding value of each pixel based on the event occurrence state of each pixel in the event pixel array;

[0100] The second encoding module 1002 is used to determine the second encoding value of the target pixel according to the event type of the event that occurred in the target pixel if the pixel is the target pixel in which the event occurred.

[0101] The output module 1003 is used to output target event data based on the first encoding value of each pixel and the second encoding value of the target pixel.

[0102] Optionally, the output module 1003 is also used for:

[0103] Based on the first preset encoding rule, the first encoding values ​​corresponding to each pixel are summarized to obtain the first encoding data;

[0104] Based on the second preset encoding rule, the second encoding values ​​corresponding to the target pixels are summarized to obtain the second encoding data;

[0105] The target event data is output based on the first encoded data and the second encoded data.

[0106] Optionally, the output module 1003 is also used for:

[0107] If the pixel is the target pixel, then the target encoded data of the pixel is obtained based on the first encoded value and the second encoded value corresponding to the pixel;

[0108] If the pixel is not the target pixel, then the target encoded data of the pixel is obtained based on the first encoded value of the pixel;

[0109] The target event data is output based on the target encoding data of each pixel.

[0110] Optionally, the output module 1003 is also used for:

[0111] If the event pixel array contains the target pixel, then the process of outputting target event data based on the first encoding value of each pixel and the second encoding value of the target pixel is performed.

[0112] If the event pixel array does not contain the target pixel, then the target event data is output according to the first encoding value of each pixel.

[0113] Optionally, the first encoding module 1001 is also used for:

[0114] If the number of target pixels in the event pixel array is greater than or equal to a preset threshold, then the target encoding value of each pixel is determined according to the event state of each pixel in the event pixel array, and the target event data is output according to the target encoding value of each pixel.

[0115] If the number of target pixels in the event pixel array is less than a preset threshold, then the process of determining the first encoding value of each pixel based on the event occurrence state of each pixel in the event pixel array is executed.

[0116] Optionally, the first encoding module 1001 is also used for:

[0117] If the event occurrence state is an event occurrence state, then the first preset value is set to the first encoding value of the pixel;

[0118] If the event occurrence state is "no event occurrence", then the second preset value is set to the first encoding value of the pixel.

[0119] Optionally, the second encoding module 1002 is also used for:

[0120] If the event type is a light intensity enhancement event, then the third preset value is set to the second encoding value of the target pixel;

[0121] If the event type is a light intensity reduction event, then the fourth preset value is set as the second encoding value of the target pixel.

[0122] This embodiment determines the first encoding value of each pixel based on the event occurrence state of each pixel in the event pixel array; if the pixel is the target pixel of the event, then the second encoding value of the target pixel is determined according to the event type of the event occurring at the target pixel; and the target event data is output based on the first encoding value of each pixel and the second encoding value of the target pixel. This significantly reduces the amount of target event data output compared to frame mode when the number of pixels experiencing events is small. Furthermore, in motion scenarios with a large number of pixels experiencing events, it ensures that the amount of target event data output will not exceed the amount of events encoded and output by frame mode, thus enabling stable output of low-volume event data, facilitating the transmission and storage of event data.

[0123] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0124] like Figure 5 As shown, Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. The electronic device 1100 includes a processor 1101 with one or more processing cores, a memory 1102 with one or more computer-readable storage media, and a computer program stored on the memory 1102 and executable on the processor. The processor 1101 and the memory 1102 are electrically connected. Those skilled in the art will understand that the electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0125] The processor 1101 is the control center of the electronic device 1100. It connects various parts of the electronic device 1100 via various interfaces and lines. By running or loading software programs and / or units stored in the memory 1102, and by calling data stored in the memory 1102, it executes various functions of the electronic device 1100 and processes data, thereby providing overall monitoring of the electronic device 1100. The processor 1101 can be a CPU, GPU, network processor (NP), etc., and can implement or execute the methods, steps, and logic diagrams disclosed in the embodiments of this invention.

[0126] In this embodiment of the invention, the processor 1101 in the electronic device 1100 loads the instructions corresponding to the processes of one or more applications into the memory 1102 according to the following steps, and the processor 1101 runs the applications stored in the memory 1102 to realize various functions, such as:

[0127] Determine the first encoded value of each pixel based on the event occurrence state of each pixel in the event pixel array;

[0128] If the pixel is the target pixel where the event occurred, then the second encoding value of the target pixel is determined according to the event type of the event that occurred to the target pixel;

[0129] Based on the first encoded value of each pixel and the second encoded value of the target pixel, the target event data is output.

[0130] In this embodiment, a first encoding value is determined for each pixel based on its event occurrence state in the event pixel array. If the pixel is a target pixel where an event has occurred, a second encoding value is determined for the target pixel based on the event type of the event occurring in the target pixel. Target event data is output based on the first encoding values ​​of each pixel and the second encoding value of the target pixel. This significantly reduces the amount of target event data output compared to frame mode when the number of pixels experiencing events is small. Furthermore, in motion scenarios with a large number of pixels experiencing events, it ensures that the amount of target event data output will not exceed the amount of events encoded and output by frame mode. This allows for stable output of low-volume event data, facilitating the transmission and storage of event data.

[0131] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0132] Optional, such as Figure 5 As shown, the electronic device 1100 also includes: a touch display screen 1103, a radio frequency circuit 1104, an audio circuit 1105, an input unit 1106, and a power supply 1107. The processor 1101 is electrically connected to the touch display screen 1103, the radio frequency circuit 1104, the audio circuit 1105, the input unit 1106, and the power supply 1107. Those skilled in the art will understand that... Figure 5 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0133] The touch display screen 1103 can be used to display a graphical user interface (GUI) and receive operation commands generated by the user interacting with the GUI. The touch display screen 1103 may include a display panel and a touch panel. The display panel can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the electronic device. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. Optionally, the display panel can be configured using a liquid crystal display (LCD), organic light-emitting diode (OLED), or other similar technologies. The touch panel can be used to collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel), generate corresponding operation commands, and execute the corresponding program according to the operation commands. Optionally, the touch panel may include a touch detection device and a touch controller. The touch detection device detects the user's touch location and the signal generated by the touch operation, transmitting the signal to the touch controller. The touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 1101. It can also receive and execute commands from the processor 1101. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it transmits the information to the processor 1101 to determine the type of touch event. Subsequently, the processor 1101 provides corresponding visual output on the display panel based on the type of touch event. In this embodiment, the touch panel and the display panel can be integrated into the touch display screen 1103 to achieve input and output functions. However, in some embodiments, the touch panel and the touch display screen 1103 can be used as two independent components to achieve input and output functions. That is, the touch display screen 1103 can also be used as part of the input unit 1106 to achieve input functions.

[0134] The radio frequency circuit 1104 can be used to transmit and receive radio frequency signals to establish wireless communication with network devices or other electronic devices, and to transmit and receive signals with network devices or other electronic devices.

[0135] Audio circuit 1105 can be used to provide an audio interface between a user and an electronic device via a speaker and a microphone. Audio circuit 1105 can convert received audio data into electrical signals and transmit them to the speaker, where the speaker converts them into sound signals for output. Conversely, the microphone converts collected sound signals into electrical signals, which are then received by audio circuit 1105, converted back into audio data, and then processed by processor 1101 before being transmitted via radio frequency circuit 1104 to, for example, another electronic device, or output to memory 1102 for further processing. Audio circuit 1105 may also include an earphone jack to provide communication between peripheral headphones and electronic devices.

[0136] The input unit 1106 can be used to receive input numbers, characters, or user characteristic information (such as fingerprints, iris, facial information, etc.), and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control.

[0137] Power supply 1107 is used to supply power to various components of electronic device 1100. Optionally, power supply 1107 can be logically connected to processor 1101 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. Power supply 1107 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0138] although Figure 5 As not shown in the diagram, the electronic device 1100 may also include a camera, sensor, wireless fidelity module, Bluetooth module, etc., which will not be described in detail here.

[0139] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0140] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0141] To this end, embodiments of the present invention provide a computer-readable storage medium storing a plurality of computer programs, which can be loaded by a processor to execute any of the event data encoding methods provided in the embodiments of the present invention. The computer program can execute the following steps of the event data encoding method:

[0142] Determine the first encoded value of each pixel based on the event occurrence state of each pixel in the event pixel array;

[0143] If the pixel is the target pixel where the event occurred, then the second encoding value of the target pixel is determined according to the event type of the event that occurred to the target pixel;

[0144] Based on the first encoded value of each pixel and the second encoded value of the target pixel, the target event data is output.

[0145] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0146] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0147] Since the computer program stored in the computer-readable storage medium can execute any of the event data encoding methods provided in the embodiments of the present invention, the beneficial effects that any of the event data encoding methods provided in the embodiments of the present invention can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0148] In the above embodiments of the event data encoding device, computer-readable storage medium, electronic device, and computer program product, the descriptions of each embodiment have different focuses. Parts not described in detail in a particular embodiment can be referred to in the relevant descriptions of other embodiments. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes and beneficial effects of the event data encoding device, computer-readable storage medium, computer program product, electronic device, and their corresponding units described above can be referred to the description of the event data encoding method in the above embodiments, and will not be repeated here.

[0149] The foregoing has provided a detailed description of an event data encoding method, event data encoding device, electronic device, computer-readable storage medium, and computer program product provided by embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. An event data encoding method, characterized in that, The event data encoding method includes: Determine the first encoded value of each pixel based on the event occurrence state of each pixel in the event pixel array; If the pixel is the target pixel where the event occurred, then the second encoding value of the target pixel is determined according to the event type of the event that occurred to the target pixel; Based on the first encoded value of each pixel and the second encoded value of the target pixel, the target event data is output.

2. The event data encoding method as described in claim 1, characterized in that, The step of outputting target event data based on the first encoding value of each pixel and the second encoding value of the target pixel includes: Based on the first preset encoding rule, the first encoding values ​​corresponding to each pixel are summarized to obtain the first encoding data; Based on the second preset encoding rule, the second encoding values ​​corresponding to the target pixels are summarized to obtain the second encoding data; The target event data is output based on the first encoded data and the second encoded data.

3. The event data encoding method as described in claim 1, characterized in that, The step of outputting target event data based on the first encoding value of each pixel and the second encoding value of the target pixel includes: If the pixel is the target pixel, then the target encoded data of the pixel is obtained based on the first encoded value and the second encoded value corresponding to the pixel; If the pixel is not the target pixel, then the target encoded data of the pixel is obtained based on the first encoded value of the pixel; The target event data is output based on the target encoding data of each pixel.

4. The event data encoding method as described in claim 1, characterized in that, After determining the first encoded value of each pixel based on the event occurrence state of each pixel in the event pixel array, the method further includes: If the event pixel array contains the target pixel, then the process of outputting target event data based on the first encoding value of each pixel and the second encoding value of the target pixel is performed. If the event pixel array does not contain the target pixel, then the target event data is output according to the first encoding value of each pixel.

5. The event data encoding method as described in claim 1, characterized in that, The method further includes: If the number of target pixels in the event pixel array is greater than or equal to a preset threshold, then the target encoding value of each pixel is determined according to the event state of each pixel in the event pixel array, and the target event data is output according to the target encoding value of each pixel. If the number of target pixels in the event pixel array is less than a preset threshold, then the process of determining the first encoding value of each pixel based on the event occurrence state of each pixel in the event pixel array is executed.

6. The event data encoding method as described in claim 1, characterized in that, The step of determining the first encoded value of each pixel based on the event occurrence state of each pixel in the event pixel array includes: If the event occurrence state is an event occurrence state, then the first preset value is set to the first encoding value of the pixel; If the event occurrence state is "no event occurrence", then the second preset value is set to the first encoding value of the pixel.

7. The event data encoding method as described in claim 1, characterized in that, Determining the second encoded value of the target pixel based on the event type of the event occurring in the target pixel includes: If the event type is a light intensity enhancement event, then the third preset value is set to the second encoding value of the target pixel; If the event type is a light intensity reduction event, then the fourth preset value is set as the second encoding value of the target pixel.

8. An event data encoding device, characterized in that, The event data encoding device includes: The first encoding module is used to determine the first encoding value of each pixel based on the event occurrence state of each pixel in the event pixel array; The second encoding module is used to determine the second encoding value of the target pixel based on the event type of the event that occurred in the target pixel if the pixel is the target pixel in which the event occurred. The output module is used to output target event data based on the first encoding value of each pixel and the second encoding value of the target pixel.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of any of the event data encoding methods of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a computer program that, when run on an electronic device, causes the electronic device to perform the steps of any of the event data encoding methods of claims 1-7.

Images