A data encoding method, apparatus, device, and readable storage medium

Abstract

The present application provides a data encoding method, apparatus, device, and readable storage medium. The method includes: generating corresponding real-time voltages according to the incident light intensities of the pixels in the pixel array of the image sensor; respectively obtaining the light intensity change characterization data corresponding to each pixel based on the real-time voltages; sequentially encoding the light intensity change characterization data and the total number of pixels of each sub-pixel array in which the light intensity change characterization data of multiple internal pixels in the pixel array are the same, to obtain encoded data. Since the events output by adjacent pixels in the pixel array usually do not differ much, through the implementation of the solution of the present application, the continuous pixels with the same event generation behavior are combined and encoded, which can make the data compact, efficient, and small in data volume, improve the image output frame rate, and reduce the complexity of subsequent decoding.

Description

Technical Field

[0001] The present application relates to the technical field of data processing, and particularly relates to a data encoding method, apparatus, device, and readable storage medium. Background Art

[0002] With the continuous development of science and technology, computer vision technology has become increasingly mature. The emergence of event cameras has attracted more and more attention in the field of vision. It simulates the human retina and responds to pixel point pulses of brightness changes caused by motion. Therefore, it can capture the brightness changes of a scene at an extremely high frame rate, record events at specific time points and specific positions in an image, and form an event stream instead of a frame stream, thus solving problems such as information redundancy, large amount of data storage, and real-time processing of traditional cameras.

[0003] In related technologies, after an event camera's analog component completes event generation, the digital circuit scans sequentially according to the pixel array layout under the control of the main clock to determine whether there are events generated for each pixel in each row and each column position. If there is an event, the event signal is output from the output interface. Since each pixel of a traditional output event outputs e=(timestamp, x, y, p), including a timestamp (timestamp), pixel coordinates (x, y), and event polarity p, the characteristic of an event camera is to output the changing contour during the movement of an object, and the proportion of pixels with events is relatively small. If pixel encoding is performed separately on all pixels of the pixel array, it will result in slow event data reading speed, high power consumption, and low image output frame rate. Summary of the Invention

[0004] Embodiments of the present application provide a data encoding method, apparatus, device, and readable storage medium, which can at least solve the problems of slow event data reading speed, high power consumption, and low image output frame rate caused by separately performing pixel encoding on all pixels of the pixel array in related technologies.

[0005] In a first aspect of the embodiments of the present application, a data encoding method is provided, which is applied to an event camera and includes:

[0006] Generating corresponding real-time voltages according to the incident light intensities of the pixels in the pixel array of the image sensor;

[0007] Based on the real-time voltages, respectively obtaining light intensity change characterization data corresponding to each of the pixels;

[0008] Sequentially encoding the light intensity change characterization data and the total number of pixels of each of a plurality of sub-pixel arrays in the pixel array to obtain encoded data; wherein, the total number of pixels in the sub-pixel array is greater than or equal to 1, and the light intensity change characterization data of all pixels in each of the sub-pixel arrays is the same.

[0009] In a second aspect of the embodiments of the present application, a data encoding device is provided, which is applied to an event camera and includes:

[0010] A generating module, configured to generate corresponding real-time voltages according to the incident light intensities of the pixels in the pixel array of the image sensor;

[0011] An obtaining module, configured to respectively obtain the light intensity change characterization data corresponding to each of the pixels based on the real-time voltages;

[0012] An encoding module, configured to sequentially encode the light intensity change characterization data of each of the multiple sub-pixel arrays in the pixel array and the total number of pixels to obtain encoded data; wherein, the total number of pixels in the sub-pixel array is greater than or equal to 1, and the light intensity change characterization data of all the pixels in each of the sub-pixel arrays is the same.

[0013] In a third aspect of the embodiments of the present application, a terminal device is provided, including: a memory and a processor, wherein the processor is configured to execute a computer program stored on the memory, and when the processor executes the computer program, implement the steps in the data encoding method provided in the first aspect of the embodiments of the present application.

[0014] In a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, implement the steps in the data encoding method provided in the first aspect of the embodiments of the present application.

[0015] As can be seen from the above, according to the data encoding method, device, equipment and readable storage medium provided by the solution of the present application, corresponding real-time voltages are generated according to the incident light intensities of the pixels in the pixel array of the image sensor; the light intensity change characterization data corresponding to each pixel is respectively obtained based on the real-time voltages; the light intensity change characterization data of each of the multiple sub-pixel arrays in which the light intensity change characterization data of all the pixels inside is the same and the total number of pixels in the pixel array are sequentially encoded to obtain encoded data. Since the events output by adjacent pixels in the pixel array usually do not differ much, through the implementation of the solution of the present application, continuous pixels with the same event generation behavior are merged and encoded, which can make the data compact, efficient, with a small data volume, improve the image output frame rate, and reduce the complexity of subsequent decoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of the basic process of a data encoding method provided in the first embodiment of the present application;

[0017] Figure 2 It is a schematic diagram of a pixel array composed of multiple pixels provided in the first embodiment of the present application;

[0018] Figure 3Schematic diagram of the refined process of a data encoding method provided in the second embodiment of this application;

[0019] Figure 4 Schematic diagram of the program modules of a data encoding device provided in the third embodiment of this application;

[0020] Figure 5 Schematic diagram of the structure of a terminal device provided in the fourth embodiment of this application. Detailed implementation manners

[0021] To make the objectives, features, and advantages of this application more obvious and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application.

[0022] In the description of the embodiments of this application, it should be understood that the orientation or positional relationship indicated by terms such as "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the embodiments of this application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation of the present invention.

[0023] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of this application, "a plurality" means two or more, unless otherwise specifically defined.

[0024] In the embodiments of this application, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed", etc. should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or integrated; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the internal connection of two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments of this application can be understood according to specific situations.

[0025] The above are only the preferred embodiments of the present application, and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0026] To solve the problems of slow event data reading speed, high power consumption, and low image output frame rate caused by the pixel encoding of all pixels in the pixel array by the event camera in the related art, the first embodiment of the present application provides a data encoding method, which is applied to an event camera. An event camera is a new type of image sensor, and each pixel of it works independently. Only when the brightness change of a certain pixel reaches a certain threshold, an event is output.

[0027] As Figure 1 is a schematic diagram of the basic process of the data encoding method provided in this embodiment. The data encoding method includes the following steps:

[0028] Step 101: Generate corresponding real-time voltages according to the incident light intensities of the pixels in the pixel array of the image sensor.

[0029] Specifically, the event camera is provided with an image sensor, and the image sensor includes a pixel array composed of multiple pixels. Each pixel sensor in the pixel array is an integrated circuit. In this integrated circuit, a photodiode can be integrated with a capacitor for accumulating charge, and a photocurrent is generated by the photodiode in response to the incident light intensity, and then a real-time voltage is generated accordingly according to the photocurrent.

[0030] Step 102: Obtain the light intensity change characterization data corresponding to each pixel based on the real-time voltage.

[0031] In an implementation manner of this embodiment, the light intensity change characterization data is a binary vector. Thus, after subtracting the preset reference voltage from each real-time voltage, each voltage difference is respectively compared with the preset first voltage threshold and the preset second voltage threshold; then, the binary vectors of each pixel are generated correspondingly according to the comparison results.

[0032] Specifically, in this embodiment, each real-time voltage is respectively subtracted from the reference voltage to obtain a voltage difference, and then based on this voltage difference, it is judged whether the incident light intensity changes (becomes stronger or weaker). In practical applications, a first comparison circuit and a second comparison circuit can be respectively configured. The first comparison circuit is used to detect whether the light intensity becomes stronger, and the second comparison circuit is used to detect whether the light intensity becomes weaker. The two comparison circuits respectively correspond to different voltage thresholds. Among them, the first voltage threshold is greater than 0, and the second voltage threshold is less than 0. When the above voltage difference is greater than the first voltage threshold, the first comparison circuit detects that the light intensity becomes stronger. When the above voltage difference is less than the second voltage threshold, the second comparison circuit detects that the light intensity becomes weaker.

[0033] It should be noted that the reference voltage can be a preset fixed voltage or a voltage that can be flexibly updated later. Additionally, in the preferred implementation of this embodiment, the first voltage threshold and the second voltage threshold can be opposite to each other. For example, if the value of the first voltage threshold is 0.1V, then the value of the second voltage threshold is -0.1V.

[0034] It should also be understood that in practical applications, each pixel will generate a corresponding 2-bit vector, which is used to represent whether the optical signal becomes stronger or weaker. When the voltage difference is less than the first voltage threshold, the value of the first number in the binary vector is 0. When the voltage difference is greater than the second voltage threshold, the value of the second number in the binary vector is 0. In this embodiment, the binary vector can be represented in the form of a pair of numbers [a, b]. Among them, the value of the number a is determined by the comparison relationship between the voltage difference and the first voltage threshold, and the value of the number b is determined by the comparison relationship between the voltage difference and the second voltage threshold. When the voltage difference is greater than the first voltage threshold, the value of a is a non-zero number. When the voltage difference is less than the second voltage threshold, the value of b is a non-zero number. In the preferred implementation of this embodiment, the non-zero values of a and b can both be 1. Of course, in some other embodiments, they can also be 1 and -1 respectively.

[0035] Next, this embodiment gives an example of the generation method of the binary vector. Assume that the reference voltage Vm is 0.5V, and the first voltage threshold A1 and the second voltage threshold A2 are 0.1V and -0.1V respectively. When the real-time voltage Vin is 0.7V, the voltage difference Vin - Vm is greater than the first voltage threshold A1. At this time, the UP event is triggered, and the vector value 1 is output. And the voltage difference Vin - Vm is greater than the second voltage threshold A2, and the DN event is not triggered, and the vector value 0 is output. Then, the final 2-bit vector output according to the comparison result is [1, 0]; when the real-time voltage Vin is 0.5V, the voltage difference Vin - Vm is less than the first voltage threshold A1. At this time, the UP event is not triggered, and the vector value 0 is output. And the voltage difference Vin - Vm is greater than the second voltage threshold A2, and the DN event is not triggered, and the vector value 0 is output. Then, the final 2-bit vector output according to the comparison result is [0, 0]; when the real-time voltage Vin is 0.3V, the voltage difference Vin - Vm is less than the first voltage threshold A1. At this time, the UP event is not triggered, and the vector value 0 is output. And the voltage difference Vin - Vm is less than the second voltage threshold A2, and the DN event is triggered, and the vector value 1 is output. Then, the final 2-bit vector output according to the comparison result is [0, 1]. It should be understood that when the 2-bit vector is the all-zero pair [0, 0], it represents that the optical signal has not changed. And when the 2-bit vector is a non-all-zero pair [1, 0] or [0, 1], it represents that the optical signal becomes stronger or weaker. Of course, in actual applications, it cannot be excluded that the system may make mistakes and output the 2-bit vector [1, 1]. This kind of information is usually recognized as an invalid event.

[0036] Step 103: Encode the light intensity change characterization data and the total number of pixels of each of the multiple sub-pixel arrays in the pixel array in sequence to obtain encoded data.

[0037] In this embodiment, the encoded data is used to generate an event image after being read out by the readout circuit of the event camera. It should be noted that the total number of pixels in the sub-pixel array of this embodiment is greater than or equal to 1, and the light intensity change characterization data of all pixels in each sub-pixel array is the same. Such as Figure 2The figure shows a schematic diagram of a pixel array provided by this embodiment. The size of the pixel array represents the resolution of the event camera. For the convenience of description, only a 16×8 matrix is shown in this embodiment. In actual applications, the real pixel array is much higher than 16×8, such as 480x640, etc. In the figure, the value "+1" represents the UP event corresponding to the binary vector [1, 0], the value "-1" represents the DN event corresponding to the binary vector [0, 1], and the value "0" represents no event. In addition, the arrays 201, 202, and 203 in the figure all represent sub-pixel arrays of the pixel array 20. In actual applications, the specific form of the sub-pixel array can be determined according to the actual situation, and this embodiment does not make a unique limitation on this. It should be understood that in this embodiment, it is possible to pre-divide the pixel array to obtain multiple sub-pixel arrays, and then scan and encode each sub-pixel array separately. It is also possible to combine consecutive pixels with the same light intensity change characterization data obtained during the real-time scanning process into a sub-pixel array, and then encode this sub-pixel array.

[0038] Further, this embodiment takes Figure 2 the binary vectors output by each pixel shown in the first row in as an example for description. It sequentially outputs 4 UP events, 8 no events, and 4 DN events. When encoding the sub-pixel arrays 201, 202, and 203 in sequence, the organization form of the attribute pair "light intensity change characterization data + total number of pixels" is used for encoding. The sub-pixel array 201 can be encoded as 4[1, 0]. Among them, "[1, 0]" represents the same binary vector value of all pixels in the array, which can be understood as an attribute, and "4" represents the total number of pixels in the array, which can be understood as the number of objects corresponding to this attribute. Similarly, the sub-pixel arrays 202 and 203 can be encoded as 8[0, 0] and 4[0, 1] respectively. Since the events of the pixel array are usually not very scattered in actual applications, this method can make the data compact, efficient, with a small data volume, and the subsequent decoding is simple. It should be noted that the above encoding data format in this embodiment is only a preferred example. In actual applications, other equivalent replacement methods with the same concept as this application can also be used. For example, the form of the number pair (a, b) can be used, where a represents the total number of pixels, and b represents the light intensity change characterization data. The value of b can be 0, 1, -1, 1. Among them, 0 corresponds to the binary vector [0, 0], 1 corresponds to the binary vector [1, 0], -1 corresponds to the binary vector [0, 1], and 1 corresponds to the binary vector [1, 1]. Thus, the above sub-pixel array 201 can be encoded as (4, 1), and this embodiment does not make a unique limitation on this.

[0039] In an implementation manner of this embodiment, it is possible to sequentially perform full-frame encoding on the light intensity change characterization data and the total number of pixels of each of the multiple sub-pixel arrays in the overall pixel array to obtain encoded data, and then transmit it to the backend processor through the interface.

[0040] As Figure 2 in the 16×8 matrix, each row has 16 pixels. As described above by way of example, the encoding of the first row is 4[1, 0], 8[0, 0], 4[0, 1], and the encoding of the second row is 4[1, 0], 12[0, 0], indicating that the 1st to 4th pixels in the first row of pixels output UP events, no events are generated for the 5th to 12th pixels, and the 13th to 16th pixels output DN events; the 1st to 4th pixels in the second row of pixels output UP events, and no events are generated for the 5th to 16th pixels. It should be noted that since the total number of pixels in each row is determined, the encoding can also be distinguished as belonging to the pixels of the second row when the second 4[1, 0] appears. It should be noted that after the encoded data of the full frame is output, the backend processor can select the pixels in the corresponding area for decoding according to requirements.

[0041] In another implementation manner of this embodiment, an interested pixel region can also be obtained from the pixel array, and then the light intensity change characterization data and the total number of pixels of each of the multiple sub-pixel arrays in the interested pixel region are sequentially encoded to obtain encoded data.

[0042] In practical applications, assuming that only a certain part of the pixel region in the entire pixel array is of interest, then only the interested pixel region needs to be encoded. As Figure 2 shown, the sub-pixel array 204 in the 16×8 matrix is the interested sub-pixel array, then only the sub-pixel array 204 needs to be encoded, and the area outside the sub-pixel array 204 is not encoded, that is, the encoding starts from pixel 2041 and ends at pixel 2042. According to the row scanning order encoding method, the first row encoding of the sub-pixel array 204 can be obtained as 5[0, 0], 3[1, 0], the second row is 8[0, 0], the third row is 5[0, 0], 3[0, 1], and the fourth row is 8[0, 0]. In this way, it is beneficial to reduce the output of data volume, reduce power consumption, and improve the frame rate.

[0043] In one implementation manner of this embodiment, before the step of sequentially encoding the light intensity change characterization data and the total number of pixels of each of the multiple sub-pixel arrays in the pixel array to obtain encoded data, it includes: obtaining the global distribution information of the light intensity change characterization data of all pixels in the pixel array; correspondingly determining an array scanning strategy according to the global distribution information, where the array scanning strategy is not limited to a row order scanning strategy or a column order scanning strategy; and correspondingly dividing the pixel array into multiple sub-pixel arrays based on the array scanning strategy.

[0044] Specifically, during the encoding process, a row scanning method or a column scanning method can be adopted to sequentially perform pixel encoding. For the same pixel array, different scanning methods correspond to different array partitioning methods, and the sub-pixel arrays to be encoded finally are different. To reduce the amount of encoded data, in this embodiment, the global distribution of the data characterized by the light intensity change can be used to determine the concentration degree of the continuously identical data characterized by the light intensity change under different array scanning strategies, and the array scanning strategy with a high concentration degree is selected to partition the pixel array, so as to reduce the number of sub-pixel arrays to be encoded finally. It should be understood that after the sub-pixel arrays are partitioned in advance according to a specific array scanning strategy in this embodiment, all sub-pixel arrays are sequentially encoded according to the partitioning order of the sub-pixel arrays during the subsequent encoding process to obtain the final encoded data.

[0045] Further, in an implementation manner of this embodiment, the scanning unit of the array scanning strategy is a single row or a single column. Correspondingly, after the step of partitioning the pixel array into multiple sub-pixel arrays according to the array scanning strategy, the following steps are further included: for each adjacent scanning unit, comparing the data characterizing the light intensity change of the last sub-pixel array of the previous scanning unit with the first sub-pixel array of the next scanning unit; merging the two sub-pixel arrays with consistent comparison results in the adjacent scanning units.

[0046] Specifically, the scanning unit in this embodiment refers to the unit for partitioning the sub-pixel arrays. Each scanning unit does not interfere with each other during array partitioning. However, for the entire pixel array, each scanning unit is formally independent, but actually is associated with the previous and subsequent scanning orders during the continuous scanning process. For example, when the single row is used as the scanning unit, the last pixel in the previous row of pixels is adjacent to the first pixel in the next row of pixels in the scanning order. In this embodiment, the data characterizing the light intensity change of the first and last sub-pixel arrays of adjacent scanning units are compared. When the data characterizing the light intensity change of the last sub-pixel array of the previous row and the first sub-pixel array of the next row are the same, these two sub-pixel arrays with the same attributes are merged across different scanning units. For example Figure 2 the encoding of the second row in [Example] is 4[1, 0], 12[0, 0], and the encoding of the third row is 2[0, 0], 2[1, 0], 5[0, 0], 3[1, 0], 4[0, 0]. There are a total of 7 sub-pixel arrays in the two rows, and 7 encodings are required. However, the binary vectors of the last sub-pixel array of the second row and the first sub-pixel array of the third row are both [0, 0], and they are adjacent in the scanning order. Therefore, after the sub-pixel arrays are partitioned according to the row-by-row sequential scanning strategy, the above two sub-pixel arrays can be further merged, so that only 6 encodings are required when encoding the pixels of the second row and the third row, improving the encoding efficiency and reducing the subsequent decoding complexity.

[0047] In another implementation manner of this embodiment, the step of encoding the light intensity change characterization data and the total number of pixels of each of the multiple sub-pixel arrays in the pixel array in sequence to obtain encoded data includes: during the scanning of the pixel array according to a preset array scanning strategy, whenever a pixel with light intensity change characterization data different from that of the previously scanned unencoded pixel is scanned, all the previously scanned unencoded pixels are classified into the same sub-pixel array, where the array scanning strategy includes a row-sequential scanning strategy or a column-sequential scanning strategy; encoding the light intensity change characterization data and the total number of pixels of each sub-pixel array respectively to obtain encoded data.

[0048] Specifically, in this embodiment, when sequentially scanning the pixels in the pixel array according to a predetermined array scanning strategy, the pixels with the same light intensity change characterization data continuously scanned in each scanning unit are classified into one sub-pixel array. Taking the first row of the pixel array mentioned above Figure 2 as an example, when the fifth pixel is scanned in the row scanning mode, four pixels with the same light intensity change characterization data have been continuously scanned. When the light intensity change characterization data is detected to be different from that before when scanning the fifth pixel, the first four scanned pixels are classified into the same sub-pixel array and encoded. Similarly, continue to scan backward according to the same principle for sub-pixel array division and encoding until the encoding is completed.

[0049] Further, in one implementation manner of this embodiment, the scanning unit of the array scanning strategy is a single row or a single column. Correspondingly, the step of encoding the light intensity change characterization data and the total number of pixels of each sub-pixel array respectively to obtain encoded data includes: encoding the light intensity change characterization data and the total number of pixels of each sub-pixel array respectively to obtain multiple sub-encoded data; comparing the encoded values corresponding to the light intensity change characterization data in adjacent sub-encoded data; merging the adjacent sub-encoded data with consistent comparison results among the multiple sub-encoded data to obtain encoded data.

[0050] Specifically, in a single scanning unit, combined encoding is performed on the continuous pixels with the same light intensity change characterization data, and the encoding behaviors between each scanning unit are independent of each other. In this embodiment, the light intensity change characterization data of adjacent sub-encoded data is compared. If the adjacent sub-encoded data has a consistent comparison result, it indicates that the adjacent sub-encoded data respectively corresponds to the last sub-pixel array in the previous scanning unit and the first sub-pixel array in the subsequent scanning unit, and the two are actually sequentially related in the scanning order. Therefore, the sub-encoded data of the two sub-pixel arrays that are formally in two different scanning units but are actually continuously scanned can be merged, and the unmerged sub-encoded data and the merged sub-encoded data are combined into the final encoded data. For example, Figure 2The encoding 12[0, 0] of the sub-pixel array where the last pixel in the second row is located is merged with the encoding 2[0, 0] of the sub-pixel array where the first pixel in the third row is located. That is, the data characterizing the light intensity change is kept unchanged, and the total number of pixels is added to obtain the merged encoding data 14[0, 0] of the two sub-encoding data. Thus, the total amount of the encoding data and the subsequent decoding complexity are reduced.

[0051] Based on the technical solution of the embodiment of the present application above, corresponding real-time voltages are generated according to the incident light intensities of the pixels in the pixel array of the image sensor; the data characterizing the light intensity change corresponding to each pixel is obtained based on the real-time voltages; the data characterizing the light intensity change and the total number of pixels of each sub-pixel array with the same data characterizing the light intensity change of all pixels in multiple internal parts of the pixel array are encoded in sequence to obtain the encoding data. Since the events output by adjacent pixels in the pixel array usually do not differ much, by implementing the solution of the present application, the continuous pixels with the same event generation behavior are merged and encoded, which can make the data compact, efficient, and small in amount, improve the image output frame rate, and reduce the subsequent decoding complexity.

[0052] Figure 3 The method in [ ] is a refined data encoding method provided by the second embodiment of the present application. The data encoding method includes:

[0053] Step 301: Generate corresponding real-time voltages according to the incident light intensities of the pixels in the pixel array of the image sensor.

[0054] Step 302: After subtracting each real-time voltage from the preset reference voltage, compare each voltage difference with the preset first voltage threshold and the preset second voltage threshold respectively to generate a binary vector for each pixel.

[0055] In this embodiment, the first voltage threshold is greater than 0, and the second voltage threshold is less than 0. When the voltage difference is less than the first voltage threshold, the value of the first number in the binary vector is 0. When the voltage difference is greater than the second voltage threshold, the value of the second number in the binary vector is 0. The values of the binary vector include [0, 0], [1, 0], [0, 1], and [1, 1].

[0056] Step 303: Obtain the global distribution information of the binary vectors of all pixels in the pixel array, and determine the array scanning strategy accordingly according to the global distribution information.

[0057] Specifically, the array scanning strategy includes a row-sequential scanning strategy or a column-sequential scanning strategy, and the scanning unit of the array scanning strategy is a single row or a single column.

[0058] Step 304: During the process of scanning the pixel array according to the array scanning strategy and the preset scanning unit, whenever a pixel different from the binary vector of the previously scanned uncoded pixel is scanned, all the previously scanned uncoded pixels are classified into the same sub-pixel array.

[0059] Step 305: Encode the binary vector and the total number of pixels of each sub-pixel array respectively to obtain a plurality of sub-encoded data.

[0060] In this embodiment, the encoding can be performed in the form of the attribute pair "light intensity change characterization data + total number of pixels", or in the form of the number pair (a, b). This embodiment does not make a unique limitation.

[0061] Step 306: Compare the encoded values corresponding to the binary vectors in adjacent sub-encoded data.

[0062] Step 307: For all sub-encoded data, merge the adjacent sub-encoded data with consistent comparison results to obtain the encoded data.

[0063] Specifically, in this embodiment, the continuous pixels with the same light intensity change characterization data obtained during the real-time scanning process are combined into a sub-pixel array, and then the sub-pixel array is encoded, reducing the amount of encoded data and the decoding complexity. In addition, the sub-encoded data of two sub-pixel arrays that are formally in two different scanning units but are actually continuously scanned are merged, and the unmerged sub-encoded data and the merged sub-encoded data are combined into the final encoded data. Thus, the total amount of encoded data and the subsequent decoding complexity are further reduced.

[0064] It should be understood that the magnitudes of the sequence numbers of the steps in this embodiment do not mean the order of execution of the steps. The order of execution of each step should be determined according to its function and internal logic, and should not constitute a unique limitation to the implementation process of the embodiments of the present application.

[0065] Figure 4 A data encoding device provided in the third embodiment of the present application. This data encoding device can be used to implement the data encoding method in the foregoing embodiments. As Figure 4 shown, this data encoding device mainly includes:

[0066] A generating module 401, configured to generate corresponding real-time voltages according to the incident light intensities of the pixels in the pixel array of the image sensor;

[0067] An obtaining module 402, configured to respectively obtain the light intensity change characterization data corresponding to each pixel based on the real-time voltages;

[0068] An encoding module 403 is configured to sequentially encode the light intensity change characterization data of each of multiple sub-pixel arrays in a pixel array and the total number of pixels to obtain encoded data; wherein, the total number of pixels in a sub-pixel array is greater than or equal to 1, and the light intensity change characterization data of all pixels in each sub-pixel array is the same.

[0069] In some embodiments of this embodiment, the encoding module is specifically configured to: obtain an interested pixel region from the pixel array; sequentially encode the light intensity change characterization data of each of multiple sub-pixel arrays in the interested pixel region and the total number of pixels to obtain encoded data.

[0070] In some embodiments of this embodiment, the scanning unit of the array scanning strategy is a single row or a single column. Correspondingly, the data encoding device further includes: a partitioning module, configured to obtain the global distribution information of the light intensity change characterization data of all pixels in the pixel array; determine the array scanning strategy accordingly based on the global distribution information; wherein, the array scanning strategy includes a row sequential scanning strategy or a column sequential scanning strategy; and partition the pixel array into multiple sub-pixel arrays based on the array scanning strategy.

[0071] Further, in some embodiments of this embodiment, the data encoding device further includes: a merging module, configured to: for each adjacent scanning unit, compare the light intensity change characterization data of the last sub-pixel array in the previous scanning unit with the first sub-pixel array in the next scanning unit; and merge the two sub-pixel arrays with consistent comparison results in the adjacent scanning units.

[0072] In some other embodiments of this embodiment, the encoding module is specifically configured to: during the scanning of the pixel array according to a preset array scanning strategy, whenever a pixel with different light intensity change characterization data from the previously scanned unencoded pixels is scanned, classify all the previously scanned unencoded pixels into the same sub-pixel array; wherein, the array scanning strategy includes a row sequential scanning strategy or a column sequential scanning strategy; and encode the light intensity change characterization data and the total number of pixels of each sub-pixel array respectively to obtain encoded data.

[0073] Further, in some embodiments of this embodiment, the scanning unit of the array scanning strategy is a single row or a single column. Correspondingly, when the encoding module executes the function of encoding the light intensity change characterization data and the total number of pixels of each sub-pixel array respectively to obtain encoded data, it is specifically configured to: encode the light intensity change characterization data and the total number of pixels of each sub-pixel array respectively to obtain multiple sub-encoded data; compare the encoded values corresponding to the light intensity change characterization data in the adjacent sub-encoded data; and merge the adjacent sub-encoded data with consistent comparison results among the multiple sub-encoded data to obtain encoded data.

[0074] In some embodiments of this embodiment, the light intensity change characterization data is a binary vector. Correspondingly, the obtaining module is specifically configured to: after subtracting each real-time voltage from the preset reference voltage, compare each voltage difference with a preset first voltage threshold and a preset second voltage threshold respectively, where the first voltage threshold is greater than 0 and the second voltage threshold is less than 0; generate the binary vector of each pixel according to the comparison result, where when the voltage difference is less than the first voltage threshold, the value of the first value in the binary vector is 0, and when the voltage difference is greater than the second voltage threshold, the value of the second value in the binary vector is 0.

[0075] It should be noted that the data encoding methods in the first and second embodiments can both be implemented based on the data encoding device provided in this embodiment. Those of ordinary skill in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the data encoding device described in this embodiment can refer to the corresponding process in the foregoing method embodiments and will not be repeated here.

[0076] According to the data encoding device provided in this embodiment, generate corresponding real-time voltages according to the incident light intensities of the pixels in the pixel array of the image sensor; respectively obtain the light intensity change characterization data corresponding to each pixel based on the real-time voltages; encode the light intensity change characterization data and the total number of pixels of each sub-pixel array with the same light intensity change characterization data among multiple internal pixels in the pixel array in sequence to obtain encoded data. Since the events output by adjacent pixels in the pixel array usually do not differ much, through the implementation of the solution of this application, the continuous pixels with the same event generation behavior are merged and encoded, which can make the data compact, efficient, and small in data volume, improve the image output frame rate, and reduce the complexity of subsequent decoding.

[0077] Figure 5 A terminal device provided in the fourth embodiment of this application. This terminal device can be used to implement the data encoding method in the foregoing embodiments, and mainly includes:

[0078] A memory 501, a processor 502, and a computer program 503 stored on the memory 501 and executable on the processor 502. The memory 501 and the processor 502 are communicatively connected. When the processor 502 executes the computer program 503, the method in the first or second foregoing embodiment is implemented. Wherein, the number of processors can be one or more.

[0079] The memory 501 can be a high-speed random access memory (RAM, Random Access Memory) or a non-volatile memory, such as a disk memory. The memory 501 is used to store executable program codes, and the processor 502 is coupled to the memory 501.

[0080] Further, the embodiment of the present application also provides a computer-readable storage medium, which may be disposed in the electronic device in the above embodiments, and the computer-readable storage medium may be the memory in the foregoing Figure 5 embodiment shown.

[0081] A computer program is stored on the computer-readable storage medium, and when the program is executed by a processor, it implements the data encoding method in the foregoing embodiments. Further, the computer-readable storage medium may also be various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a RAM, a magnetic disk, or an optical disc that can store program codes.

[0082] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces, and the indirect coupling or communication connection of devices or modules can be in an electrical, mechanical or other form.

[0083] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place, or may be distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0084] In addition, in each embodiment of the present application, the functional modules can be integrated into a processing module, or each module exists physically alone, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0085] When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned readable storage medium includes: various media such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0086] It should be noted that for the foregoing method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, some steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily all essential to this application.

[0087] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0088] The above is the description of the data encoding method, device, equipment, and readable storage medium provided by this application. For those skilled in the art, according to the idea of the embodiments of this application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to this application.

Claims

1. A data encoding method, applied to an event camera, characterized in that, The described data encoding method includes: Generating corresponding real-time voltages according to the incident light intensities of the pixels in the pixel array of the image sensor; Respectively obtaining the light intensity change characterization data corresponding to each of the pixels based on the real-time voltages; Obtaining the global distribution information of the light intensity change characterization data of all the pixels in the pixel array; Correspondingly determining an array scanning strategy according to the global distribution information; wherein, the array scanning strategy includes a row sequential scanning strategy or a column sequential scanning strategy; Correspondingly dividing the pixel array into a plurality of sub-pixel arrays based on the array scanning strategy; Successively encoding the light intensity change characterization data and the total number of pixels of each of the plurality of sub-pixel arrays in the pixel array to obtain encoded data; wherein, the total number of pixels in each sub-pixel array is greater than or equal to 1, and the light intensity change characterization data of all the pixels in each sub-pixel array is the same.

2. The data encoding method according to claim 1, wherein The step of successively encoding the light intensity change characterization data and the total number of pixels of each of the plurality of sub-pixel arrays in the pixel array to obtain encoded data includes: Obtaining an interested pixel region from the pixel array; Successively encoding the light intensity change characterization data and the total number of pixels of each of the plurality of sub-pixel arrays in the interested pixel region to obtain encoded data.

3. The data encoding method according to claim 1, wherein The scanning unit of the array scanning strategy is a single row or a single column; after the step of correspondingly dividing the pixel array into a plurality of the sub-pixel arrays based on the array scanning strategy, it further includes: For each adjacent scanning unit, comparing the light intensity change characterization data of the last sub-pixel array of the previous scanning unit with the first sub-pixel array of the next scanning unit; Merging the two sub-pixel arrays with consistent comparison in the adjacent scanning units.

4. The data encoding method according to claim 1, wherein The step of successively encoding the light intensity change characterization data and the total number of pixels of each of the plurality of sub-pixel arrays in the pixel array to obtain encoded data includes: During the process of scanning the pixel array according to a preset array scanning strategy, whenever a pixel with different light intensity change characterization data from the previously scanned unencoded pixels is scanned, classifying all the previously scanned unencoded pixels into the same sub-pixel array; Respectively encoding the light intensity change characterization data and the total number of pixels of each of the sub-pixel arrays to obtain encoded data.

5. The data encoding method according to claim 4, wherein The scanning unit of the array scanning strategy is a single row or a single column; the step of respectively encoding the light intensity change characterization data and the total number of pixels of each of the sub-pixel arrays to obtain encoded data includes: Respectively encoding the light intensity change characterization data and the total number of pixels of each of the sub-pixel arrays to obtain a plurality of sub-encoded data; Comparing the encoded values corresponding to the light intensity change characterization data in the adjacent sub-encoded data; Merging the adjacent sub-encoded data with consistent comparison among the plurality of sub-encoded data to obtain encoded data.

6. The data encoding method according to any one of claims 1 to 5, characterized in that The light intensity change characterization data is a binary vector; the step of respectively obtaining the light intensity change characterization data corresponding to each of the pixels based on the real-time voltages includes: After subtracting each of the real-time voltages from the preset reference voltage, the voltage differences are respectively compared with a preset first voltage threshold and a preset second voltage threshold; wherein, the first voltage threshold is greater than 0, and the second voltage threshold is less than 0; According to the comparison results, binary vectors of the respective pixels are generated correspondingly; wherein, when the voltage difference is less than the first voltage threshold, the value of the first numerical value in the binary vector is 0, and when the voltage difference is greater than the second voltage threshold, the value of the second numerical value in the binary vector is 0.

7. A data encoding device, applied to an event camera, characterized in that The data encoding device includes: a generating module, configured to generate corresponding real-time voltages according to the incident light intensities of the respective pixels in the pixel array of the image sensor; an obtaining module, configured to respectively obtain the light intensity change characterization data corresponding to the respective pixels based on the real-time voltages; a partitioning module, configured to obtain the global distribution information of the light intensity change characterization data of all the pixels in the pixel array; correspondingly determine an array scanning strategy according to the global distribution information; wherein, the array scanning strategy includes a row sequential scanning strategy or a column sequential scanning strategy; and correspondingly partition the pixel array into a plurality of sub-pixel arrays based on the array scanning strategy; an encoding module, configured to sequentially encode the light intensity change characterization data and the total number of pixels of each of the plurality of sub-pixel arrays in the pixel array to obtain encoded data; wherein, the total number of pixels in the sub-pixel array is greater than or equal to 1, and the light intensity change characterization data of all the pixels in each of the sub-pixel arrays is the same.

8. A terminal device, characterized in that, including a memory and a processor, wherein: the processor is configured to execute a computer program stored on the memory; when the processor executes the computer program, the steps in the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by the processor, the steps in the method according to any one of claims 1 to 6 are implemented.

Images