Data encoding method, device, equipment and readable storage medium

Abstract

The present application provides a data encoding method, apparatus, device, and readable storage medium. The method includes: generating a corresponding real-time voltage based on the incident light intensity of each pixel in the pixel array of an image sensor; obtaining light intensity variation characterization data corresponding to each pixel based on the real-time voltage; using a sub-pixel array including multiple pixels as an encoding unit, performing pixel encoding based on the light intensity variation characterization data of each target sub-pixel array in the pixel array to obtain encoded data. Through the implementation of the solution of the present application, the pixel array is divided into multiple sub-pixel arrays, and pixel encoding is performed using the sub-pixel arrays as encoding units, which can make the data compact, efficient, and small in data volume, thereby improving the image output frame rate.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a data encoding method, apparatus, device, and readable storage medium. Background Art

[0002] With the continuous advancement of science and technology, computer vision technology is becoming increasingly mature. The emergence of event cameras has attracted increasing attention in the field of vision. They mimic the human retina, responding to pixel pulses of brightness changes caused by motion. Therefore, they can capture scene brightness changes at extremely high frame rates, recording events at specific points in time and locations within the image. This creates an event stream rather than a frame stream, thus addressing the issues of information redundancy, large amounts of data storage, and real-time processing inherent in traditional cameras.

[0003] In related art event cameras, after the analog components complete event generation, the digital circuitry, controlled by a master clock, sequentially scans the pixel array to determine whether an event has occurred at each row and column of pixels. If an event has occurred, the event signal is output through an output interface. Conventional event output, where each pixel outputs e = (timestamp, x, y, p), including a timestamp, pixel coordinates (x, y), and event polarity p, is characterized by outputting the changing outline of an object in motion, where the proportion of pixels with events is relatively small. Pixel encoding for all pixels in the pixel array results in slow event data readout, high power consumption, and low image output frame rates. Summary of the Invention

[0004] The embodiments of the present application provide a data encoding method, apparatus, device, and readable storage medium, which can at least solve the problems in the related art of event cameras encoding all pixels in a pixel array separately, resulting in slow event data readout speed, high power consumption, and low image output frame rate.

[0005] A first aspect of an embodiment of the present application provides a data encoding method, applied to an event camera, comprising:

[0006] Generate a corresponding real-time voltage according to the incident light intensity of each pixel in the pixel array of the image sensor;

[0007] Based on the real-time voltage, respectively obtain light intensity change representation data corresponding to each pixel;

[0008] A sub-pixel array including a plurality of the pixels is used as an encoding unit, and pixel encoding is performed based on the light intensity change representation data of each target sub-pixel array in the pixel array to obtain encoded data.

[0009] A second aspect of an embodiment of the present application provides a data encoding device, applied to an event camera, comprising:

[0010] A generating module, configured to generate a corresponding real-time voltage according to the incident light intensity of each pixel in the pixel array of the image sensor;

[0011] An acquisition module, configured to acquire light intensity change representation data corresponding to each pixel based on the real-time voltage;

[0012] The encoding module is used to use a sub-pixel array including a plurality of the pixels as an encoding unit, and perform pixel encoding based on the light intensity change representation data of each target sub-pixel array in the pixel array to obtain encoded data.

[0013] The third aspect of an embodiment of the present application provides a terminal device, including: a memory and a processor, wherein the processor is used to execute a computer program stored in the memory. When the processor executes the computer program, it implements the steps of the data encoding method provided in the first aspect of the embodiment of the present application.

[0014] The fourth aspect of the embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the data encoding method provided in the first aspect of the embodiment of the present application are implemented.

[0015] As can be seen from the above, according to the data encoding method, device, equipment, and readable storage medium provided by the present application, a corresponding real-time voltage is generated based on the incident light intensity of each pixel in the pixel array of the image sensor; based on the real-time voltage, the light intensity change representation data corresponding to each pixel is obtained; and using a sub-pixel array including multiple pixels as an encoding unit, pixel encoding is performed based on the light intensity change representation data of each target sub-pixel array in the pixel array to obtain encoded data. Through the implementation of the present application, the pixel array is divided into multiple sub-pixel arrays, and pixel encoding is performed using the sub-pixel arrays as encoding units, which can make the data compact, efficient, and small in data volume, thereby improving the image output frame rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of a basic flow chart of a data encoding method provided in the first embodiment of the present application;

[0017] Figure 2 A schematic diagram of a pixel array composed of a plurality of pixels provided in a first embodiment of the present application;

[0018] Figure 3 A detailed flowchart of a data encoding method provided in the second embodiment of the present application;

[0019] Figure 4A schematic diagram of a program module of a data encoding device provided in the third embodiment of the present application;

[0020] Figure 5 A schematic structural diagram of a terminal device provided in the fourth embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the purpose, features, and advantages of the invention of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of this application.

[0022] In the description of the embodiments of the present application, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the embodiments of the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0023] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, the meaning of "plurality" is two or more, unless otherwise clearly specified.

[0024] In the embodiments of the present application, unless otherwise expressly specified or limited, the terms "installed," "connected," "connected," "fixed," etc. should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integration; mechanical connections, electrical connections; direct connections, or indirect connections through an intermediate medium; and can refer to internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in the embodiments of the present application based on specific circumstances.

[0025] The above description is only a preferred embodiment of the present application and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of protection of the present application.

[0026] In order to solve the problems in the related art that event cameras perform pixel encoding on all pixels in the pixel array separately, resulting in slow event data readout speed, high power consumption and low image output frame rate, the first embodiment of the present application provides a data encoding method applied to event cameras. An event camera (Event-based Camera) is a new type of image sensor. Each pixel of the event camera works independently, and an event is output only when the brightness change of a pixel reaches a certain threshold.

[0027] like Figure 1 This is a basic flow chart of the data encoding method provided in this embodiment. The data encoding method includes the following steps:

[0028] Step 101: Generate a corresponding real-time voltage according to the incident light intensity of each pixel in the pixel array of the image sensor.

[0029] Specifically, the event camera is provided with an image sensor, which includes a pixel array composed of a plurality of pixels. Each pixel sensor in the pixel array is an integrated circuit, in which a photodiode can be integrated with a charge-collecting capacitor, which generates a photocurrent by the photodiode in response to the intensity of incident light, and then generates a real-time voltage according to the photocurrent.

[0030] Step 102: Obtain light intensity change representation data corresponding to each pixel based on the real-time voltage.

[0031] In one implementation of this embodiment, the data representing the light intensity change is a binary vector. Therefore, after subtracting each real-time voltage from a preset reference voltage, each voltage difference is compared with a preset first voltage threshold and a preset second voltage threshold respectively; then, a binary vector is generated for each pixel according to the comparison result.

[0032] Specifically, this embodiment subtracts the reference voltage from each real-time voltage to obtain a voltage difference, and then determines whether the incident light intensity has changed (become stronger or weaker) based on the voltage difference. In actual applications, a first comparison circuit and a second comparison circuit can be configured respectively. The first comparison circuit is used to detect whether the light intensity has become stronger, and the second comparison circuit is used to detect whether the light intensity has become weaker. The two comparison circuits correspond to different voltage thresholds, respectively, wherein the first voltage threshold is greater than 0, and the second voltage threshold is less than 0. When the above voltage difference is greater than the first voltage threshold, the first comparison circuit detects that the light intensity has increased, and when the above voltage difference is less than the second voltage threshold, the second comparison circuit detects that the light intensity has decreased.

[0033] It should be noted that the reference voltage may be a pre-set fixed voltage or a voltage that can be flexibly updated subsequently. Furthermore, in a preferred embodiment of this embodiment, the first voltage threshold and the second voltage threshold may be opposite numbers. For example, if the first voltage threshold is 0.1V, then the second voltage threshold is -0.1V.

[0034] It should also be understood that, in actual applications, each pixel will generate a corresponding 2-bit vector, which is used to characterize whether the light signal is getting stronger or weaker. When the voltage difference is less than the first voltage threshold, the first value in the binary vector is 0, and when the voltage difference is greater than the second voltage threshold, the second value in the binary vector is 0. In this embodiment, the binary vector can be represented in the form of a number pair [a, b], where the value a is determined by the comparison relationship between the voltage difference and the first voltage threshold, and the value b is determined by the comparison relationship between the voltage difference and the second voltage threshold. When the voltage difference is greater than the first voltage threshold, the value of a is a non-zero value, and when the voltage difference is less than the second voltage threshold, the value of b is a non-zero value. In the preferred embodiment of this embodiment, the non-zero values ​​of a and b can both be 1. Of course, in other embodiments, they can also be 1 and -1, respectively.

[0035] Next, this embodiment illustrates a method for generating a binary vector by way of example. Assume that the reference voltage Vm is 0.5V, and the first voltage threshold A1 and the second voltage threshold A2 are 0.1V and -0.1V, respectively. When the real-time voltage Vin is 0.7V, the voltage difference Vin-Vm is greater than the first voltage threshold A1, at which time the UP event is triggered and the vector value 1 is output, and the voltage difference Vin-Vm is greater than the second voltage threshold A2, the DN event is not triggered, and the vector value 0 is output. Then, according to the comparison result, the final output 2-bit vector is [1, 0]; when the real-time voltage Vin is 0.5V, the voltage difference Vin-Vm is less than the first voltage threshold A1, at which time the UP event is not triggered and the vector value 0 is output, and the voltage difference Vin-Vm is greater than the second voltage threshold A2, the DN event is not triggered and the vector value 0 is output, then, according to the comparison result, the final output 2-bit vector is [0, 0]; when the real-time voltage Vin is 0.3V, the voltage difference Vin-Vm is less than the first voltage threshold A1, at which time the UP event is not triggered and the vector value 0 is output, and the voltage difference Vin-Vm is less than the second voltage threshold A2, the DN event is triggered and the vector value 1 is output, then, according to the comparison result, the final output 2-bit vector is [0, 1]. It should be understood that when the 2-bit vector is the all-zero pair [0, 0], it indicates that the optical signal has not changed, while when the 2-bit vector is the non-all-zero pair [1, 0] or [0, 1], it indicates that the optical signal has become stronger or weaker. Of course, in practical applications, the system may fail and output the 2-bit vector [1, 1]. This information is generally considered invalid.

[0036] Step 103 : Using a sub-pixel array including a plurality of pixels as a coding unit, pixel coding is performed based on the light intensity variation representation data of each target sub-pixel array in the pixel array to obtain coded data.

[0037] In this embodiment, the coded data is read out by the readout circuit of the event camera to generate an event image. Figure 2 FIG. 1 is a schematic diagram of a pixel array provided by this embodiment. The size of the pixel array represents the resolution of the event camera. For the sake of convenience, this embodiment only shows a 16×8 matrix. In actual applications, the actual pixel array is much larger than 16×8, such as 480x640. In the figure, the value "+1" represents an UP event corresponding to the binary vector [1, 0], the value "-1" represents a DN event corresponding to the binary vector [0, 1], and the value "0" represents no event. In this embodiment, the pixel array (such as Figure 2 20) are grouped to obtain multiple sub-pixel arrays (as shown in FIG. Figure 2 201, 202, 203 and 205), each sub-pixel array includes a plurality of pixels. The specific form of the sub-pixel array may depend on the actual situation. This embodiment does not make a sole limitation on this. Figure 2 The 16×8 pixel array can be divided into 4×8 sub-pixel arrays by dividing the array as shown in 201. Then, the target sub-pixel array in the pixel array is encoded, which can make the data compact, efficient, and small in data volume, thereby improving the image output frame rate. It should be understood that the target sub-pixel array in this embodiment can be all the divided sub-pixel arrays or only a specific part of the sub-pixel arrays.

[0038] In one implementation of this embodiment, after the above-mentioned step of obtaining the light intensity change characterization data corresponding to each pixel based on the real-time voltage, it also includes: determining the array division granularity based on the total number of pixels in the pixel array and / or the data reading performance of the readout circuit; dividing the pixel array into multiple sub-pixel arrays according to the array division granularity.

[0039] Specifically, in practical applications, when the total number of pixels in the pixel array is large, the amount of event data is correspondingly large, resulting in a greater data readout load for the event camera. Furthermore, the data readout performance of the readout circuit determines the amount of data that the event camera can effectively read per unit time. Based on this, this embodiment can adaptively set the sub-pixel array size based on the total number of pixels and / or the readout circuit, thereby avoiding data overload when reading encoded data and improving the image output frame rate.

[0040] In one implementation of the present embodiment, the data encoding method of the present embodiment also includes: performing statistics on the event generation rate of the pixel array; when the event generation rate is lower than a preset event generation rate threshold, executing the step of using a sub-pixel array including multiple pixels as an encoding unit, and performing pixel encoding based on the light intensity change representation data of each target sub-pixel array in the pixel array.

[0041] Specifically, in actual applications, each pixel has a light intensity change characterization data. When there are fewer pixels among these pixels that actually generate events, the data encoding of a large number of pixels without events will occupy a large amount of data processing performance. Therefore, it is necessary to optimize the pixel encoding strategy in this scenario. Based on this, the present embodiment can count the event generation rate of the pixel array based on the binary vector of the pixel, and compare it with the preset threshold. When it is determined that the event generation rate of the pixel array is low, the pixel encoding strategy of the present embodiment with the sub-pixel array as the encoding unit is triggered to be executed, which can make the data compact and the data volume smaller.

[0042] In one implementation of the present embodiment, before the above-mentioned step of performing pixel encoding based on the light intensity change characterization data of each target sub-pixel array in the pixel array, it also includes: determining the event generation status of each sub-pixel array based on the light intensity change characterization data of all pixels in each sub-pixel array; and determining the sub-pixel array with an event generation status of event generation as the target sub-pixel array.

[0043] Specifically, in this embodiment, only the sub-pixel arrays with event generation are encoded to further reduce the amount of data. This embodiment can determine the event generation status of the sub-pixel array based on the light intensity change characterization data of each pixel, where the event generation status includes event generation and no event generation. In the preferred embodiment of this embodiment, if at least one pixel in the sub-pixel array has an event generation, the event generation status of the sub-pixel array is event generation; if all pixels in the sub-pixel array have no event generation, the event generation status of the sub-pixel array is no event generation. Of course, in other implementations, the sub-pixel array in which the proportion of pixels with event generation is greater than a preset ratio threshold can also be determined as the target sub-pixel array, and this embodiment does not limit this to a single factor.

[0044] Furthermore, in one implementation of the present embodiment, the above-mentioned steps of performing pixel encoding based on the light intensity change characterization data of each target sub-pixel array in the pixel array to obtain the encoded data include: obtaining the event polarity of each pixel based on the light intensity change characterization data of each target sub-pixel array in the pixel array; performing pixel encoding using the event polarity and pixel coordinate position of the pixel in each target sub-pixel array to obtain the encoded data.

[0045] Specifically, such as Figure 2 As shown, the polarity of the event at the desired pixel position may include a value of "+1" or a value of "-1" when the current event is compared with the immediately previous event at the associated position, wherein "+1" indicates that the light intensity of the pixel has a positive change during the desired time period, and "-1" indicates that the light intensity of the pixel has a negative change during the desired time period. In addition, it should be noted that Figure 2 The median value "0" is simply used to indicate that no event has occurred at the pixel at that location. In practice, "0" is not actually output. This embodiment scans the pixel array, perhaps in the form of a row scan, and encodes the pixel coordinates and polarity of the event to generate a code sequence. After obtaining the pixel code characters for all events, the readout module can obtain the pixel position and polarity of each event based on the code characters.

[0046] Furthermore, in one implementation of the present embodiment, the above-mentioned step of performing pixel encoding using the event polarity and pixel coordinate position of the pixels in each target sub-pixel array to obtain encoded data includes: determining the array coordinate position of each target sub-pixel array according to the pixel coordinate position of the pixels in each target sub-pixel array; performing pixel encoding using the array coordinate position of each target sub-pixel array and the event polarity of all pixels to obtain encoded data.

[0047] Specifically, in actual applications, the pixel coordinate positions and event polarities of all pixels in the sub-pixel array scanned between the frame start time and the frame end time can be encoded, or the array position of the sub-pixel array and the event polarities of all pixels can be encoded. The frame start time and the frame end time have specific coding marks. For example, the frame start time can be FFFF0000, and the frame end time can be FFFF0101. All event signals between the two constitute an event frame. In this embodiment, from the frame start time to the frame end time, it is determined by row scanning whether an event is generated in the sub-pixel array. If an event is generated, the row and column positions of the sub-pixel array and the event polarities of all pixels in the array are encoded. In this embodiment, the array coordinate position is preferably encoded in binary and the event polarity is encoded in hexadecimal. For example Figure 2 In the example, an event is generated in sub-pixel array 201. The array coordinate position of the sub-pixel array is (1, 1), i.e., the first row and first column. The event polarity of all pixels in the array is +1, and the encoding is 010155, where the binary value 0101 represents the first row and first column, and the hexadecimal value 55 is represented as 01010101 in binary, indicating that all four pixels have an UP event. Similarly, sub-pixel array 205 can be encoded as 010441, where the binary value 0104 represents the first row and fourth column, and the hexadecimal value 41 represents 01000001, indicating that the first pixel in sub-pixel array 205 has an UP event generated, the second and third pixels have no events generated, and the fourth pixel has an UP event generated. It should be understood that in actual applications, the encoding rules can be flexibly set, and the above example in this embodiment is only used to illustrate its solution and is not a sole limitation.

[0048] Based on the technical solution of the above-mentioned embodiment of the present application, a corresponding real-time voltage is generated based on the incident light intensity of each pixel in the pixel array of the image sensor; based on the real-time voltage, light intensity variation characterization data corresponding to each pixel is obtained; and using a sub-pixel array including multiple pixels as an encoding unit, pixel encoding is performed based on the light intensity variation characterization data of each target sub-pixel array in the pixel array to obtain encoded data. Through the implementation of the solution of the present application, the pixel array is divided into multiple sub-pixel arrays, and pixel encoding is performed using the sub-pixel arrays as encoding units, which can make data compact, efficient, and small in data volume, thereby improving the image output frame rate.

[0049] Figure 3 The method in is a refined data encoding method provided in the second embodiment of the present application, and the data encoding method includes:

[0050] Step 301: Generate a corresponding real-time voltage according to the incident light intensity of each pixel in the pixel array of the image sensor.

[0051] Step 302 : After calculating the difference between each real-time voltage and a preset reference voltage, each voltage difference is compared with a preset first voltage threshold and a preset second voltage threshold.

[0052] In this embodiment, the first voltage threshold is greater than 0, and the second voltage threshold is less than 0.

[0053] Step 303: Generate a binary vector for each pixel according to the comparison result.

[0054] In this embodiment, when the voltage difference is less than a first voltage threshold, the first value in the binary vector is 0, and when the voltage difference is greater than a second voltage threshold, the second value in the binary vector is 0. Compared to the immediately preceding event state at the associated position (e.g., a pixel of an event camera), the event state at the desired pixel position may include a 2-bit vector [0, 0], [1, 0], [0, 1], [1, 1]. The 2-bit vector [0, 0] indicates that the pixel has no event output during the desired time period, i.e., the light intensity remains unchanged; the 2-bit vector [1, 0] indicates that the pixel outputs an UP event during the desired time period, i.e., the light intensity increases; the 2-bit vector [0, 1] indicates that the pixel outputs a DN event during the desired time period, i.e., the light intensity decreases; and the 2-bit vector [1, 1] indicates that an error event occurs at the pixel during the desired time period.

[0055] Step 304 : Divide the pixel array into a plurality of sub-pixel arrays, and determine the event generation state of each sub-pixel array based on the binary vectors of all pixels in each sub-pixel array.

[0056] Step 305: Determine the sub-pixel array whose event generation status is event generation as the target sub-pixel array.

[0057] In this embodiment, the event generation state includes event generation and no event generation. By encoding only the sub-pixel arrays with event generation, the amount of data can be further reduced.

[0058] Step 306 : Obtain the event polarity of each pixel based on the binary vector of each target sub-pixel array in the pixel array, and determine the array coordinate position of each target sub-pixel array accordingly according to the pixel coordinate position of the pixel in each target sub-pixel array.

[0059] Step 307 : Pixel encoding is performed using the array coordinate position of each target sub-pixel array and the event polarity of all pixels to obtain encoded data.

[0060] Specifically, this embodiment scans the pixel array, and the scanning form can be row scanning. The pixel coordinate position of the event and the event polarity are encoded to generate a coding sequence. After obtaining the pixel coding characters of all events, the readout module can obtain the position and event polarity of each event pixel according to the coding characters, and then generate an event image based on this.

[0061] It should be understood that the size of the serial numbers of the steps in this embodiment does not mean the order in which the steps are executed. The order in which the steps are executed should be determined by their functions and internal logic, and should not constitute a sole limitation on the implementation process of the embodiments of this application.

[0062] Figure 4 A data encoding device is provided in the third embodiment of the present application. The data encoding device can be used to implement the data encoding method in the above embodiment. Figure 4 As shown, the data encoding device mainly includes:

[0063] A generating module 401 is configured to generate a corresponding real-time voltage according to the incident light intensity of each pixel in the pixel array of the image sensor;

[0064] An acquisition module 402 is configured to acquire light intensity variation characterization data corresponding to each pixel based on the real-time voltage;

[0065] The encoding module 403 is configured to use a sub-pixel array including a plurality of pixels as an encoding unit and perform pixel encoding based on the light intensity variation representation data of each target sub-pixel array in the pixel array to obtain encoded data.

[0066] In some implementations of this embodiment, the light intensity variation representation data is a binary vector. Accordingly, the acquisition module is specifically configured to: after subtracting each real-time voltage from a preset reference voltage, compare each voltage difference with a preset first voltage threshold and a preset second voltage threshold; wherein the first voltage threshold is greater than 0 and the second voltage threshold is less than 0; and generate a binary vector for each pixel based on the comparison result; wherein, when the voltage difference is less than the first voltage threshold, the first value in the binary vector is 0, and when the voltage difference is greater than the second voltage threshold, the second value in the binary vector is 0.

[0067] In some implementations of this embodiment, the data encoding device also includes: a determination module for determining the event generation status of each sub-pixel array based on the light intensity change representation data of all pixels in each sub-pixel array; wherein the event generation status includes event generation and no event generation; and the sub-pixel array whose event generation status is event generation is determined as the target sub-pixel array.

[0068] Furthermore, in some implementations of this embodiment, the encoding module is specifically used to: obtain the event polarity of each pixel based on the light intensity change characterization data of each target sub-pixel array in the pixel array; use the event polarity and pixel coordinate position of the pixel in each target sub-pixel array to perform pixel encoding to obtain encoded data.

[0069] Furthermore, in some other implementations of this embodiment, when the encoding module performs the above-mentioned function of performing pixel encoding using the event polarity and pixel coordinate position of the pixels in each target sub-pixel array to obtain encoded data, it is specifically used to: determine the array coordinate position of each target sub-pixel array according to the pixel coordinate position of the pixels in each target sub-pixel array; and perform pixel encoding using the array coordinate position of each target sub-pixel array and the event polarity of all pixels to obtain encoded data.

[0070] In some implementations of this embodiment, the data encoding device also includes: a division module, used to determine the array division granularity based on the total number of pixels in the pixel array and / or the data reading performance of the readout circuit; and divide the pixel array into multiple sub-pixel arrays according to the array division granularity.

[0071] In some implementations of this embodiment, the data encoding device further includes a statistics module configured to collect statistics on the event generation rate of the pixel array. Accordingly, the encoding module is configured to, when the event generation rate is lower than a preset event generation rate threshold, perform pixel encoding based on the light intensity variation representation data of each target sub-pixel array in the pixel array, using a sub-pixel array comprising multiple pixels as an encoding unit, to generate encoded data.

[0072] It should be noted that the data encoding methods in the first and second embodiments can be implemented based on the data encoding device provided in this embodiment. Ordinary technical personnel in the relevant field can clearly understand that for the convenience and conciseness of description, the specific working process of the data encoding device described in this embodiment can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0073] According to the data encoding device provided in this embodiment, a corresponding real-time voltage is generated based on the incident light intensity of each pixel in the pixel array of the image sensor; based on the real-time voltage, light intensity variation representation data corresponding to each pixel is obtained; and pixel encoding is performed based on the light intensity variation representation data of each target sub-pixel array in the pixel array, using a sub-pixel array including multiple pixels as an encoding unit, to obtain encoded data. Through the implementation of the solution of this application, the pixel array is divided into multiple sub-pixel arrays, and pixel encoding is performed using the sub-pixel arrays as encoding units, which can make data compact, efficient, and small in data volume, thereby improving the image output frame rate.

[0074] Figure 5A terminal device is provided in the fourth embodiment of the present application. The terminal device can be used to implement the data encoding method in the above embodiment, mainly including:

[0075] Memory 501, processor 502, and computer program 503 stored in memory 501 and executable on processor 502. Memory 501 and processor 502 are connected via communication. When processor 502 executes computer program 503, the method of the aforementioned embodiment 1 or 2 is implemented. The number of processors may be one or more.

[0076] The memory 501 can be a high-speed random access memory (RAM) memory or a non-volatile memory such as a disk memory. The memory 501 is used to store executable program codes. The processor 502 is coupled to the memory 501 .

[0077] Furthermore, the embodiment of the present application also provides a computer-readable storage medium, which can be provided in the electronic device in the above embodiments. The computer-readable storage medium can be the above Figure 5 Memory in the illustrated embodiment.

[0078] The computer-readable storage medium stores a computer program that, when executed by a processor, implements the data encoding method of the aforementioned embodiment. Furthermore, the computer-readable storage medium may be a USB flash drive, a mobile hard drive, a read-only memory (ROM), RAM, a magnetic disk, or an optical disk, among other media capable of storing program code.

[0079] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules is only a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0080] Modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of these modules may be selected to achieve the purpose of this embodiment based on actual needs.

[0081] In addition, the functional modules in the various embodiments of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into a single module. The above-mentioned integrated modules may be implemented in the form of hardware or software functional modules.

[0082] If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a readable storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned readable storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0083] It should be noted that for the aforementioned method embodiments, for ease of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0084] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0085] The above is a description of the data encoding method, device, equipment and readable storage medium provided by this application. For those skilled in the art, based on the ideas of the embodiments of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A data encoding method, applied to an event camera, characterized in that: The data encoding method comprises: Generate a corresponding real-time voltage according to the incident light intensity of each pixel in the pixel array of the image sensor; Based on the real-time voltage, respectively obtain light intensity change representation data corresponding to each pixel; determining an array partitioning granularity according to the total number of pixels in the pixel array and / or the data reading performance of a readout circuit; Dividing the pixel array into a plurality of sub-pixel arrays according to the array division granularity; Taking a sub-pixel array including a plurality of the pixels as an encoding unit, pixel encoding is performed based on the light intensity variation representation data of each target sub-pixel array in the pixel array to obtain encoded data; Before the step of performing pixel encoding based on the light intensity variation characterization data of each target sub-pixel array in the pixel array, the method further includes: Determining the event generation status of each sub-pixel array based on the light intensity change characterization data of all pixels in each sub-pixel array; wherein the event generation status includes event generation and no event generation; Determine the sub-pixel array whose event generation state is event generation as the target sub-pixel array; Before the step of using a sub-pixel array including a plurality of the pixels as an encoding unit and performing pixel encoding based on the light intensity variation representation data of each target sub-pixel array in the pixel array, the method further includes: Collecting statistics on the event generation rate of the pixel array; When the event generation rate is lower than the preset event generation rate threshold, the step of performing pixel encoding based on the light intensity change representation data of each target sub-pixel array in the pixel array and using the sub-pixel array including the plurality of pixels as an encoding unit is executed.

2. The data encoding method according to claim 1, wherein: The step of performing pixel encoding based on the light intensity variation characterization data of each target sub-pixel array in the pixel array to obtain encoded data includes: acquiring an event polarity of each pixel based on the light intensity variation characterization data of each target sub-pixel array in the pixel array; Pixel encoding is performed using the event polarity and pixel coordinate position of the pixels in each target sub-pixel array to obtain encoded data.

3. The data encoding method according to claim 2, wherein: The step of performing pixel encoding using the event polarity and pixel coordinate position of the pixels in each target sub-pixel array to obtain encoded data includes: determining the array coordinate position of each target sub-pixel array according to the pixel coordinate position of the pixel in each target sub-pixel array; Pixel encoding is performed using the array coordinate position of each target sub-pixel array and the event polarity of all pixels to obtain encoded data.

4. The data encoding method according to any one of claims 1 to 3, characterized in that: The light intensity change characterization data is a binary vector; and the step of respectively obtaining the light intensity change characterization data corresponding to each pixel based on the real-time voltage includes: After subtracting each of the real-time voltages from a preset reference voltage, each voltage difference is compared with a preset first voltage threshold and a preset second voltage threshold; wherein the first voltage threshold is greater than 0 and the second voltage threshold is less than 0; A binary vector is generated for each pixel according to the comparison result; wherein, when the voltage difference is less than the first voltage threshold, the value of the first value in the binary vector is 0, and when the voltage difference is greater than the second voltage threshold, the value of the second value in the binary vector is 0.

5. A data encoding device, applied to an event camera, characterized in that: The data encoding device comprises: A generating module, configured to generate a corresponding real-time voltage according to the incident light intensity of each pixel in the pixel array of the image sensor; An acquisition module, configured to acquire light intensity change representation data corresponding to each pixel based on the real-time voltage; a partitioning module, configured to determine an array partitioning granularity based on the total number of pixels in the pixel array and / or the data reading performance of the readout circuit; and to divide the pixel array into a plurality of sub-pixel arrays according to the array partitioning granularity; an encoding module, configured to use a sub-pixel array including a plurality of the pixels as an encoding unit, and perform pixel encoding based on the light intensity variation representation data of each target sub-pixel array in the pixel array to obtain encoded data; a determination module, configured to determine, based on the light intensity variation characterization data of all pixels in each of the sub-pixel arrays, an event generation state of each of the sub-pixel arrays, and further configured to determine the sub-pixel array having an event generation state as the target sub-pixel array, wherein the event generation state includes event generation and no event generation; A statistics module, configured to collect statistics on the event generation rate of the pixel array; The encoding module is used to, when the event generation rate is lower than a preset event generation rate threshold, use a sub-pixel array including multiple pixels as an encoding unit, and perform pixel encoding based on the light intensity change characterization data of each target sub-pixel array in the pixel array to obtain encoded data.

6. A terminal device, characterized in that: Comprising a memory and a processor, wherein: The processor is configured to execute a computer program stored in the memory; When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images