Self-adaptive motion sensing method and system based on space-time optical information coding method
By adopting an adaptive motion sensing system based on spatio-temporal optical information encoding method in motion perception technology, using the short-time memory characteristics of distributed photoelectric sensing units and optoelectronic devices, high-precision and adaptive perception of moving objects are achieved, and the problems of insufficient accuracy and poor adaptability in the prior art are solved.
Patent Information
- Application Number
- CN202411846857.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-16
- Publication Date
- 2025-03-25
AI Technical Summary
Existing motion perception technologies are difficult to achieve high-precision and adaptive spatio-temporal optical information encoding, which limits its application and accuracy in different tasks.
Adaptive motion sensing system based on space-time optical information encoding method is adopted to detect optical signals through distributed photoelectric sensing units, map spatial information of moving objects to different optoelectronic devices, and use the short-term memory characteristics of the optoelectronic devices to encode the time domain changes of the optical signal into the photoresponse current.
The compactness of channel motion information and spatial position information is improved, a self-adjusted motion perception system is formed, and the accuracy of perception of moving objects is improved.
Smart Images
Figure CN119691681A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of motion perception technology, and in particular relates to an adaptive motion perception method and system based on a spatiotemporal optical information encoding method. Background Art
[0002] In the era of artificial intelligence, data generated by machine vision accounts for a large part of the massive IoT data. Motion perception is an important and highly complex task that aims to generate corresponding electrical signals based on changes in light information and further process them. This ability to accurately detect spatiotemporal information has attracted extensive research interest in fields such as human behavior prediction and autonomous driving. However, designing accurate sampling methods is a difficult problem because it requires processing multidimensional data that includes spatial and temporal information.
[0003] At present, the mainstream solution is to use bionic visual neurons to encode light information to realize a high-precision motion perception system. In recent years, many biological visual bionic systems have brought great development to motion perception technology. At present, the implementation methods of motion perception based on biological visual bionic systems can be divided into signal filtering and encoding of time information. In terms of signal filtering, most of the application devices are transformed from short-term plasticity to long-term plasticity to improve the spatial resolution and signal-to-noise ratio of the signal. In the encoding of time information, photoelectric sensors are used to nonlinearly couple the input time series information. In the process of processing long sequence tasks, it will be limited by the number of states of photoelectric sensors. In addition, photoelectric sensors are required to have different encoding capabilities in different tasks, which limits their further application.
[0004] Therefore, it is of great significance to study an adaptive motion perception system and method with high-precision spatiotemporal optical information encoding method that can self-adjust according to different tasks. Summary of the invention
[0005] In view of the problems existing in the above-mentioned prior art, the present invention proposes an adaptive motion perception system and method based on a spatiotemporal light information coding method, designs a spatiotemporal light information coding method that integrates time and space information, improves the compactness of channel motion information and spatial position information, forms a self-adjusting motion perception system, and improves the accuracy of perception of moving objects.
[0006] The technical solution of the present invention is as follows:
[0007] A method for encoding spatiotemporal optical information, characterized in that distributed photoelectric sensing units are used to detect input optical signals, and the spatial information of moving objects is mapped onto different photoelectric devices. On this basis, the short-term memory characteristics of the photoelectric devices are used to encode the changes of the optical signals in the time domain into a photoresponse current.
[0008] Furthermore, the spatio-temporal optoelectronic information encoding method is characterized in that the optoelectronic device in the optoelectronic sensing unit has a short-term memory characteristic of photocurrent response under the action of different gate voltages. When a light pulse acts, the photocurrent response increases instantaneously. When the light pulse stops, the photocurrent response decays slowly with the increase of time. The short-term memory characteristic of the photocurrent response is expressed by the following formula:
[0009] I t = I0 + ΔIexp(-t / τ)
[0010] where I t is the photocurrent response of the optoelectronic device at time t, I0 is the current value of the optoelectronic device at the initial time, ΔI is the change in the current of the optoelectronic device, and τ is the time decay constant of the optoelectronic device. The optoelectronic device is a phototransistor device or a device composed of a photosensitive resistor and a capacitor. When it is a phototransistor device, the value of τ is regulated by applying the gate voltage of the device. When it is a device composed of a photosensitive resistor and a capacitor, the value of τ is regulated by changing the capacitance value.
[0011] Furthermore, the spatio-temporal optical information encoding method is characterized in that the method is specifically as follows:
[0012] Set the area where the optoelectronic sensing units are distributed as D*D, the object movement speed as v, the number of pixel points (the number of optoelectronic devices) occupied by the object as n, and the interval between optoelectronic devices as d. The initial photocurrent response of the optoelectronic device is I0, the maximum photocurrent response of the optoelectronic device is I max , the change in photocurrent response is ΔI = I max - I0. When the photocurrent response decays to 1 / e times the change in photocurrent response ΔI, the required time is the time decay constant τ of the optoelectronic device. During the initial speed detection, the sampling time is consistent with the time decay constant of the optoelectronic device. The distance moved by the object within τ time is L = v*τ, where L < D. The number of optoelectronic device responses caused by the object movement trajectory is N = L / d + n. When the time decay constant of the optoelectronic device matches the object movement speed, the photocurrent response of the pixel unit where the object is located at time t is the maximum value I max , and the photocurrent response value of the optoelectronic device at the initial time t0 position is ΔI / e. The photocurrent response I of the optoelectronic devices passed by the object movement trajectory from t0 to t satisfies I0 < I < I max ; At this time, the sum of the currents of the optoelectronic devices passed by the moving object trajectory is the reference value. Therefore, the movement trajectory and movement speed of the object are encoded as photocurrent response values through the short-term memory characteristic of the optoelectronic device unit.
[0013] Based on the spatiotemporal optical information encoding method, an adaptive motion perception method is proposed, which is characterized in that, for different moving object movement rates, in order to sample the complete object movement trajectory, a reference rate and a corresponding time decay constant are set as initial conditions. When the object movement rate reference value is v, the time decay constant of the photoelectric device matched with it is τ; at this time, the object moves a distance L within a fixed sampling time τ. ref =v*τ, L within this distance ref The sum of the current changes of the responses of / d+n photoelectric devices is set as the reference value, that is, The moving speed of the object to be detected is v x , the distance moved by the object in τ time is L real =v x *τ, if the total number of devices that generate photoresponse within this range (L real / d+n)<(L ref / d+n), then the total photoresponse current Prove that the object's velocity v x Less than v, increase the device time decay constant τ; when (L real / d+n)>(L ref / d+n), then the total photoresponse current Prove that the object's velocity v x When the object's motion rate matches the photoresponse current decay rate of the photoelectric device, the photoresponse current of the photoelectric device is converted into a voltage and multiplied and added in parallel with the weight values in the artificial neural network trained by the processing unit to obtain the motion trajectory information.
[0014] The present invention also provides an adaptive motion sensing system, characterized in that it includes a photoelectric sensing unit, a processing unit and a control unit; the photoelectric sensing unit is distributed near the object to be detected, collects the changing light information around the moving object and converts it into a light response current; the processing unit is used to receive the real-time light response current data of the photoelectric sensing unit, and senses and feeds back the object's motion rate and trajectory based on the dynamic light response current; the control unit controls and adjusts the light response current attenuation rate and sampling time of the photoelectric sensing unit based on the feedback signal of the processing unit; in this system, the adaptive motion sensing method is used to realize the light information encoding of the moving object and adaptive motion sensing.
[0015] The technical effects of the present invention are as follows:
[0016] The present invention proposes an adaptive motion perception method and system based on a spatiotemporal optical information encoding method, which utilizes distributed photoelectric sensing units to detect input optical signals, maps the spatial information of a moving object onto different photoelectric devices, and on this basis, utilizes the short-term memory characteristics of the photoelectric devices to encode the changes of the optical signal in the time domain into a photoresponse current; the present invention fuses the time and space information, improves the compactness of the channel motion information and the spatial position information, and reduces the accuracy requirements of the subsequent data processing unit; then the processing unit and the control unit parallelly process the spatiotemporal data of the photoelectric sensing unit and feed it back to the photoelectric sensing unit to form a self-regulating motion perception system, thereby ultimately improving the accuracy of the perception of the moving object. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 This is a schematic diagram of the spatiotemporal optical information encoding method in an embodiment of the present invention;
[0018] Figure 2 is the light response current decay rate of the optoelectronic device under different gate voltages in the embodiment of the present invention;
[0019] Figure 3 Schematic diagram of an adaptive motion sensing system in an embodiment of the present invention;
[0020] Figure 4 A schematic diagram of a circuit for implementing an adaptive motion sensing system in an embodiment of the present invention;
[0021] Figure 5 4 is a flow chart of an adaptive motion sensing method in an embodiment of the present invention. DETAILED DESCRIPTION
[0022] The present invention will be further clearly and completely described below through specific embodiments in conjunction with the accompanying drawings.
[0023] Figure 1 This is the principle diagram of the spatiotemporal optical information encoding method. Based on distributed optoelectronic devices and the attenuation characteristics of the photoresponse current of optoelectronic devices, the speed and trajectory information of the moving object is mapped to the photoresponse current of different optoelectronic devices to achieve the information encoding of the moving object.
[0024] The short-term memory characteristics of the photoresponse current of the optoelectronic device can be expressed as follows:
[0025] I t =I0+ΔIexp(-t / τ)
[0026] Among them I tis the response current of the photoelectric device at time t, I0 is the current value of the photoelectric device at the initial moment, ΔI is the change in the current of the photoelectric device, and τ is the time decay constant of the device. The photoelectric device is a phototransistor device, or a device composed of a photoresistor and a capacitor; when it is a phototransistor device, the τ value is regulated by applying a device gate voltage; when it is a device composed of a photoresistor and a capacitor, the τ value is regulated by changing the capacitor value. This embodiment realizes optical signal detection through a MoS2 back-gate transistor array structure, and the MoS2 back-gate transistor can use the gate voltage to adjust the time decay constant of the photoelectric device.
[0027] Figure 2 The short-term memory characteristics of the photoresponse current of the photoelectric device under different gate voltages are given, that is, the change of the photocurrent decay over time. When the light pulse acts, the photoresponse current increases instantly, and when the light pulse stops, the photoresponse current decays slowly with the increase of time.
[0028] Figure 3 This is a schematic diagram of an adaptive motion perception system based on the spatiotemporal optical information encoding method, which mainly includes a photoelectric sensor unit, a processing unit and a control unit. The changing optical signal caused by the movement of the object is the system input.
[0029] Figure 4 The circuit diagram of the adaptive motion sensing system based on the spatiotemporal optical information encoding method of this embodiment is given. The photoelectric sensing unit includes a distributed array composed of photoelectric devices with a certain interval, which are distributed near the object to be detected. Each photoelectric device serves as an information input channel and works simultaneously, detects the surrounding changing light signals of the moving object and converts them into light response current data, and outputs the light response current to the processing unit; the processing unit includes processing unit 1 and processing unit 2, and processing unit 1 includes a switch circuit. The switch circuit is an array composed of multiple switches including positions 0 and 1, which is used to receive the real-time light response current data of the photoelectric sensing unit, sum the light response current, and convert it into a voltage to give to the comparator circuit. The comparator circuit includes two comparators, each of which is preset with a reference voltage V for comparison. ref1 and V ref2, the comparator circuit outputs the result to the CPU of the control unit for detecting the object's motion rate; the processing unit 2 includes a 1T1R memristor RRAM array circuit and a peripheral ADC circuit, the input end of which is connected to the processing unit 1 through a transimpedance amplifier array circuit TIAARRAY, and the output is converted into a digital signal through a peripheral ADC circuit and transmitted to the CPU of the control unit for sensing the object's motion trajectory; the control unit includes a central processing unit CPU, a digital-to-analog converter DAC circuit and a voltage regulator LDO circuit, wherein the CPU controls the LDO circuit to provide a stable and small reading voltage for the photoelectric device array, and reads the light response current data in real time. In addition, the control unit reads the feedback signal output by the processing unit 1 through the comparator, and uses the DAC circuit to apply a control voltage to the photoelectric sensor unit to realize self-regulation of the light response current attenuation rate, thereby realizing accurate matching of the speed of the moving object and improving the accuracy of motion perception; the control unit reads the signal of the motion trajectory data output by the processing unit 2; wherein, the number of devices of the photoelectric device array constituting the photoelectric sensor unit, the number of switches of the switch circuit array, and the number of TIAs in the transimpedance amplifier array are consistent, and are connected accordingly.
[0030] Figure 5 A flowchart of the adaptive motion perception based on spatiotemporal optical information coding method is given. Figure 4 The system circuit uses a photoelectric sensing unit and a spatiotemporal optical information encoding method to encode the object's motion trajectory and motion rate into a light response current value through the short-term memory characteristics of the photoelectric device, and realizes adaptive motion perception through the processing unit and the control unit. The specific steps include:
[0031] 1) Spatiotemporal optical information encoding: Use distributed photoelectric sensing units to detect input optical signals, map the spatial information of moving objects onto different photoelectric devices, and use distributed photoelectric sensing units to map the spatiotemporal information of moving objects onto different optical device units. Assume that the area of the photoelectric sensing unit is D*D, the object movement rate is v, the number of pixels (photoelectric devices) occupied by the object is n, and the interval between photoelectric devices is d. The initial photoresponse current of the photoelectric device is I0, and the maximum photoresponse current of the photoelectric device is I max , the variable of light response is ΔI=I max-I0, where τ (the time decay constant of the optoelectronic device) is the time required for the photocurrent response to decay to 1 / e times the change in photocurrent response. During the initial rate detection, the sampling time is consistent with the time decay constant of the optoelectronic device. The distance traveled by the object within τ time is L = v * τ, where (L < D). The number of optoelectronic device responses caused by the object's motion trajectory is N = L / d + n. When the time decay constant of the phototransistor matches the object's motion rate, the photocurrent response of the pixel unit where the object is located at time t is the maximum value, while the photocurrent response value of the optoelectronic device at the initial time t0 position is ΔI / e. The photocurrent I of the optoelectronic device through which the object's motion trajectory passes from t0 to t satisfies (I0 < I < I max ). At this time, the total number of devices that generate photocurrent responses due to the moving object's trajectory is (L / d + n), and the sum of the device currents passed through is the reference value. Therefore, the object's motion trajectory and motion rate can be encoded as photocurrent response values through the short-term memory characteristics of the optoelectronic device unit.
[0032] 2) Motion perception and adaptive adjustment
[0033] 2-1) Detection and judgment of the object's motion rate. The reference rate of this rate detection system is selected as V2. There are three possible rates to be detected: the object's motion rate is equal to V2; the object's motion rate is greater than V2, and the object's motion rate is less than V2. To sample the complete object motion trajectory, the time decay constants of the optoelectronic devices matching them are τ3 < τ2 < τ1. At this time, the reference time decay constant of the device is τ2. At this time, the distance traveled by the object within one sampling time (τ2) is L ref = v2 * τ2. The sum of the current change amounts of (L ref / d + n) optoelectronic device responses within this distance is set as the reference value, that is multiplied by the resistance in processing unit 1 to be converted into voltage and provided to V ref1 and V ref2 . The motion rate of the object to be detected is v x . The distance traveled by the object within τ2 time is L real = v x * τ2. The total number of devices that generate photocurrent responses within this range (L real / d + n) is compared with (L ref / d + n). Specifically, the control unit uses the CPU to control the LDO circuit to apply a read voltage to all optoelectronic devices of the optoelectronic sensing unit simultaneously, and read out the photocurrent responses of the optoelectronic devices in parallel; use the switch circuit in processing unit 1 to place the switch at position 0, and sum up the photocurrent responses of all optoelectronic devices; multiply the total photocurrent response by the flowing resistance, convert the current into voltage and give it to the comparator. The comparator will compare the input voltage with the reference voltages V ref1 and Vref2 Compare, when the comparator circuit output value is 00, it proves that the object movement speed is less than v2; when the comparator circuit output value is 11, it proves that the object movement speed is greater than v2, and enter step 2-2) of adaptive adjustment; when the comparator circuit output value is 01, it proves that the object movement speed matches the light response current decay rate of the current photoelectric device, and enter step 2-3);
[0034] 2-2) Adaptive Adjustment of Moving Object Rate Detection
[0035] When the object movement rate does not match the decay rate of the photoresponse current of the photoelectric device, that is, the comparator circuit output in the processing unit 1 is 00 or 11, the control unit CPU uses the DAC circuit to adjust the gate voltage of the photoelectric device, thereby adjusting the decay rate of the photoresponse current: when the comparator circuit output in the processing unit 1 is 00, the moving object rate is too small, and the gate voltage of the photoelectric device is increased, that is, the time decay constant of the photoelectric device (τ1) is increased; when the comparator circuit output in the processing unit 1 is 11, the moving object rate is too large, and the gate voltage of the photoelectric device is reduced, that is, the time decay constant of the photoelectric device (τ3) is reduced; return to step 1); until the comparator output is 01, the next step of the moving object trajectory is inferred. In addition, while using the gate voltage to adjust the time decay constant of the photoelectric device, the CPU is used to control the LDO application sampling time to be consistent with the τ of the photoelectric device;
[0036] 2-3) Reasoning about the trajectory of moving objects
[0037] When the object's movement rate matches the photoresponse current decay rate of the current photoelectric device, that is, the comparator output in processing unit 1 is 01, the switch in processing unit 1 is set to 1; at this time, the photoresponse current of each photoelectric device is respectively connected to the corresponding transimpedance amplifier array circuit, and the transimpedance amplifier array circuit converts the current signal into a voltage signal, and then outputs it to the memristor RRAM array in processing unit 2, and performs parallel multiplication and addition operations with the weight values in the trained artificial neural network stored in the RRAM array, and outputs the motion trajectory information to the CPU of the control unit to realize the reasoning of the object's motion trajectory.
[0038] Adaptive perception of moving objects is achieved through the above method and system.
[0039] The adaptive motion perception system of the present invention fully utilizes the advantages of high precision and high parallelism of the above encoding method.
[0040] Finally, it should be noted that the purpose of publishing the embodiments is to help further understand the present invention, but those skilled in the art can understand that various substitutions and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the contents disclosed in the embodiments, and the scope of protection claimed by the present invention shall be subject to the scope defined in the claims.
Claims
1. A method for encoding spatiotemporal optical information, characterized in that: Distributed photoelectric sensing units are used to detect the input light signal and map the spatial information of the moving object onto different photoelectric devices. On this basis, the short-term memory characteristics of the photoelectric devices are used to encode the changes of the light signal in the time domain into a light response current.
2. The spatiotemporal photoelectric information encoding method according to claim 1, characterized in that: The photoelectric device in the photoelectric sensor unit has a short-term memory characteristic of light response current under the action of different gate voltages. When the light pulse acts, the light response current increases instantly. When the light pulse stops, the light response current slowly decays with the increase of time. The short-term memory characteristic of light response current is expressed as the following formula: I t =I0+ΔIexp(-t / τ) Among them I t is the response current of the photoelectric device at time t, I0 is the current value of the photoelectric device at the initial moment, ΔI is the current change of the photoelectric device, and τ is the time decay constant of the photoelectric device; the photoelectric device is a phototransistor device, or a device composed of a photoresistor and a capacitor; when it is a phototransistor device, the τ value is regulated by applying a gate voltage of the device; when it is a device composed of a photoresistor and a capacitor, the τ value is regulated by changing the capacitor value.
3. The spatiotemporal optical information encoding method according to claim 1, characterized in that: The specific method is: Set the area where the photoelectric sensing units are distributed as D*D, the moving speed of the object as v, the number of pixel points (the number of optoelectronic devices) occupied by the object as n, and the interval between optoelectronic devices as d; the initial light response current of the optoelectronic device is I0, and the maximum light response current of the optoelectronic device is I max , and the change in light response current is ΔI = I max - I0. When the light response current decays to 1 / e times of the change in light response current ΔI, the required time is the time decay constant τ of the optoelectronic device; during the initial speed detection, the sampling time is consistent with the time decay constant of the optoelectronic device. The distance moved by the object within τ time is L = v * τ, where L < D; the number of optoelectronic device responses caused by the object's movement trajectory is N = L / d + n; when the time decay constant of the optoelectronic device matches the moving speed of the object, the light response current of the pixel unit where the object is located at time t is the maximum value I max , and the light response current value of the optoelectronic device at the initial time t0 position is ΔI / e. The light response current I of the optoelectronic device through which the object's movement trajectory passes from t0 to t satisfies I0 < I < I max ; at this time, the sum of the light currents of the optoelectronic devices through which the moving object's trajectory passes is the reference value; therefore, the object's movement trajectory and moving speed are encoded as light response current values through the short-term memory characteristics of the optoelectronic device unit.
4. An adaptive motion sensing method, characterized in that: Using the spatiotemporal optical information encoding method as claimed in claim 1, for different moving object movement rates, in order to sample the complete object movement trajectory, set the reference rate and the corresponding time decay constant as the initial conditions, when the object movement rate reference value is v, the time decay constant τ of the photoelectric device matched therewith; at this time, the object moves a distance L within the fixed sampling time τ ref =v*τ, L within this distance ref The sum of the current changes of the responses of / d+n photoelectric devices is set as the reference value, that is, The moving speed of the object to be detected is v x , the distance moved by the object in τ time is L real =v x *τ, if the total number of devices that generate photoresponse within this range (L real / d+n)<(L ref / d+n), then the total photoresponse current Prove that the object's velocity v x Less than v, increase the device time decay constant τ; when (L real / d+n)>(L ref / d+n), then the total photoresponse current Prove that the object's velocity v x When the object's motion rate matches the photoresponse current decay rate of the photoelectric device, the photoresponse current of the photoelectric device is converted into a voltage and multiplied and added in parallel with the weight values in the artificial neural network trained by the processing unit to obtain the motion trajectory information.
5. An adaptive motion sensing system, characterized in that: It comprises a photoelectric sensing unit, a processing unit and a control unit; the photoelectric sensing unit is distributed near the object to be detected, collects the changing light information around the moving object and converts it into a light response current; the processing unit is used to receive the real-time light response current data of the photoelectric sensing unit, and senses and feeds back the object's movement speed and trajectory according to the dynamic light response current; the control unit controls and adjusts the light response current attenuation rate and sampling time of the photoelectric sensing unit according to the feedback signal of the processing unit; in the system, the adaptive motion sensing method as described in claim 4 is used to realize the light information encoding and adaptive motion sensing of the moving object.
6. The adaptive motion sensing system according to claim 5, characterized in that: The photoelectric sensing unit includes a distributed array composed of photoelectric devices with a certain interval, which are distributed near the object to be detected. Each photoelectric device serves as an information input channel and works simultaneously to detect the surrounding changing light signals of the moving object and convert them into light response current data, and output the light response current to the processing unit; the processing unit includes processing unit 1 and processing unit 2, and processing unit 1 includes a switch circuit. The switch circuit is an array composed of multiple switches including 0 and 1 positions, which is used to receive the real-time light response current data of the photoelectric sensing unit, sum the light response current, and convert it into a voltage to give to the comparator circuit. The comparator circuit includes two comparators, each of which is preset with a reference voltage V for comparison. ref1 and V ref2 , the comparator circuit outputs the result to the CPU of the control unit for detecting the object's motion rate; the processing unit 2 includes a 1T1R memristor RRAM array circuit and a peripheral ADC circuit, the input end of which is connected to the processing unit 1 through a transimpedance amplifier array circuit TIAARRAY, and the output is converted into a digital signal through a peripheral ADC circuit and transmitted to the CPU of the control unit for sensing the object's motion trajectory; the control unit includes a central processing unit CPU, a digital-to-analog converter DAC circuit and a voltage regulator LDO circuit, wherein the CPU controls the LDO circuit to provide a stable and small reading voltage for the photoelectric device array, and reads the light response current data in real time. In addition, the control unit reads the feedback signal output by the processing unit 1 through the comparator, and uses the DAC circuit to apply a control voltage to the photoelectric sensor unit to realize self-regulation of the light response current attenuation rate, thereby realizing accurate matching of the speed of the moving object and improving the accuracy of motion perception; the control unit reads the signal of the motion trajectory data output by the processing unit 2; wherein, the number of devices of the photoelectric device array constituting the photoelectric sensor unit, the number of switches of the switch circuit array, and the number of TIAs in the transimpedance amplifier array are consistent, and are connected accordingly.