A neuromorphic-based visual image acquisition method and system

Abstract

The application relates to the field of artificial intelligence, and discloses a neuromorphic-based visual image acquisition method and system, which comprises the following steps: firstly, a neuromorphic unit containing a pulse conversion and an event sensing module is configured, a light pulse sequence is obtained through pulse conversion, synchronous event stream is obtained to obtain synchronous event data, response delay and image fidelity are analyzed to calculate an overall energy efficiency ratio, image acquisition elements corresponding to the energy efficiency ratio are analyzed, acquisition hardware configuration is determined, and an image acquisition process is reconstructed, and based on the new process, acquisition is enhanced, after core demand protocols are checked, a complete acquisition report of visual images in a visual scene is output. The application can improve the acquisition efficiency of visual images.

Description

Technical Field

[0001] This invention relates to a neuromorphic visual image acquisition method and system, belonging to the field of artificial intelligence. Background Technology

[0002] Visual image acquisition, as a core component of computer vision systems, is widely used in fields such as real-time monitoring, autonomous driving, and intelligent sensing.

[0003] Currently, most mainstream image acquisition methods rely on traditional frame cameras, which perform global exposure and sampling of the scene at a fixed frame rate, and then output a continuous image sequence through an image sensor. However, when faced with visual scenes with high dynamic range, high speed motion, or drastic changes in lighting, such methods suffer from problems such as high data redundancy, large response latency, and low energy efficiency. They are difficult to achieve high-fidelity and low-latency image perception under low bandwidth and low power consumption conditions, which limits their practical application performance in edge computing and embedded vision systems. Summary of the Invention

[0004] This invention provides a neuromorphic visual image acquisition method and system, the main purpose of which is to improve the acquisition efficiency of visual images.

[0005] To achieve the above objectives, the present invention provides a neuromorphic-based visual image acquisition method, comprising: Configure a neuromorphic unit in a visual scene. The neuromorphic unit includes a pulse conversion module and an event perception module. Based on the pulse conversion module, pulse conversion is performed on the luminous thixotropic points in the visual scene to obtain a luminous pulse sequence. Synchronous control is performed on the event flow in the event perception module to obtain synchronous event data. Based on the synchronous event data, the response delay and image fidelity corresponding to the visual image in the visual scene are analyzed. Based on the response delay and the image fidelity, the overall energy efficiency ratio corresponding to the visual image in the visual scene is calculated. The image acquisition elements corresponding to the overall energy efficiency ratio are analyzed, and based on the image acquisition elements, the acquisition hardware configuration corresponding to the visual image is determined. Based on the hardware configuration, the image acquisition process corresponding to the visual image is reconstructed. Based on the image acquisition process, the visual image is acquired and enhanced to obtain the enhanced acquisition result. The core requirement protocol in the enhanced acquisition result is verified, and based on the core requirement protocol, a complete acquisition report corresponding to the visual image in the visual scene is output.

[0006] Optionally, the step of performing pulse conversion on the luminous thixotropic points in the visual scene based on the pulse conversion module to obtain a luminous pulse sequence includes: Based on the photosensitive element in the pulse conversion module, continuous illumination data of the visual scene is collected; The comparison circuit in the pulse conversion module is used to analyze the change in illumination at different times based on the sampled illumination data. Identify thixotropic points in the illumination variation that exceed a preset threshold; Based on the sequence generation circuit in the pulse conversion module, a bright thixotropic sequence corresponding to the bright thixotropic point is generated. The luminous thixotropic sequence is pulse-converted to obtain a luminous pulse sequence.

[0007] Optionally, generating the luminous thixotropic sequence corresponding to the luminous thixotropic point based on the sequence generation circuit in the pulse conversion module includes: Latch the point data corresponding to the bright thixotropic point; Convert the location data into standard address encoding; The standard address code is encoded and marked to obtain encoded mark data; The encoded tag data is input into the output buffer of the sequence generation circuit; Read the buffered data in the output buffer to generate the luminous thixotropic sequence corresponding to the luminous thixotropic point.

[0008] Optionally, the step of synchronizing the event flow in the event sensing module to obtain synchronized event data includes: Identify the event buffer queue in the event sensing module; Monitor the event indication frequency corresponding to the event stream in the event buffer queue; Based on the event indication frequency, calibrate the trigger interval corresponding to the internal clock unit in the event sensing module; Based on the calibrated trigger interval, extract the buffered events from the event buffer queue; The buffered events are synchronized to obtain synchronization event data.

[0009] Optionally, the step of analyzing the response latency and image fidelity corresponding to the visual image in the visual scene based on the synchronization event data includes: Parse the event timestamps and event data items in the synchronized event data; Analyze the average time difference corresponding to the event timestamps; Based on the average time difference, the response delay corresponding to the visual scene is determined; Map the scene event image corresponding to the event data item; Analyze the edge sharpness and signal-to-noise ratio of the scene event images; Based on the edge sharpness and the signal-to-noise ratio, the image fidelity corresponding to the visual image in the visual scene is determined.

[0010] Optionally, calculating the overall energy efficiency ratio corresponding to the visual image in the visual scene based on the response delay and the image fidelity includes: Based on the response delay, determine the consumed response time corresponding to the visual image in the visual scene; Based on the image fidelity, query the valid pixel values ​​in the visual image; Based on the consumed response time and the effective pixel value, analyze the acquisition and processing efficiency of the visual image in the visual scene; Obtain the energy consumption value of the neuromorphic unit during operation; Based on the acquisition and processing efficiency and the energy consumption value, the overall energy efficiency ratio corresponding to the visual image in the visual scene is calculated using the following formula: ; in, This represents the overall energy efficiency ratio corresponding to the visual images in the aforementioned visual scene. Indicates the image acquisition time. This represents the energy consumption value. This indicates the number of events in the synchronized event data. This represents the event index in the synchronization event data. Indicates the first Image fidelity corresponding to each event Indicates the first Response latency for each event

[0011] Optionally, the step of analyzing the image acquisition elements corresponding to the overall energy efficiency ratio includes: Analyze the energy efficiency components in the overall efficiency ratio; Based on the energy efficiency components, the light and shadow features in the visual scene are determined; Based on the light and shadow features, the acquisition synchronization range in the visual scene is defined; Based on the acquisition synchronization range, configure the operating mode corresponding to the neuromorphic unit in the visual scene; Based on the aforementioned working mode, the image acquisition elements in the visual scene are analyzed.

[0012] Optionally, the process of reconstructing the image acquisition process corresponding to the visual image based on the hardware configuration includes: Analyze the hardware parameters corresponding to the hardware configuration; Based on the hardware parameters, scan instructions are generated in the neuromorphic unit; Based on the scanning command, the image scanning period corresponding to the visual image is set; Extract the synchronization timing from the image scanning cycle; Based on the synchronization timing, the image acquisition process corresponding to the visual image is reconstructed.

[0013] Optionally, the step of enhancing the visual image based on the image acquisition process to obtain an enhanced acquisition result includes: Based on the image acquisition process, original image events in the visual scene are acquired; Synchronize the image event sequence corresponding to the original image event; Map the image event sequence to an image density map; Filter out image noise from the image density map to obtain a noise-filtered image; Based on the current background brightness, the noise-filtered image is acquired and enhanced to obtain the enhanced acquisition result.

[0014] To address the above problems, the present invention also provides a neuromorphic visual image acquisition system, the system comprising: A pulse conversion module is used to configure neuromorphic units in a visual scene. The neuromorphic unit includes a pulse conversion module and an event perception module. Based on the pulse conversion module, pulse conversion is performed on the luminous thixotropic points in the visual scene to obtain a luminous pulse sequence. The energy efficiency ratio calculation module is used to perform synchronous control on the event flow in the event perception module to obtain synchronous event data. Based on the synchronous event data, it analyzes the response delay and image fidelity corresponding to the visual image in the visual scene, and calculates the overall energy efficiency ratio corresponding to the visual image in the visual scene based on the response delay and the image fidelity. The process reconstruction module is used to parse the image acquisition elements corresponding to the overall energy efficiency ratio, determine the acquisition hardware configuration corresponding to the visual image based on the image acquisition elements, and reconstruct the image acquisition process corresponding to the visual image based on the hardware configuration. The report output module is used to enhance the visual image based on the image acquisition process, obtain the enhanced acquisition result, verify the core requirement protocol in the enhanced acquisition result, and output a complete acquisition report corresponding to the visual image in the visual scene based on the core requirement protocol.

[0015] Compared to the problems described in the background art, this invention, by configuring neuromorphic units in the visual scene, effectively reduces redundant data in image acquisition, significantly improves the response speed and image fidelity to changes in the visual scene, and fundamentally optimizes the energy efficiency and operational adaptability of the visual image acquisition system. Furthermore, by synchronously controlling the event flow in the event perception module to obtain synchronized event data, this invention eliminates timing deviations in the transmission and processing of different event signals, ensuring that all event data maintains consistency in the time dimension, avoiding analysis errors caused by data disorder, and guaranteeing subsequent acquisition optimization processes from a data perspective. In one step, this invention analyzes the image acquisition elements corresponding to the overall energy efficiency ratio, clarifying the influence weight of each acquisition stage on energy efficiency from an energy efficiency perspective. This maximizes energy efficiency while ensuring image quality and response speed, enhancing the system's adaptability and practicality in low-power scenarios. Finally, based on the image acquisition process, this invention enhances the visual image to obtain enhanced acquisition results. This can specifically strengthen key visual details of the scene, suppress invalid noise interference, significantly improve the image fidelity of the enhanced acquisition results, and ensure that the image more accurately reflects the true characteristics of the scene, further promoting the synergistic improvement of the acquisition system's performance and energy efficiency. Therefore, the neuromorphic visual image acquisition method and system provided by this invention can improve the acquisition efficiency of visual images. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a neuromorphic-based visual image acquisition method according to an embodiment of the present invention. Figure 2 This is a flowchart illustrating the optimization of acquisition energy efficiency in a neuromorphic visual image acquisition method according to an embodiment of the present invention. Figure 3 This is a schematic diagram of a module for implementing a neuromorphic visual image acquisition system according to an embodiment of the present invention.

[0017] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0018] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0019] This application provides a neuromorphic visual image acquisition method. The execution subject of this neuromorphic visual image acquisition method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the neuromorphic visual image acquisition method can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0020] Reference Figure 1 The diagram shown is a flowchart illustrating a neuromorphic-based visual image acquisition method according to an embodiment of the present invention. In this embodiment, the neuromorphic-based visual image acquisition method includes: S1. Configure the neuromorphic unit in the visual scene. The neuromorphic unit includes a pulse conversion module and an event perception module. Based on the pulse conversion module, pulse conversion is performed on the luminous thixotropic points in the visual scene to obtain a luminous pulse sequence.

[0021] This invention effectively reduces redundant data in image acquisition by configuring neuromorphic units in the visual scene, significantly improves the response speed and image fidelity to changes in the visual scene, and optimizes the energy efficiency and operational adaptability of the visual image acquisition system from a fundamental level.

[0022] The visual scene refers to the target environment that the visual image acquisition system needs to perceive and capture. This environment includes key elements such as light intensity, dynamic target motion state, and scene complexity. It is the core object and foundation for image acquisition and needs to adapt to the acquisition requirements of different application scenarios. For example, in the scenario of autonomous driving on urban roads, the visual scene covers the environment of strong daylight, weak nightlight, and sudden changes in light at tunnel entrances and exits. It also includes the real-time motion state of dynamic targets such as vehicles, pedestrians, and obstacles. These elements directly affect the fidelity and response efficiency of image acquisition. The neuromorphic unit refers to the core functional unit that simulates the working mechanism of the biological visual nervous system. It consists of a pulse conversion module and an event perception module and is used to solve the defects of traditional frame-based acquisition. It can accurately respond to changes in the visual scene. Optionally, the neuromorphic unit configured in the visual scene can be implemented by neuromorphic chip design methods, such as using the Intel Loihi neuromorphic development kit to define neuron parameters and connection weights to obtain the neuromorphic unit.

[0023] Specifically, the neuromorphic unit includes a pulse conversion module and an event perception module. The pulse conversion module is the core component of the neuromorphic unit responsible for light signal processing. Its core function is to capture light thixotropic points in the visual scene in real time and convert the light intensity change signals of these points into discrete light pulse sequences. For example, in intelligent traffic monitoring scenarios, when a vehicle passes through the monitored area and causes local light intensity changes, this module can accurately identify these light thixotropic points. The event perception module is a key component of the neuromorphic unit that realizes event flow management and data synchronization. Its main function is to monitor and synchronize various dynamic events generated in the visual scene in real time, such as target movement and sudden changes in illumination, and output synchronized event data with consistent timing. At the same time, it provides a data foundation for response delay and image fidelity analysis. For example, in autonomous driving visual acquisition scenarios, when dynamic events such as pedestrians crossing the road or adjacent vehicles changing lanes occur in front, this module can synchronize and integrate the signal streams corresponding to these events to ensure that the data of different events remain consistent in the time dimension.

[0024] Furthermore, based on the pulse conversion module, the present invention performs pulse conversion on the thixotropic points of light in the visual scene to obtain a light pulse sequence, which can accurately focus on the key areas of light intensity change in the scene, avoid the invalid sampling of the unchanging background by traditional frame acquisition, and ensure the accuracy and effectiveness of subsequent image acquisition and optimization.

[0025] The thixotropic point refers to a specific spatiotemporal point where the change in illumination exceeds a preset threshold. It represents the location and time of a significant change in light intensity in the visual scene and is the core object of subsequent data processing. This point can exclude slight changes in light intensity caused by dust obstruction in the environment and focus only on effective scene changes. For example, in factory workshop monitoring, when the movement of a robotic arm obstructs the light source, causing the local change in light intensity to exceed the preset brightness threshold, this location and time is the thixotropic point. The light pulse sequence refers to the set of discrete electrical pulse signals output by the sequence generation circuit after pulse encoding conversion of the thixotropic sequence. It only contains the key information of the thixotropic point. For example, in a drone inspection scenario, the changes in terrain shadows captured by the drone will form a thixotropic sequence. After pulse conversion, a light pulse sequence is generated, and each pulse corresponds to an effective thixotropic point.

[0026] As an embodiment of the present invention, the step of performing pulse conversion on luminous thixotropic points in the visual scene based on the pulse conversion module to obtain a luminous pulse sequence includes: acquiring continuous illumination data of the visual scene based on the photosensitive element in the pulse conversion module; analyzing the amount of illumination change of the sampled illumination data at different times using the comparison circuit in the pulse conversion module; identifying luminous thixotropic points in the amount of illumination change that exceed a preset threshold; generating a luminous thixotropic sequence corresponding to the luminous thixotropic point based on the sequence generation circuit in the pulse conversion module; and performing pulse conversion on the luminous thixotropic sequence to obtain a luminous pulse sequence.

[0027] The photosensitive element refers to the core component in the pulse conversion module responsible for collecting light signals. It can convert light signals in the visual scene into electrical signals in real time, providing raw input for subsequent light data processing. This element has high sensitivity and fast response characteristics, and can continuously sense changes in scene light intensity. For example, in an autonomous driving vision acquisition system, the CMOS photosensitive element can capture the light information of the road in front of the vehicle thousands of times per second, accurately converting light signals such as sunlight and vehicle headlights at different times into calculable electrical signals. The continuous light data refers to the time-stamped time-series light information continuously collected by the photosensitive element at preset time intervals, which includes the complete process of light intensity changes in the visual scene over time, rather than the intermittent data of traditional frame-based acquisition. For example, in a smart building monitoring scenario, the photosensitive element collects illumination information of the corridor area at specific times. The resulting continuous illumination data records all light intensity fluctuations, such as changes in natural light during the day and light switching at night, fully reflecting the dynamics of the scene's illumination. The comparison circuit is a key component in the pulse conversion module used to analyze the amount of illumination change. Its core function is to receive the continuous illumination data output by the photosensitive element, compare the illumination data at different timestamps using a preset algorithm, calculate and output the amount of illumination change between the two. For example, in an urban traffic monitoring scenario, the comparison circuit compares the road illumination data at two adjacent times in real time to quickly calculate the amount of illumination change within that time period. The change in illumination refers to the difference in light intensity at different times in continuous illumination data. Its quantification reflects the degree of fluctuation in light intensity in a visual scene and is a core indicator for judging whether the change in illumination has practical significance. The thixotropic sequence refers to the structured data set formed by the sequence generation circuit in the pulse conversion module, which arranges the identified thixotropic points in the order of timestamps. It contains the spatiotemporal information of the thixotropic points and the characteristics of light intensity changes. This sequence can completely reflect the dynamic process of effective illumination changes in a visual scene. For example, in intelligent monitoring of a park, people walking will cause multiple thixotropic points to appear along the way. The sequence generation circuit will integrate these points into a thixotropic sequence according to the walking time.

[0028] Furthermore, the acquisition of continuous illumination data of the visual scene can be achieved through high dynamic range imaging methods, such as using a Sony IMX290 sensor to capture brightness frame by frame to obtain continuous illumination data; the analysis of illumination changes at different times of the sampled illumination data can be achieved through frame difference calculation methods, such as using the absdiff function in the OpenCV library to calculate the brightness difference between adjacent frames to obtain illumination changes; the identification of thixotropic points in the illumination changes that exceed a preset threshold can be achieved through threshold comparison algorithms, such as using functions in Python to filter points that exceed a set threshold to obtain thixotropic points; the generation of thixotropic sequences corresponding to the thixotropic points can be achieved through time series sorting methods, such as organizing point data by timestamps based on the Series function of the Pandas library to obtain thixotropic sequences; the pulse conversion of the thixotropic sequences can be achieved through spiking neural network encoding methods, such as using the Brian simulator to rate-encode pulse events to obtain luminous pulse sequences.

[0029] In detail, as another embodiment of the present invention, the step of generating the luminous thixotropic sequence corresponding to the luminous thixotropic point based on the sequence generation circuit in the pulse conversion module includes: latching the point data corresponding to the luminous thixotropic point; converting the point data into a standard address code; encoding and marking the standard address code to obtain encoded marker data; inputting the encoded marker data into the output buffer in the sequence generation circuit; and reading the buffered data in the output buffer to generate the luminous thixotropic sequence corresponding to the luminous thixotropic point.

[0030] The location data refers to the core set of original information of the thixotropic points in the visual scene, including the pixel position of the thixotropic point, the timestamp of the occurrence, and the corresponding illumination change value. For example, in urban intersection monitoring, when a vehicle obstructs a traffic light, causing a thixotropic change, the location data will record the pixel coordinates, occurrence time, and illumination change of the thixotropic point. The standard address code refers to the standardized data format after converting non-uniform format information such as spatial coordinates and timestamps in the location data according to a preset encoding rule. For example, in a park intelligent monitoring system, the location data formats collected by different monitoring cameras may be different. By converting the pixel coordinates and timestamps of the thixotropic points into a unified encoding form, a standard address code is formed. The coded label data refers to the data formed by adding specific identification information to the standard address code. The identification content may include the thixotropic point type, data priority, etc. For example, in the autonomous driving vision acquisition scenario, the thixotropic point code caused by a pedestrian crossing is labeled with "pedestrian-high priority", and the thixotropic point code caused by the movement of tree shadows is labeled with "environment-low priority", forming a coded label. The data is recorded; this data helps subsequent modules quickly identify key thixotropic information, prioritize high-priority data, and improve overall processing efficiency. The output buffer is a component in the sequence generation circuit used to temporarily store coded marker data. It has the functions of temporary data storage and ordered output, which can avoid transmission congestion or loss caused by excessively fast data generation speed and ensure the stability of data transmission. For example, in the scenario of drone inspection, the drone generates hundreds of coded marker data per second when flying at high speed. The output buffer will temporarily store these data in the order of generation. The buffered data refers to the set of coded marker data temporarily stored in the output buffer. It maintains the generation time sequence and integrity of the coded marker data and is the direct data source for subsequent reading and generation of the light thixotropic sequence. For example, in the workshop production monitoring, the frequent movement of the robotic arm will generate continuous coded marker data. After these data enter the output buffer, they form buffered data. The buffered data will completely retain the marker information of each data such as "robotic arm occlusion - medium priority" and the generation order. When reading later, there is no need to reprocess the original data. An ordered sequence can be generated directly based on the buffered data, shortening the data processing cycle.

[0031] Furthermore, the latching of the point data corresponding to the luminous thixotropic point can be implemented using a digital trigger circuit, such as using a chip to latch the current point coordinates to obtain the point data; the conversion of the point data into a standard address code can be implemented using a priority encoder circuit, such as using an encoder to map the point to a binary address to obtain a standard address code; the encoding and marking of the standard address code can be implemented using a parity check method, such as applying an XOR operation to add a parity bit to the code to obtain the encoded marker data; the input of the encoded marker data to the output buffer in the sequence generation circuit can be implemented using a parallel loading method, such as writing data to the buffer through a register; the reading of the buffered data in the output buffer can be implemented using a microcontroller interface method, such as using the GPIO of an ARM series chip to read the buffer content to obtain the buffered data.

[0032] S2. Perform synchronous control on the event flow in the event perception module to obtain synchronous event data. Based on the synchronous event data, analyze the response delay and image fidelity corresponding to the visual image in the visual scene. Calculate the overall energy efficiency ratio corresponding to the visual image in the visual scene based on the response delay and the image fidelity.

[0033] This invention obtains synchronized event data by synchronously controlling the event flow in the event sensing module, which can eliminate the timing deviation of different event signals in transmission and processing, ensure that all event data are consistent in the time dimension, avoid analysis errors caused by data disorder, and guarantee subsequent acquisition and optimization from the data level.

[0034] The event stream refers to the continuous data sequence generated by various dynamic events in the visual scene, which is received in real time by the event perception module. It contains key information such as the time of occurrence, spatial location, and event type of the event, and is the core raw data for the event perception module to carry out synchronous control. The synchronous event data refers to the set of events with a unified time base and no time deviation, obtained by performing time-series calibration and redundancy filtering on the extracted buffer events. It contains standardized information of all valid events. For example, in the scenario of urban elevated monitoring, through synchronous control, the two types of buffer events, "vehicle lane change" and "guardrail shadow change", which originally had millisecond-level time differences, are calibrated into event data under the same time base to form synchronous event data.

[0035] As an embodiment of the present invention, the step of synchronizing the event stream in the event sensing module to obtain synchronized event data includes: identifying the event buffer queue in the event sensing module; monitoring the event indication frequency corresponding to the event stream in the event buffer queue; calibrating the trigger interval corresponding to the internal clock unit in the event sensing module according to the event indication frequency; extracting buffered events in the event buffer queue based on the calibrated trigger interval; and synchronizing the buffered events to obtain synchronized event data.

[0036] The event buffer queue refers to an ordered data storage structure in the event perception module used to temporarily store the stream of events to be processed. It has the function of receiving and temporarily storing event data according to the event generation sequence, avoiding data loss or congestion caused by the event generation speed exceeding the processing speed. For example, in an urban intersection monitoring scenario, events such as vehicle passage, pedestrian crossing, and traffic light changes occur intensively during the morning rush hour. The event buffer queue temporarily stores the coordinates, types, and time stamp data of these events in chronological order, ensuring that all event information can be completely obtained during subsequent synchronization control. The event indication frequency refers to the number of new events added to the event buffer queue per unit time, quantitatively reflecting the density of dynamic events in the visual scene. It is the core basis for judging event processing needs and calibrating the internal clock. The internal clock unit... This refers to the core component in the event perception module responsible for providing timing benchmarks and controlling the event extraction rhythm. Its output trigger interval directly determines the frequency of event extraction from the event buffer queue. The trigger interval refers to the time interval between extracting one event data from the event buffer queue under the control of the internal clock unit. Its length needs to be dynamically calibrated according to the event indication frequency to match the event processing requirements. The buffer event refers to a single event data that is temporarily stored in the event buffer queue and is waiting to be extracted and processed. It contains complete information such as the spatial coordinates of the event occurrence, timestamp, and event type. It is the direct processing object of synchronization control. For example, in the monitoring of a shopping mall parking lot, events such as vehicle entry, location search, and exit are temporarily stored in the queue in the form of buffer events. Each buffer event contains information such as vehicle location, time, and vehicle entry.

[0037] Furthermore, the identification of the event buffer queue in the event sensing module can be achieved through a circular buffer initialization method, such as configuring a dual-port BRAM module as a circular storage structure on the FPGA to obtain the event buffer queue; the monitoring of the event indication frequency corresponding to the event flow in the event buffer queue can be achieved through a digital counter circuit, such as using a counter chip to count the amount of events flowing in per unit time to obtain the event indication frequency; the calibration of the trigger interval corresponding to the internal clock unit in the event sensing module can be achieved through a phase-locked loop synchronization method, such as using a frequency synthesizer to adjust the clock output period to obtain the trigger interval; the extraction of buffered events in the event buffer queue can be achieved through a direct memory access controller, such as reading event data packets from the buffer through the DMA module of the STM chip to obtain the buffered events; the synchronization control of the buffered events can be achieved through a timestamp alignment algorithm, such as applying a unified time base to the event data through a precise time protocol to obtain synchronized event data.

[0038] Based on the synchronous event data, this invention analyzes the response delay and image fidelity of visual images in the visual scene, which can accurately locate the shortcomings of the current acquisition system in terms of response speed and information reproduction. At the same time, it lays a data foundation for analyzing image acquisition elements and determining hardware configuration, ensuring that subsequent optimization directions are in line with the actual needs of the system.

[0039] The visual image refers to an image carrier generated after the visual scene is processed by a neuromorphic acquisition system. It integrates light pulse sequences and synchronous event data, containing both spatial pixel information of the scene and temporal features associated with dynamic events. It can fully reflect the static details and dynamic changes of the scene. For example, in an autonomous driving scenario, this image can present static information such as lane lines and traffic signs within a certain range ahead, while also being a core carrier of temporal data associated with dynamic events such as vehicle lane changes and pedestrian crossings. The response delay refers to the time interval from the occurrence of a dynamic event in the visual scene to the acquisition and presentation of the corresponding visual image information by the system. For example, in a drone power line inspection scenario, the time interval between when abnormal heating of the line in the inspection area occurs and the system acquires and presents the visual image of the abnormal area is the response delay. The image fidelity refers to the degree to which the visual image reproduces the real features of the visual scene. It is determined by edge sharpness and signal-to-noise ratio, directly reflecting the image quality of the acquisition system. For example, in an autonomous driving scenario, if the visual image can accurately reproduce the edges of lane lines, has no obvious noise, and is highly consistent with the real road scene, then the image fidelity is high.

[0040] As an embodiment of the present invention, the step of analyzing the response latency and image fidelity corresponding to the visual image in the visual scene based on the synchronous event data includes: parsing the event timestamps and event data items in the synchronous event data; analyzing the average time difference corresponding to the event timestamps; determining the response latency corresponding to the visual scene based on the average time difference; mapping the scene event image corresponding to the event data item; analyzing the edge sharpness and signal-to-noise ratio corresponding to the scene event image; and determining the image fidelity corresponding to the visual image in the visual scene based on the edge sharpness and the signal-to-noise ratio.

[0041] The event timestamp refers to the time information in the synchronized event data that marks the specific moment of occurrence of each dynamic event. It is usually recorded in a high-precision format and is the core benchmark for judging the temporal relationship of events and calculating response latency. For example, in an autonomous driving vision acquisition scenario, when a pedestrian crosses the road ahead, the synchronized event data will add a timestamp to the event to accurately record the moment of occurrence. The event data item refers to the information unit in the synchronized event data that describes the specific attributes of a single dynamic event, including key parameters such as event type, spatial coordinates, and illumination change. It is the basic data for mapping scene event images. The average time difference is the average value calculated after statistically analyzing the time differences between the "event occurrence time" and the "corresponding image information presentation time" of multiple events. It is the core indicator for quantifying response latency. Scene event images refer to local or overall images that focus on a specific dynamic event, generated by mapping parameters such as spatial coordinates and illumination changes in event data items to basic image information of the visual scene. Edge sharpness refers to the clarity of the edge contours of objects in a scene event image, manifested as the contrast between the edge and the background and the continuity of the contour lines. It is a key indicator for measuring the ability to restore image details. For example, in a traffic sign monitoring scene, if the edge lines of a "speed limit 60" sign are continuous, unblurred, and clearly distinguishable from the background, then the edge sharpness is high; otherwise, the sharpness is low. The signal-to-noise ratio (SNR) refers to the ratio of effective visual signal to invalid noise in a scene event image. The higher the ratio, the more prominent the effective information in the image and the less noise interference. It is a core indicator reflecting the purity of the image.

[0042] Furthermore, parsing the event timestamps in the synchronized event data can be achieved using string splitting functions, such as using Python's `split` method to extract the timestamp field, thus obtaining the event timestamps; parsing the event data items in the synchronized event data can be achieved using a JSON parsing library, such as using the `RapidJSON` library to parse the event type and coordinate information, thus obtaining the event data items; analyzing the average time difference corresponding to the event timestamps can be achieved using statistical analysis algorithms, such as using the `mean` function of the NumPy library to calculate the average timestamp difference, thus obtaining the average time difference; determining the response latency corresponding to the visual scene can be achieved using time difference calculation methods, such as using the C++ `chrono` library to calculate the time interval between the event occurrence and image rendering, thus obtaining the average time difference. The response delay is considered. Mapping the scene event image corresponding to the event data item can be achieved using an image reconstruction algorithm, such as using OpenCV's `remap` function to map event coordinates to an image grid to obtain the scene event image. Analyzing the edge sharpness corresponding to the scene event image can be achieved using gradient operators, such as applying the Sobel operator to calculate the mean edge intensity to obtain the edge sharpness. Analyzing the signal-to-noise ratio corresponding to the scene event image can be achieved using frequency domain analysis methods, such as using MATLAB's `psnr` function to calculate the signal-to-noise power ratio to obtain the signal-to-noise ratio. Determining the image fidelity corresponding to the visual image in the visual scene can be achieved using a structural similarity index, such as using the SSIM algorithm to compare the original scene with the reconstructed image to obtain the image fidelity.

[0043] This invention calculates the overall energy efficiency ratio of the visual image in the visual scene based on the response delay and the image fidelity, which can correlate the dynamic performance indicators of the system with the image quality indicators and energy consumption, achieve optimal energy consumption, and improve the adaptability of the system in low-power scenarios.

[0044] The overall energy efficiency ratio refers to the effective pixel processing volume corresponding to a unit of energy consumption, calculated based on the acquisition and processing efficiency and energy consumption value. It is a core indicator for comprehensively measuring the system's "performance-energy consumption" balance. The higher the ratio, the more effective information the system can process under the same energy consumption, or the lower the energy consumption under the same processing volume.

[0045] As an embodiment of the present invention, the step of calculating the overall energy efficiency ratio corresponding to the visual image in the visual scene based on the response delay and the image fidelity includes: determining the consumed response time corresponding to the visual image in the visual scene based on the response delay; querying the effective pixel value in the visual image based on the image fidelity; analyzing the acquisition and processing efficiency corresponding to the visual image in the visual scene based on the consumed response time and the effective pixel value; obtaining the energy consumption value of the neuromorphic unit during operation; and calculating the overall energy efficiency ratio corresponding to the visual image in the visual scene based on the acquisition and processing efficiency and the energy consumption value.

[0046] The "response time" refers to the total time consumed from the acquisition of a visual image to the completion of processing, determined based on response latency. It includes the entire process of event perception, data conversion, and analysis, and is a key indicator for measuring system processing speed. For example, in intelligent traffic monitoring, the time from when a vehicle enters the monitoring area to when the system completes vehicle contour recognition and generates valid image data is the response time. The "effective pixel value" refers to the number of pixels selected from the visual image based on image fidelity that accurately reflect the key features of the visual scene. For example, in the visual image of an autonomous driving scenario, the pixels of key targets such as lane lines and traffic lights can accurately reproduce their actual shapes. The quantity refers to the effective pixel value; the acquisition and processing efficiency is calculated by combining the consumption response time and the effective pixel value, indicating the number of effective pixels processed by the system per unit time. It quantitatively reflects the system's ability to acquire and process effective image information per unit time and is a comprehensive indicator that balances speed and information effectiveness; the energy consumption value refers to the total energy consumed by the neuromorphic unit in the process of pulse conversion, event perception, data processing, etc., usually measured in electrical energy. It is a basic indicator for measuring the system's energy consumption level. For example, in a UAV inspection system, the electrical energy consumed by the neuromorphic unit in completing the image acquisition and processing of the power transmission line within a specific time is the energy consumption value.

[0047] Furthermore, determining the response time corresponding to the visual image in the visual scene can be achieved using a high-precision timer, such as using an Intel Xeon processor's timestamp counter to record the entire process time, thereby obtaining the response time; querying the effective pixel values ​​in the visual image can be achieved using a threshold segmentation method, such as applying the Otsu algorithm to binarize the image and then counting the number of non-zero pixels, thereby obtaining the effective pixel values; analyzing the acquisition and processing efficiency corresponding to the visual image in the visual scene can be achieved using a throughput calculation model, such as calculating the processing volume per unit time based on the ratio of the effective pixel values ​​to the response time, thereby obtaining the acquisition and processing efficiency; obtaining the energy consumption value of the neuromorphic unit during operation can be achieved using a power monitoring circuit, such as using a TI INA219 sensor to measure the chip's operating current and voltage, thereby obtaining the energy consumption value; calculating the overall energy efficiency ratio corresponding to the visual image in the visual scene can be achieved using the following formula.

[0048] In detail, as another embodiment of the present invention, the overall energy efficiency ratio corresponding to the visual image in the visual scene is calculated according to the acquisition and processing efficiency and the energy consumption value using the following formula: ; in, This represents the overall energy efficiency ratio (unit: ) corresponding to the visual images in the aforementioned visual scene. ), Indicates the image acquisition time (unit: seconds). The energy consumption value is expressed in units of: ), This indicates the number of events in the synchronized event data. This represents the event index in the synchronization event data. Indicates the first Image fidelity corresponding to each event Indicates the first The response latency (in seconds) for each event.

[0049] In detail, the overall energy efficiency ratio can represent the core indicator that comprehensively measures "energy consumption - time efficiency - image quality" in the visual image acquisition and processing process, reflecting the system's effective processing capability of visual scene information per unit of energy consumption; the image acquisition time can represent the total time from the triggering of the first dynamic event in the visual scene to the system completing the acquisition of all visual images in that scene, reflecting the time span of the acquisition process. For example, in an urban traffic intersection scene, the time taken from when the first car enters the intersection to when the system completes the acquisition of visual images of all vehicles, pedestrians, and traffic lights at the intersection is T=3s, which is a key parameter in the time dimension when calculating the energy efficiency ratio; the energy consumption value can represent the neuromorphic unit The total electrical energy consumed in the entire process of a single visual image acquisition and processing quantifies the energy consumption cost of the system operation. For example, in the acquisition and processing process of the traffic intersection scene mentioned above, the overall electrical energy consumed by the neuromorphic unit chip, perception module, etc., is C=6J. Combined with the acquisition time T, it can reflect the efficiency relationship of "time-energy consumption". The image fidelity can represent the degree of restoration of the real scene features of the i-th event in the synchronous event data of the visual image part. It is usually represented by a value of 0 to 1. The closer it is to 1, the higher the fidelity. For example, in the traffic intersection scene, the i=2th "pedestrian crossing the road" event, the corresponding image can clearly restore the pedestrian's clothing texture and movement posture. =0.95; This parameter reflects the high fidelity of image quality in relation to energy efficiency; The response delay can represent the time interval from the "occurrence time" of the i-th event in the synchronous event data to "its corresponding visual image information being effectively acquired and presented by the system", reflecting the system's real-time response capability to dynamic events. For example, the occurrence time of the above-mentioned "pedestrian crossing the road" event is... The system displays a clear image of the pedestrian at the time when +0.2 s, then =0.2s; the lower the delay, the more valid events the system can process within a unit acquisition time.

[0050] Specifically, for a more intuitive understanding of the execution logic and data flow relationships corresponding to the visual image acquisition energy efficiency optimization process in this solution, please refer to [reference needed]. Figure 2 ,Should Figure 2 As the core process framework of the visual image acquisition energy efficiency optimization system, it clearly presents the complete link from event flow input to energy efficiency optimization and strategy output: The input layer focuses on the dynamic event flow captured by the event perception module, which is the basis for subsequent synchronization control and energy efficiency analysis; the processing layer transforms the raw event data into executable energy efficiency optimization and acquisition strategies through a step-by-step logic of "event buffer queue → synchronization control processing → synchronization event data → response latency / image fidelity analysis → acquisition processing efficiency → overall energy efficiency ratio calculation → decision and strategy center"; the output layer takes "optimized acquisition process and enhanced results" as the final result.

[0051] S3. Analyze the image acquisition elements corresponding to the overall energy efficiency ratio, determine the acquisition hardware configuration corresponding to the visual image based on the image acquisition elements, and reconstruct the image acquisition process corresponding to the visual image based on the hardware configuration.

[0052] This invention analyzes the image acquisition elements corresponding to the overall energy efficiency ratio, thereby clarifying the influence weight of each acquisition link on energy efficiency from the perspective of energy efficiency. It maximizes energy efficiency while ensuring image quality and response speed, and enhances the adaptability and practicality of the system in low-power scenarios.

[0053] The image acquisition elements refer to the set of core factors that affect the quality and energy efficiency of visual image acquisition, as analyzed based on the operating mode. These include hardware parameters, software strategies, timing control, etc. For example, in an autonomous driving scenario, the image acquisition elements obtained from the analysis may include: photosensitive element response speed, event synchronization trigger threshold, and pulse sequence encoding format. These elements directly determine the adaptability and energy efficiency of the acquisition system.

[0054] As an embodiment of the present invention, the step of parsing the image acquisition elements corresponding to the overall energy efficiency ratio includes: analyzing the energy efficiency components in the overall energy efficiency ratio; determining the light and shadow features in the visual scene based on the energy efficiency components; defining the acquisition synchronization range in the visual scene based on the light and shadow features; configuring the operating mode corresponding to the neuromorphic unit in the visual scene based on the acquisition synchronization range; and parsing the image acquisition elements in the visual scene based on the operating mode.

[0055] The energy efficiency components refer to the set of key elements constituting the overall energy efficiency ratio, including the distribution of energy consumption, effective information processing efficiency, and time cost ratio, which form the basis for analyzing energy efficiency composition. The light and shadow characteristics refer to key lighting attributes in the visual scene, such as light intensity, light intensity change frequency, shadow distribution, and dynamic light and shadow trajectories, derived from the analysis of the energy efficiency components. These directly affect the energy consumption and information processing efficiency of neuromorphic units. The acquisition synchronization range refers to the visual scene area and time interval that require precise synchronous acquisition, defined based on the light and shadow characteristics, ensuring that within this range... The thixotropic points of light and dynamic events can be captured synchronously, avoiding increased energy consumption due to excessive range or information loss due to insufficient range. The operating mode refers to the working state configured for the neuromorphic unit based on the acquisition synchronization range, including parameter combinations such as pulse conversion frequency, event perception sensitivity, and energy consumption control strategy, so that the unit can adapt to the acquisition needs of specific scenarios. For example, during off-peak hours on urban roads, the neuromorphic unit adopts a "low-power mode": reducing the pulse conversion frequency and lowering the event perception sensitivity; while during peak hours, it switches to a "high-speed response mode" to increase the frequency and sensitivity, thus optimizing energy efficiency through mode adaptation.

[0056] Furthermore, the analysis of the energy efficiency components in the overall efficiency ratio can be achieved through principal component analysis, such as using MATLAB's pca function to extract the main feature vectors from the energy consumption dataset to obtain the energy efficiency components; the determination of the light and shadow features in the visual scene can be achieved through optical flow field calculation, such as using OpenCV functions to analyze light motion vectors to obtain light and shadow features; the delineation of the acquisition synchronization range in the visual scene can be achieved through time window division, such as using a GPS clock synchronization module to determine a unified acquisition time interval to obtain the acquisition synchronization range; the configuration of the operating mode corresponding to the neuromorphic unit in the visual scene can be achieved through state machine programming, such as using TensorFlow to configure neuron activation mode parameters to obtain the operating mode; the parsing of the image acquisition elements in the visual scene can be achieved through metadata extraction, such as using the ExifTool library to read image resolution and color depth information to obtain image acquisition elements.

[0057] Based on the image acquisition elements, this invention determines the acquisition hardware configuration corresponding to the visual image, which can accurately adapt the hardware parameters to the light and shadow characteristics of the visual scene, the acquisition synchronization range and other requirements, and at the same time match the operating mode of the neuromorphic unit, thereby optimizing energy consumption and improving the overall energy efficiency ratio while ensuring the image acquisition quality.

[0058] The acquisition hardware configuration refers to the hardware components and parameter combinations that support visual image acquisition, determined based on image acquisition elements. This includes the selection of core components and the setting of operating parameters, which must be adapted to the light and shadow characteristics of the visual scene and the operating mode of the neuromorphic unit. Optionally, the acquisition hardware configuration corresponding to the visual image can be determined through performance benchmark testing methods, such as using NVIDIA tools to analyze the compatibility parameters of the sensor and processor to obtain the acquisition hardware configuration.

[0059] Furthermore, based on the aforementioned hardware configuration, the present invention reconstructs the image acquisition process corresponding to the visual image, which can eliminate the compatibility deviation between hardware performance and the original acquisition process, avoid idle or overloaded hardware resources, ensure that the hardware capabilities are fully utilized to support acquisition needs, guarantee image fidelity while further compressing response latency, and improve image acquisition efficiency.

[0060] The image acquisition process refers to the complete workflow of visual image processing from scene scanning, signal conversion, event perception to data output, based on synchronous temporal reconstruction. It covers the collaborative logic and operation steps of various hardware modules and is the specific execution path for hardware configuration implementation. For example, in the scenario of intelligent building monitoring, the reconstructed acquisition process is as follows: scan the public area of ​​the building at a specific cycle → the pulse conversion module converts the light thixotropic points into a pulse sequence → the event perception module synchronously analyzes the event flow → outputs structured image data.

[0061] As an embodiment of the present invention, the step of reconstructing the image acquisition process corresponding to the visual image based on the hardware configuration includes: analyzing the hardware parameters corresponding to the hardware configuration; generating scanning instructions in the neuromorphic unit based on the hardware parameters; setting the image scanning cycle corresponding to the visual image based on the scanning instructions; extracting the synchronization timing in the image scanning cycle; and reconstructing the image acquisition process corresponding to the visual image based on the synchronization timing.

[0062] The hardware parameters refer to the core performance indicators and operating settings of each hardware component in the acquisition hardware configuration, including the response speed of the photosensitive element, the processing frequency of the pulse conversion chip, and the synchronization accuracy of the event perception module, which are the basis for generating scanning instructions. The scanning instructions are operation instructions generated based on the hardware parameters to control the neuromorphic unit to perform image scanning. They include key information such as the scanning area, scanning frequency, and data acquisition priority, guiding the hardware to complete scene scanning as required. The image scanning cycle refers to the time interval set based on the scanning instructions for the neuromorphic unit to complete one complete scan of the visual scene, reflecting the scanning frequency and rhythm. It needs to match the hardware processing capability and the scene event density. For example, in the morning rush hour monitoring of urban intersections, the image scanning cycle is set to a specific time period according to the scanning instructions, that is, a full-area scan of the intersection is completed once every specific time period. The synchronization timing refers to the time reference extracted from the image scanning cycle to coordinate the work of each module of the neuromorphic unit, ensuring that each module completes data acquisition, processing, and transmission at a unified rhythm within the scanning cycle, and ensuring the integrity and consistency of the data within the scanning cycle.

[0063] Furthermore, the analysis of the hardware parameters corresponding to the hardware configuration can be achieved through system diagnostic tools, such as using HWiNFO software to read sensor sampling rate and interface bandwidth data to obtain hardware parameters; the generation of scanning instructions in the neuromorphic unit can be achieved through G-code programming methods, such as using a GRBL controller to generate a two-dimensional scanning path instruction set to obtain scanning instructions; the setting of the image scanning period corresponding to the visual image can be achieved through timer interrupt configuration, such as configuring a SysTick timer in the ARM core to generate a fixed-interval trigger signal to obtain the image scanning period; the extraction of synchronization timing in the image scanning period can be achieved through a phase-locked loop, such as using a chip to extract the rising edge synchronization pulse of the clock signal to obtain the synchronization timing; the reconstruction of the image acquisition process corresponding to the visual image can be achieved through a multi-threaded scheduling algorithm, such as using the FreeRTOS task manager to reallocate image acquisition thread resources to obtain the image acquisition process.

[0064] S4. Based on the image acquisition process, the visual image is acquired and enhanced to obtain the enhanced acquisition result. The core requirement protocol in the enhanced acquisition result is checked, and based on the core requirement protocol, a complete acquisition report corresponding to the visual image in the visual scene is output.

[0065] Based on the image acquisition process, this invention enhances the visual image to obtain enhanced acquisition results. It can specifically enhance key visual details of the scene, suppress invalid noise interference, significantly improve the image fidelity of the enhanced acquisition results, ensure that the image can more accurately reflect the real features of the scene, and further promote the synergistic improvement of the performance and energy efficiency of the acquisition system.

[0066] The enhanced acquisition result refers to a high-quality image obtained by specifically optimizing the noise-filtered image based on the background brightness of the current visual scene. This image can clearly present scene details. For example, in the monitoring of urban intersections in the evening backlight, the noise-filtered image has problems such as the vehicle area being too dark and the license plate being blurry. After adjusting the background brightness, the enhanced acquisition result will brighten the vehicle area and enhance the contrast of the license plate, making the license plate characters clearly distinguishable, and providing high-quality visual data support for subsequent vehicle recognition and event analysis.

[0067] As an embodiment of the present invention, the step of enhancing the visual image based on the image acquisition process to obtain an enhanced acquisition result includes: acquiring original image events in the visual scene based on the image acquisition process; synchronizing the image event sequence corresponding to the original image events; mapping the image event sequence to an image density map; filtering out image noise in the image density map to obtain a denoised image; and enhancing the denoised image based on the current background brightness to obtain an enhanced acquisition result.

[0068] The original image events refer to unprocessed dynamic visual information units directly acquired from the visual scene based on the image acquisition process. These include the spatial location, timestamp, and basic visual features of the event. For example, in a shopping mall shelf monitoring scenario, original image events might be represented as "customer's hand touching shelf merchandise" or "merchandise moving from the shelf to the shopping basket." These unprocessed events directly reflect the scene's dynamics, providing a basis for synchronously generating an image event sequence. The image event sequence refers to a structured time-series set formed by arranging the original image events in time-stamp order after synchronous processing. It completely records the occurrence order and correlation of dynamic events in the visual scene, eliminating temporal deviations in the original events. The image density map refers to a visual image reflecting the spatial distribution density of events, generated by statistically analyzing the image event sequence according to the spatial location of the events. Pixel values ​​represent the frequency of events occurring within the corresponding spatial area. For example, in a shopping mall monitoring image event sequence, the mapped image density map displays the shelf area with high brightness and the channel area with low brightness, intuitively presenting the concentrated event areas. The noise-filtered image refers to a clean image obtained by filtering out invalid noise from the image density map using a preset algorithm, retaining the density distribution characteristics corresponding to valid events.

[0069] Furthermore, the acquisition of original image events in the visual scene can be achieved through an event camera sensing method, such as using a PM sensor to capture event stream data based on pixel brightness changes, thereby obtaining original image events; the synchronization of the image event sequence corresponding to the original image events can be achieved through a timestamp alignment algorithm, such as applying a unified time reference to multi-source event data using the IEEE 1588 Precision Time Protocol, thereby obtaining an image event sequence; the mapping of the image event sequence to an image density map can be achieved through a histogram statistical method, such as using OpenCV functions to generate an event count spatial distribution map, thereby obtaining an image density map; the filtering of image noise in the image density map can be achieved through a median filtering algorithm, such as using OpenCV functions to eliminate impulse noise interference, thereby obtaining a denoised image; the acquisition enhancement of the denoised image can be achieved through a contrast-limited adaptive histogram equalization method, such as using the CLAHE algorithm to enhance the local contrast features of the image, thereby obtaining an enhanced acquisition result.

[0070] By verifying the core requirement protocols in the enhanced acquisition results, this invention can ensure that the enhanced results are highly consistent with the preset acquisition targets, avoid acquisition function failure due to the results deviating from the core requirements, and ensure the stability of the acquisition system.

[0071] The core requirement protocol refers to a pre-defined specification document that constrains the entire acquisition process to ensure that visual image acquisition meets the application goals of the scenario. It includes three key components: core performance indicators, functional coordination requirements, and data output standards. It is the core basis for verifying whether the enhanced acquisition results meet the standards. Optionally, the verification of the core requirement protocol in the enhanced acquisition results can be achieved through protocol consistency verification methods, such as using the Wireshark network analysis tool to detect packet conformity, thereby obtaining the core requirement protocol.

[0072] Furthermore, based on the core requirement protocol, this invention outputs a complete acquisition report corresponding to the visual images in the visual scene, which can comprehensively quantify the degree of conformity between the entire acquisition process and the protocol requirements. By clearly defining the compliance status of key indicators such as response latency and image fidelity, the stability of image acquisition is guaranteed.

[0073] The complete acquisition report refers to a standardized document that presents the entire process of visual image acquisition based on the core requirements protocol. It includes the background of the acquisition scenario, an overview of the hardware configuration, key steps in the acquisition process, enhanced result features, and analysis of the fit between each step and the protocol requirements, deviation explanations, and optimization directions. It is a concentrated reflection of the validity and compliance of the acquisition results. For example, in a smart factory monitoring scenario, the report may cover: the scenario is a key equipment area on the production line, the hardware configuration is adapted to high-speed event acquisition, the response performance meets the real-time requirements, the image quality meets the detail recognition standard, the matching status with the protocol requirements, and targeted energy efficiency optimization suggestions. It provides a complete basis for the evaluation and iteration of the acquisition system. Optionally, the output of the complete acquisition report corresponding to the visual images in the visual scenario can be achieved through a report generation template, such as using the Apache POI library to automatically generate a PDF document containing image parameters and acquisition indicators, thereby obtaining a complete acquisition report.

[0074] Specifically, the output of the complete acquisition report first systematically presents key information of the entire visual image acquisition process, including hardware configuration parameters, core performance indicators, and the degree of matching between each stage and the core requirement protocol. This allows for an intuitive understanding of the acquisition system's operating status and compliance, eliminating the need to trace scattered acquisition data one by one. Secondly, it helps to quickly locate potential problems in the acquisition process. For example, if the image fidelity does not meet expectations, the report can link it to noise filtering parameters or hardware sensitivity settings, providing clear direction for targeted adjustments and avoiding wasting time on blind troubleshooting. Furthermore, the report can serve as a core basis for subsequent decisions. Whether it's optimizing the acquisition process, adjusting hardware configuration, or reusing the acquisition solution in similar scenarios, its recorded historical data and analysis conclusions can reduce decision-making risks. At the same time, the standardized report format can form a traceable acquisition archive, ensuring the continuous stability of the acquisition work.

[0075] Compared to the problems described in the background art, this invention, by configuring neuromorphic units in the visual scene, effectively reduces redundant data in image acquisition, significantly improves the response speed and image fidelity to changes in the visual scene, and fundamentally optimizes the energy efficiency and operational adaptability of the visual image acquisition system. Furthermore, by synchronously controlling the event flow in the event perception module to obtain synchronized event data, this invention eliminates timing deviations in the transmission and processing of different event signals, ensuring that all event data maintains consistency in the time dimension, avoiding analysis errors caused by data disorder, and guaranteeing subsequent acquisition optimization processes from a data perspective. In one step, this invention analyzes the image acquisition elements corresponding to the overall energy efficiency ratio, clarifying the influence weight of each acquisition stage on energy efficiency from an energy efficiency perspective. This maximizes energy efficiency while ensuring image quality and response speed, enhancing the system's adaptability and practicality in low-power scenarios. Finally, based on the image acquisition process, this invention enhances the visual image to obtain enhanced acquisition results. This can specifically strengthen key visual details of the scene, suppress invalid noise interference, significantly improve the image fidelity of the enhanced acquisition results, and ensure that the image more accurately reflects the true characteristics of the scene, further promoting the synergistic improvement of the acquisition system's performance and energy efficiency. Therefore, the neuromorphic visual image acquisition method and system provided by this invention can improve the acquisition efficiency of visual images.

[0076] like Figure 3 The diagram shown is a functional block diagram of a neuromorphic visual image acquisition system according to the present invention.

[0077] The neuromorphic visual image acquisition system 200 described in this invention can be installed in an electronic device. Depending on the functions implemented, the neuromorphic visual image acquisition system may include a pulse conversion module 201, an energy efficiency ratio calculation module 202, a process reconstruction module 203, and a report output module 204. The module described in this invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0078] In this embodiment of the invention, the functions of each module / unit are as follows: The pulse conversion module 201 is used to configure the neuromorphic unit in the visual scene. The neuromorphic unit includes the pulse conversion module and the event perception module. Based on the pulse conversion module, the pulse conversion is performed on the luminous thixotropic points in the visual scene to obtain a luminous pulse sequence. The energy efficiency ratio calculation module 202 is used to perform synchronous control on the event flow in the event perception module to obtain synchronous event data. Based on the synchronous event data, it analyzes the response delay and image fidelity corresponding to the visual image in the visual scene, and calculates the overall energy efficiency ratio corresponding to the visual image in the visual scene based on the response delay and the image fidelity. The process reconstruction module 203 is used to parse the image acquisition elements corresponding to the overall energy efficiency ratio, determine the acquisition hardware configuration corresponding to the visual image based on the image acquisition elements, and reconstruct the image acquisition process corresponding to the visual image based on the hardware configuration. The report output module 204 is used to enhance the visual image based on the image acquisition process, obtain the enhanced acquisition result, verify the core requirement protocol in the enhanced acquisition result, and output a complete acquisition report corresponding to the visual image in the visual scene based on the core requirement protocol.

[0079] In detail, the modules in the neuromorphic visual image acquisition system 200 described in this embodiment of the invention employ the same methods as described above. Figure 1 The technique used is the same as that described in the neuromorphic visual image acquisition method, and can produce the same technical effect, so it will not be repeated here.

[0080] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. In the above multiple embodiments, each embodiment can be combined with each other or independent. Deleting any one of them will not affect the technical implementation of other embodiments. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for acquiring visual images based on neuromorphology, characterized in that, The method includes: Configure a neuromorphic unit in a visual scene. The neuromorphic unit includes a pulse conversion module and an event perception module. Based on the pulse conversion module, pulse conversion is performed on the luminous thixotropic points in the visual scene to obtain a luminous pulse sequence. Synchronous control is performed on the event flow in the event perception module to obtain synchronous event data. Based on the synchronous event data, the response delay and image fidelity corresponding to the visual image in the visual scene are analyzed. Based on the response delay and the image fidelity, the overall energy efficiency ratio corresponding to the visual image in the visual scene is calculated. The image acquisition elements corresponding to the overall energy efficiency ratio are analyzed, and based on the image acquisition elements, the acquisition hardware configuration corresponding to the visual image is determined. Based on the hardware configuration, the image acquisition process corresponding to the visual image is reconstructed. Based on the image acquisition process, the visual image is acquired and enhanced to obtain the enhanced acquisition result. The core requirement protocol in the enhanced acquisition result is verified, and based on the core requirement protocol, a complete acquisition report corresponding to the visual image in the visual scene is output.

2. The neuromorphic visual image acquisition method as described in claim 1, characterized in that, The step of performing pulse conversion on the luminous thixotropic points in the visual scene based on the pulse conversion module to obtain a luminous pulse sequence includes: Based on the photosensitive element in the pulse conversion module, continuous illumination data of the visual scene is collected; The comparison circuit in the pulse conversion module is used to analyze the change in illumination at different times based on the sampled illumination data. Identify thixotropic points in the illumination variation that exceed a preset threshold; Based on the sequence generation circuit in the pulse conversion module, a bright thixotropic sequence corresponding to the bright thixotropic point is generated. The luminous thixotropic sequence is pulse-converted to obtain a luminous pulse sequence.

3. The neuromorphic visual image acquisition method as described in claim 2, characterized in that, The step of generating the luminous thixotropic sequence corresponding to the luminous thixotropic point based on the sequence generation circuit in the pulse conversion module includes: Latch the point data corresponding to the bright thixotropic point; Convert the location data into standard address encoding; The standard address code is encoded and marked to obtain encoded mark data; The encoded tag data is input into the output buffer of the sequence generation circuit; Read the buffered data in the output buffer to generate the luminous thixotropic sequence corresponding to the luminous thixotropic point.

4. The neuromorphic visual image acquisition method as described in claim 1, characterized in that, The synchronization control of the event flow in the event sensing module to obtain synchronized event data includes: Identify the event buffer queue in the event sensing module; Monitor the event indication frequency corresponding to the event stream in the event buffer queue; Based on the event indication frequency, calibrate the trigger interval corresponding to the internal clock unit in the event sensing module; Based on the calibrated trigger interval, extract the buffered events from the event buffer queue; The buffered events are synchronized to obtain synchronization event data.

5. The neuromorphic visual image acquisition method as described in claim 1, characterized in that, The step of analyzing the response latency and image fidelity of visual images in the visual scene based on the synchronization event data includes: Parse the event timestamps and event data items in the synchronized event data; Analyze the average time difference corresponding to the event timestamps; Based on the average time difference, the response delay corresponding to the visual scene is determined; Map the scene event image corresponding to the event data item; Analyze the edge sharpness and signal-to-noise ratio of the scene event images; Based on the edge sharpness and the signal-to-noise ratio, the image fidelity corresponding to the visual image in the visual scene is determined.

6. The neuromorphic visual image acquisition method as described in claim 1, characterized in that, The step of calculating the overall energy efficiency ratio of the visual image in the visual scene based on the response latency and the image fidelity includes: Based on the response delay, determine the consumed response time corresponding to the visual image in the visual scene; Based on the image fidelity, query the valid pixel values ​​in the visual image; Based on the consumed response time and the effective pixel value, analyze the acquisition and processing efficiency of the visual image in the visual scene; Obtain the energy consumption value of the neuromorphic unit during operation; Based on the acquisition and processing efficiency and the energy consumption value, the overall energy efficiency ratio corresponding to the visual image in the visual scene is calculated using the following formula: ； in, This represents the overall energy efficiency ratio corresponding to the visual images in the aforementioned visual scene. Indicates the image acquisition time. This represents the energy consumption value. This indicates the number of events in the synchronized event data. This represents the event index in the synchronization event data. Indicates the first Image fidelity corresponding to each event Indicates the first The response delay corresponding to each event.

7. The neuromorphic visual image acquisition method as described in claim 1, characterized in that, The process of analyzing the image acquisition elements corresponding to the overall energy efficiency ratio includes: Analyze the energy efficiency components in the overall efficiency ratio; Based on the energy efficiency components, the light and shadow features in the visual scene are determined; Based on the light and shadow features, the acquisition synchronization range in the visual scene is defined; Based on the acquisition synchronization range, configure the operating mode corresponding to the neuromorphic unit in the visual scene; Based on the aforementioned working mode, the image acquisition elements in the visual scene are analyzed.

8. The neuromorphic visual image acquisition method as described in claim 1, characterized in that, The process of reconstructing the image acquisition process corresponding to the visual image based on the hardware configuration includes: Analyze the hardware parameters corresponding to the hardware configuration; Based on the hardware parameters, scan instructions are generated in the neuromorphic unit; Based on the scanning command, the image scanning period corresponding to the visual image is set; Extract the synchronization timing from the image scanning cycle; Based on the synchronization timing, the image acquisition process corresponding to the visual image is reconstructed.

9. The neuromorphic visual image acquisition method as described in claim 1, characterized in that, The process of enhancing the visual image based on the image acquisition process to obtain an enhanced acquisition result includes: Based on the image acquisition process, original image events in the visual scene are acquired; Synchronize the image event sequence corresponding to the original image event; Map the image event sequence to an image density map; Filter out image noise from the image density map to obtain a noise-filtered image; Based on the current background brightness, the noise-filtered image is acquired and enhanced to obtain the enhanced acquisition result.

10. A neuromorphic visual image acquisition system, characterized in that, The system is used to perform a neuromorphic-based visual image acquisition method as described in any one of claims 1-9, the system comprising: A pulse conversion module is used to configure neuromorphic units in a visual scene. The neuromorphic unit includes a pulse conversion module and an event perception module. Based on the pulse conversion module, pulse conversion is performed on the luminous thixotropic points in the visual scene to obtain a luminous pulse sequence. The energy efficiency ratio calculation module is used to perform synchronous control on the event flow in the event perception module to obtain synchronous event data. Based on the synchronous event data, it analyzes the response delay and image fidelity corresponding to the visual image in the visual scene, and calculates the overall energy efficiency ratio corresponding to the visual image in the visual scene based on the response delay and the image fidelity. The process reconstruction module is used to parse the image acquisition elements corresponding to the overall energy efficiency ratio, determine the acquisition hardware configuration corresponding to the visual image based on the image acquisition elements, and reconstruct the image acquisition process corresponding to the visual image based on the hardware configuration. The report output module is used to enhance the visual image based on the image acquisition process, obtain the enhanced acquisition result, verify the core requirement protocol in the enhanced acquisition result, and output a complete acquisition report corresponding to the visual image in the visual scene based on the core requirement protocol.

Images