Pulse visual signal image reconstruction and target detection combined processing method and related equipment

Abstract

The embodiment of the invention provides a pulse visual signal image reconstruction and target detection combined processing method and related equipment, and the method comprises the steps: obtaining pulse visual signal sequences of a plurality of pixel points of an observation scene, and obtaining a pulse interval sequence based on the interval between non-zero pulse elements in each pulse visual signal sequence; based on the maximum absolute difference value between any two elements in each pulse interval sequence, performing stable division calculation on the pulse interval sequence to obtain a stable pulse frame number sequence and a corresponding division number sequence; performing dynamic region detection based on the stable pulse frame number sequence to obtain a dynamic region pulse image, and performing filtering and communication processing on the dynamic region pulse image to obtain a dynamic target detection position; based on the ratio of each stable pulse continuous frame number sequence to the corresponding divided number sequence, the reconstructed image is calculated, and the observation result of the observation scene is obtained based on the reconstructed image and the dynamic target detection position, so that the dynamic scene observation efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method for jointly processing pulse visual signal image reconstruction and target detection and related equipment. Background Art

[0002] In areas such as autonomous driving, industrial inspection, and security monitoring, simultaneous imaging and detection of dynamic targets are often required. With the rapid advancement of technology, higher requirements are being placed on imaging and detection of high-speed targets. Traditional vision systems are based on the principle of frame-based imaging (such as CCD / CMOS cameras). Since their sensor output is an image suitable for observation, direct target detection in the image domain is generally sufficient to achieve target observation. However, the frame rate of traditional vision systems is typically limited to tens of hertz, making it difficult to capture microsecond-level light intensity changes. For high-speed targets, this can easily lead to both blurred imaging and failed target detection, making it impossible to accurately observe and measure the target.

[0003] To address the problem of observing high-speed targets, neuromorphic vision sensors are used to acquire continuous pulsed visual signals from dynamic targets. Unlike traditional vision sensors, the raw pulsed visual perception signals from neuromorphic vision sensors are not suitable for direct observation by the human eye. Accurate observation of high-speed targets requires simultaneous image reconstruction and target detection. To achieve this, one approach is to reconstruct the pulsed visual signals from the pulse domain into an image domain for caching, then use multiple consecutive frames for target detection. However, this approach requires repeated "image reconstruction followed by target detection" and conversions between the pulse and image domains, resulting in redundant computations and low computational efficiency. Another approach is to simultaneously execute unrelated image reconstruction and target detection algorithms and then display their results synchronously. This unrelated task of image reconstruction and target detection also leads to low computational efficiency. Summary of the Invention

[0004] The embodiments of the present application provide a method for jointly processing pulse visual signal image reconstruction and target detection, which can improve the efficiency of dynamic scene observation.

[0005] To achieve the above objectives, a first aspect of an embodiment of the present application proposes a method for jointly processing pulse visual signal image reconstruction and target detection, the method comprising:

[0006] Acquire a pulse visual signal sequence of a plurality of pixel points of an observed scene, and obtain a pulse interval sequence based on the interval between non-zero pulse elements in each of the pulse visual signal sequences;

[0007] Based on the maximum absolute difference between any two elements in each of the pulse interval sequences, performing a stable partition calculation on the pulse interval sequence to obtain a stable pulse frame number sequence and a corresponding partition number sequence;

[0008] Performing dynamic region detection based on the stable pulse frame number sequence to obtain a dynamic region pulse image, and performing filtering and connectivity processing on the dynamic region pulse image to obtain a dynamic target detection position;

[0009] Based on the ratio of each stable pulse duration frame number sequence and the corresponding divided division number sequence, a reconstructed image is calculated, and based on the reconstructed image and the dynamic target detection position, an observation result of the observation scene is obtained.

[0010] In some embodiments, performing stable partition calculation on the pulse interval sequence based on the maximum absolute difference between any two elements in each pulse interval sequence to obtain a stable pulse frame number sequence and a corresponding partition number sequence includes:

[0011] First, the first interval element is selected from the interval elements of the pulse interval sequence to be added to the temporary stable interval sequence, and then subsequent interval elements are added to the temporary stable interval sequence in sequence. When the maximum absolute difference between the added subsequent interval element and all the interval elements in the temporary stable interval sequence exceeds a preset threshold, the pulse interval sequence is divided and truncated, and the temporary stable interval sequence is used as the stable interval sequence;

[0012] Accumulate all the interval elements in each stable interval sequence and add one to obtain a stable pulse frame number sequence;

[0013] Based on the number of the interval elements in each of the stable interval sequences, the number of pulse interval elements is obtained, and all the numbers of pulse interval elements are combined to obtain the corresponding partition number sequence.

[0014] In some embodiments, the calculating and obtaining the reconstructed image based on the ratio of the sequence of the number of frames of each stabilization pulse duration to the sequence of the number of divisions corresponding to the divisions comprises:

[0015] Obtaining a single pixel intensity sequence corresponding to each pixel point based on a ratio of each stable pulse frame number sequence to the number of corresponding pulse interval elements in each stable pulse frame number sequence;

[0016] The reconstructed image is obtained based on all the single pixel intensity sequences.

[0017] In some embodiments, the performing dynamic region detection based on the stable pulse frame number sequence to obtain a dynamic region pulse image includes:

[0018] Comparing each stable pulse frame number sequence in the stable pulse frame number sequence with a binarization threshold, and obtaining a dynamic region pulse image pixel value based on the comparison result;

[0019] The dynamic area pulse image is obtained by sorting and combining pixel values ​​of all the dynamic area pulse images.

[0020] In some embodiments, obtaining the dynamic region pulse image pixel value based on the comparison result includes:

[0021] When the comparison result indicates that the stable pulse frame number sequence does not exceed the binarization threshold, obtaining the dynamic region pulse image pixel value based on the first value;

[0022] When the comparison result indicates that the stable pulse frame number sequence exceeds the binarization threshold, the dynamic region pulse image pixel value is obtained based on the second value.

[0023] In some embodiments, filtering and connecting the dynamic region pulse image to obtain the dynamic target detection position includes:

[0024] performing filtering processing on the dynamic region pulse image to obtain a filtered pulse image;

[0025] Based on connected component analysis, the filtered pulse images of the plurality of pixel points are connected to obtain the dynamic target detection position.

[0026] In some embodiments, filtering the dynamic region pulse image to obtain a filtered pulse image includes:

[0027] performing summation filtering processing on the dynamic region pulse image based on the first filtering window and the first filtering pixel value to obtain an initial filtered image;

[0028] Based on a preset corrosion element value, the initial filtered image is corroded to obtain a corroded filtered image;

[0029] Based on a preset expansion element value, the eroded filter image is expanded to obtain an expanded filter image;

[0030] Based on the second filtering window and the second filtering pixel value, the dilated filtered image is subjected to summation filtering processing to obtain the filtered pulse image.

[0031] To achieve the above objectives, a second aspect of an embodiment of the present application proposes a system for joint processing of pulsed visual signal image reconstruction and target detection, the system comprising:

[0032] an acquisition module, configured to acquire a pulse visual signal sequence of a plurality of pixel points of an observed scene, and obtain a pulse interval sequence based on an interval between non-zero pulse elements in each of the pulse visual signal sequences;

[0033] a stable partitioning module, configured to perform a stable partitioning calculation on the pulse interval sequence based on the maximum absolute difference between any two elements in each pulse interval sequence, to obtain a stable pulse frame number sequence and a corresponding partition number sequence;

[0034] a target detection position calculation module, configured to perform dynamic region detection based on the stable pulse frame number sequence to obtain a dynamic region pulse image, and perform filtering and connectivity processing on the dynamic region pulse image to obtain a dynamic target detection position;

[0035] An image reconstruction module is used to calculate a reconstructed image based on the ratio of each stable pulse duration frame number sequence to the corresponding divided division number sequence, and obtain an observation result of the observation scene based on the reconstructed image and the dynamic target detection position.

[0036] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, and the memory stores a computer program. When the processor executes the computer program, it implements the joint processing method of pulse visual signal image reconstruction and target detection as described in the first aspect.

[0037] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, it implements the joint processing method of pulse visual signal image reconstruction and target detection described in the first aspect above.

[0038] The embodiments of the present application propose a method for joint processing of pulse visual signal image reconstruction and target detection and related equipment, the method comprising: first, obtaining a pulse visual signal sequence of multiple pixel points of the observation scene, and obtaining a pulse interval sequence based on the intervals between non-zero pulse elements in each pulse visual signal sequence; then, performing a stable division calculation on the pulse interval sequence based on the maximum absolute difference between any two elements in each pulse interval sequence to obtain a stable pulse frame number sequence and a corresponding division number sequence; next, performing dynamic area detection based on the stable pulse frame number sequence to obtain a dynamic area pulse image, and filtering and connecting the dynamic area pulse image to obtain a dynamic target detection position; finally, calculating a reconstructed image based on the ratio of each stable pulse duration frame number sequence to the corresponding divided division number sequence, and obtaining an observation result of the observation scene based on the reconstructed image and the dynamic target detection position. The embodiment of the present application directly extracts the pulse interval from the pulse visual signal sequence, and adaptively segments the pulse interval sequence in the pulse domain based on pulse stability analysis, thereby using the segmented sequence for dynamic area detection and image reconstruction, avoiding the time-consuming process of having to completely reconstruct each frame of the pulse visual signal into an image domain image and cache it before target detection, as well as the resulting repeated conversion between the pulse domain and the image domain. During the calculation process, the pulse image is cached in binary representation, which greatly reduces the demand for cache and computing resources, and significantly reduces redundant calculations and data processing. While ensuring the quality of image reconstruction and the accuracy of dynamic target extraction, it can effectively improve the overall processing efficiency and real-time performance of imaging and target detection of high-speed dynamic targets, greatly improve the efficiency of dynamic scene observation, and better meet the needs of high-speed target observation in fields such as autonomous driving, industrial inspection, and security monitoring.

[0039] Other features and advantages of the present application will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present application. The purposes and other advantages of the present application can be achieved and obtained through the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a flowchart of a method for jointly processing pulse visual signal image reconstruction and target detection provided by one embodiment of the present application.

[0041] Figure 2 yes Figure 1 Flowchart of step 102 in FIG.

[0042] Figure 3 yes Figure 1 Flowchart of step 103 in FIG.

[0043] Figure 4 yes Figure 3 Flowchart of step 301 in FIG.

[0044] Figure 5 yes Figure 1 Another flow chart of step 103 in FIG.

[0045] Figure 6 yes Figure 5 Flowchart of step 501 in FIG.

[0046] Figure 7 yes Figure 1 Flowchart of step 104 in FIG.

[0047] Figure 8 This is a schematic diagram of an example of joint processing of pulse visual signal image reconstruction and target detection provided by an embodiment of the present application.

[0048] Figure 9 This is a flowchart of a method for joint processing of pulse visual signal image reconstruction and target detection provided in one embodiment of the present application.

[0049] Figure 10 It is a structural diagram of a pulse visual signal image reconstruction and target detection joint processing system provided in one embodiment of the present application.

[0050] Figure 11 This is a schematic diagram of the hardware structure of the system provided in one embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0052] It should be noted that although the functional modules are divided in the device schematic and the logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in a different order than the module division in the device or the order in the flowchart.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0054] In areas such as autonomous driving, industrial inspection, and security monitoring, simultaneous imaging and detection of dynamic targets are often required. With the rapid advancement of technology, higher requirements are being placed on imaging and detection of high-speed targets. Traditional vision systems are based on the principle of frame-based imaging (such as CCD / CMOS cameras). Since their sensor output is an image suitable for observation, direct target detection in the image domain is generally sufficient to achieve target observation. However, the frame rate of traditional vision systems is typically limited to tens of hertz, making it difficult to capture microsecond-level light intensity changes. For high-speed targets, this can easily lead to both blurred imaging and failed target detection, making it impossible to accurately observe and measure the target.

[0055] To address the problem of observing high-speed targets, neuromorphic vision sensors are used to acquire continuous pulsed visual signals from dynamic targets. Unlike traditional vision sensors, the raw pulsed visual perception signals from neuromorphic vision sensors are not suitable for direct observation by the human eye. Accurate observation of high-speed targets requires simultaneous image reconstruction and target detection. To achieve this, one approach is to reconstruct the pulsed visual signals from the pulse domain into an image domain for caching, then use multiple consecutive frames for target detection. However, this approach requires repeated "image reconstruction followed by target detection" and conversions between the pulse and image domains, resulting in redundant computations and low computational efficiency. Another approach is to simultaneously execute unrelated image reconstruction and target detection algorithms and then display their results synchronously. This unrelated task of image reconstruction and target detection also leads to low computational efficiency.

[0056] In order to improve the efficiency of dynamic scene observation, the embodiment of the present application directly extracts the pulse interval from the pulse visual signal sequence, and adaptively segments the pulse interval sequence in the pulse domain based on pulse stability analysis, so as to use the segmented sequence for dynamic area detection and image reconstruction, thereby avoiding the time-consuming process of having to completely reconstruct each frame of pulse visual signal into an image domain image and cache it before target detection, as well as the resulting repeated conversion between the pulse domain and the image domain. During the calculation process, the pulse image is cached in binary representation, which greatly reduces the demand for cache and computing resources, and significantly reduces redundant calculations and data processing. While ensuring the quality of image reconstruction and the accuracy of dynamic target extraction, it can effectively improve the overall processing efficiency and real-time performance of imaging and target detection of high-speed dynamic targets, greatly improving the efficiency of dynamic scene observation, and better meeting the needs of high-speed target observation in fields such as autonomous driving, industrial inspection, and security monitoring.

[0057] The following will further describe the pulse visual signal image reconstruction and target detection joint processing method and related equipment provided by this application. Figure 1, which is an optional flow chart of the method for joint processing of pulse visual signal image reconstruction and target detection provided in an embodiment of the present application, Figure 1 The method may include but is not limited to steps 101 to 104. It is also understood that this embodiment is for Figure 1 The order of steps 101 to 104 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs. The method provided in this application can be applied to any server or intelligent processing terminal, etc.

[0058] Step 101: Obtain a pulse visual signal sequence of multiple pixel points of an observed scene, and obtain a pulse interval sequence based on the interval between non-zero pulse elements in each pulse visual signal sequence.

[0059] Step 101 is described in detail below.

[0060] In some embodiments, in response to a request for dynamic scene image reconstruction and target detection, a pulse visual signal sequence S0 is first obtained from a pulse camera for a plurality of pixel points at pulse camera coordinates (x, y) for the observed scene. A pulse visual signal sequence refers to a discrete pulse event (e.g., a sequence of 0s and 1s) output by a neuromorphic visual sensor, such as a pulse camera, for each pixel point, representing the temporal variation of light intensity. For example, the pulse visual signal sequence corresponding to the pixel points at the pulse camera coordinates (x, y) for 28 consecutive frames is S0 = {1 0 0 0 1 0 0 1 0 0 0 1 0 00 1 0 1 0 1 0 1 0 0 0 0 0 1}.

[0061] Subsequently, based on the frame number interval between non-zero pulse elements (i.e., pulse elements with a value of 1) in each pulse visual signal sequence, a corresponding pulse interval sequence S1 is calculated and obtained, S1 = {T0(i)} (T0(i) represents the i-th element in S1). Each element in the sequence represents the number of frames between one pulse emission and the next pulse emission, reflecting the frequency characteristics of the pulse emission.

[0062] For example, the pulse interval sequence of the pulse visual signal sequence S0 is S1 = {4 3 4 3 2 2 2 5}.

[0063] Step 102: Based on the maximum absolute difference between any two elements in each pulse interval sequence, a stable partition calculation is performed on the pulse interval sequence to obtain a stable pulse frame number sequence and a corresponding partition number sequence.

[0064] Step 102 is described in detail below.

[0065] In some embodiments, after obtaining the pulse interval sequence corresponding to each pixel point, a stable division calculation is further performed based on the maximum absolute difference between any two interval elements in the sequence. This calculation identifies and segments the relatively stable continuous paragraphs in the pulse interval sequence according to a preset stability criterion (i.e., the maximum absolute difference between any two interval elements does not exceed a certain threshold). Through this stable division calculation, two key output sequences can be generated for each pixel point: one is the stable pulse frame number sequence, which is composed of several stable pulse frame number sequences, each of which usually represents the sum of all interval elements in a stable paragraph in the original pulse interval sequence plus a fixed value (such as 1), representing the total duration characteristics of the stable paragraph; the other is the corresponding division number sequence, which records the specific number of original interval elements contained in each stable paragraph. How to perform stable division calculation will be further described below.

[0066] Referring to Figure 2 , based on the maximum absolute difference between any two elements in each pulse interval sequence, a stable division calculation is performed on the pulse interval sequence to obtain a stable pulse frame number sequence and a corresponding division number sequence, including steps 201 to 203.

[0067] Step 201: First, select the first interval element from the interval elements of the pulse interval sequence into the temporary stable interval sequence, then add the subsequent interval elements to the temporary stable interval sequence one by one, when the maximum absolute difference between the added subsequent interval element and all interval elements in the temporary stable interval sequence exceeds the preset threshold, perform pulse interval sequence division truncation, and take the temporary stable interval sequence as the stable interval sequence.

[0068] Step 202: Accumulate all interval elements in each stable interval sequence, and add one to obtain the stable pulse frame number sequence.

[0069] Step 203: Based on the number of interval elements in each stable interval sequence, obtain the pulse interval element number, and combine all pulse interval element numbers to obtain the corresponding division number sequence.

[0070] The steps 201 to 203 are described in detail below.

[0071] In some embodiments, for each pixel point, the pulse interval sequence is first selected to enter the first interval element into a temporary division sequence; then, the subsequent interval elements are sequentially added to the temporary division sequence, and during the addition, a sequential check process is performed to calculate the absolute difference between the newly added interval element and the existing interval elements in the temporary division sequence, to ensure that the maximum absolute difference does not exceed a preset interval threshold (i.e., 1). If it exceeds, the addition of elements to the temporary division sequence is stopped, the temporary division sequence becomes a stable interval sequence, and the interval element to be added is taken as the first interval element of the next temporary division sequence. Through the above process, a stable interval sequence can be divided.

[0072] After obtaining a plurality of stable interval sequences, a corresponding stable pulse frame number sequence is calculated for each stable interval sequence. The calculation process is as follows: the values of all interval elements T0(i) contained in each stable interval sequence are subjected to arithmetic accumulation processing to obtain a cumulative sum, and then the cumulative sum is added by a fixed number (i.e., 1), and the final result is the stable pulse frame number sequence Fts(x,y) corresponding to the stable interval sequence, as shown in the following formula (1).

[0073] Fts(x,y) = 1 +∑ i＝k0～k0+N(x,y)-1 T0(i) (1)

[0074] The stable pulse frame number sequence can be regarded as a measure of the total duration of the stable pulse emission period. For example, in the above example, the pulse interval sequence S1 = {4 3 4 3 2 2 2 5}, the maximum absolute difference between any two elements of the first 4 elements {4 3 4 3} is ≤1; when the 5th element "2" is added to form {4 3 4 3 2}, the maximum absolute difference between any two elements is >1, so it needs to be divided at this point; Similarly, through the pulse stability determination, S1 can be divided into 3 stable stable interval sequences S1 = {4 3 4 3} {2 2} {5}; Next, based on the above formula (1), the stable pulse frame number sequence Fts(x,y) = {15} {7} {6} corresponding to each stable interval sequence is obtained, and the number of pulse interval elements N(x,y) = {4} {3} {1} corresponding to each stable interval sequence.

[0075] After the stable pulse frame number sequence is calculated for each stable interval sequence of the pixel point, the stable pulse frame number sequences corresponding to all the stable interval sequences are combined and arranged according to the order in which they appear in the original pulse interval sequence. Therefore, for a given pixel point, the stable pulse frame number sequence obtained finally will contain one or more stable pulse frame number sequences, which correspond to the respective stable paragraphs identified in the original pulse interval sequence.

[0076] Then, the number of pulse interval elements of all the stable interval sequences corresponding to a pixel point is also combined according to the order in which they appear in the original pulse interval sequence, thereby obtaining the division number sequence corresponding to the pixel point. Each element in this division number sequence matches the stable pulse frame number sequence at the corresponding position in the stable pulse frame number sequence, and together describes the characteristics of the stable paragraph.

[0077] Through the above steps 201 to 203, it is specified in detail how to accurately extract the key parameters that characterize the stability of pulse firing, i.e., the stable pulse frame number sequence and the corresponding division number sequence, from the original pulse interval sequence based on the stable division principle based on the maximum absolute difference comparison. This series of operations ensures the fine analysis of the time dynamics of the pulse signal, and provides accurate and structured data basis for subsequent dynamic region detection and image reconstruction. The stable pulse frame number sequence and the number of pulse interval elements defined in this way can more accurately reflect the stability characteristics of the original pulse visual signal, thereby ensuring the effectiveness of the image reconstruction and target detection synchronous processing in the pulse domain.

[0078] Step 103: Perform dynamic region detection based on the stable pulse frame number sequence to obtain a dynamic region pulse image, and perform filtering and connected processing on the dynamic region pulse image to obtain a dynamic target detection position.

[0079] The following describes step 103 in detail.

[0080] In some embodiments, the generated stable pulse frame number sequence is used for dynamic region detection, and the principle is that a dynamic target usually causes rapid changes in the pulse firing pattern, thereby causing the corresponding stable pulse frame number sequence to be relatively small. Therefore, by comparing each stable pulse frame number sequence with a preset binary threshold, a dynamic region pulse image can be generated, which is a binary image used to preliminarily identify potential motion regions in the scene.

[0081] Because target motion can disrupt the stability of pulse signal generation, the aforementioned pulse stability segmentation results can be used to further detect dynamic regions. Compared to static regions, the duration of stable pulses in dynamic regions is shorter. Therefore, using the stable pulse frame number sequence Fts(x,y), we can roughly obtain the dynamic region pulse image B.

[0082] The following further describes how to perform dynamic region detection based on a stable pulse frame number sequence to obtain a dynamic region pulse image.

[0083] Reference Figure 3 , dynamic area detection is performed based on a stable pulse frame number sequence to obtain a dynamic area pulse image, including the following steps 301 to 302.

[0084] Step 301: Compare each stable pulse frame number sequence in the stable pulse frame number sequence with a binarization threshold, and obtain a dynamic region pulse image pixel value based on the comparison result.

[0085] Step 301 is described in detail below.

[0086] In some embodiments, each stable pulse frame sequence Fts(x,y) in the stable pulse frame sequence corresponding to each pixel is compared with a preset binarization threshold Td0. This binarization threshold is a boundary value used to distinguish between dynamic areas and static areas in a scene.

[0087] Based on the comparison result, a corresponding dynamic region pulse image pixel value can be generated for each stable pulse frame sequence, as described below.

[0088] Reference Figure 4 Based on the comparison result, the dynamic area pulse image pixel value corresponding to each stable pulse frame sequence is obtained, including the following steps 401 to 402.

[0089] Step 401: When the comparison result indicates that the stable pulse frame number sequence does not exceed the binarization threshold, a dynamic region pulse image pixel value is obtained based on a first value.

[0090] Step 402: When the comparison result indicates that the stable pulse frame number sequence exceeds the binarization threshold, a dynamic region pulse image pixel value is obtained based on the second value.

[0091] Steps 401 to 402 are described in detail below.

[0092] In some embodiments, when the comparison result explicitly represents that the stable pulse frame number sequence does not exceed (i.e. is less than or equal to) the preset binarization threshold Td0, the single-point pixel value in the dynamic region pulse image, i.e. the dynamic region pulse image pixel value, will be generated based on a predefined first value. The first value corresponding to this comparison result value is usually set to "1", which represents the logical "true" or "activated" state in the context of a binary image, thereby marking the pixel region in the time period corresponding to the stable pulse frame number sequence as dynamic.

[0093] On the contrary, when the comparison result explicitly represents that the stable pulse frame number sequence exceeds (i.e. is greater than) the preset binarization threshold Td0, the single-point pixel value in the dynamic region pulse image, i.e. the dynamic region pulse image pixel value, will be generated based on a predefined second value. The second value corresponding to this comparison result value is usually set to "0", which represents the logical "false" or "non-activated" state in the context of a binary image, thereby marking the pixel region in the time period corresponding to the stable pulse frame number sequence as static.

[0094] The corresponding calculation formula is shown in the following formula (2).

[0095]

[0096] Through the above steps 401 and 402, by directly mapping the comparison result to the first value (such as "1", representing dynamic) or the second value (such as "0", representing static), this mechanism realizes a simple and efficient binarization process, ensuring the determinacy and consistency of dynamic region pulse image generation, so that the subsequent image processing steps (such as filtering and connected component analysis) can operate based on clear binary input, thereby providing a solid foundation for accurate identification and positioning of dynamic targets. This direct binarization discrimination at the pulse characteristic analysis level helps to quickly filter information and reduce data dimensionality, which is a key link to realize efficient pulse domain target detection.

[0097] Step 302: Based on the sorting of all dynamic region pulse image pixel values, a dynamic region pulse image is obtained.

[0098] After traversing all pixel points (x, y), all single-point pixel values (i.e. dynamic region pulse image pixel values) in the dynamic region pulse image can be combined to obtain the dynamic region pulse image B. In this way, for each 1 frame of original pulse signal, a corresponding dynamic region pulse image (binary image) representing the dynamic / static properties of the observed scene at the corresponding time will be finally obtained, which constitutes the basis for subsequent connected processing and target positioning.

[0099] Based on the above example, assuming that the binarization threshold Td0 is set to 5, since {15}{7}{6} in the stable pulse frame number sequence Fts(x, y) are all > 5, in these 28 consecutive frames, the (x, y) pixel point corresponds to a static region, and the dynamic region pulse image corresponding pixel value B(x, y) is represented as B(x, y) = {0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0} after binarization threshold comparison processing. After the above binarization processing is performed on all pixel points, the dynamic region pulse image B of the 28 consecutive frames can be obtained.

[0100] Through the above steps 301 and 302, by comparing the quantized stable pulse frame number sequence with the preset threshold, potential dynamic change regions can be quickly identified in the pulse domain, and these judgment results are organized into a structured dynamic region pulse image, which directly utilizes the time dynamic information of the pulse signal, avoids complete image reconstruction at an early stage, and thus helps to reduce the amount of calculation and improve the response speed of dynamic detection. The dynamic region pulse image generated thereby provides a clear input for subsequent more refined filtering and connected component analysis, and is a key link in realizing an efficient and real-time target detection process.

[0101] Next, in order to accurately locate the target and suppress noise interference, the obtained dynamic region pulse image needs to be subjected to connected processing, which usually includes first performing a filtering operation (such as a summation filtering using a preset window and threshold, and morphological erosion, dilation, etc.) on the dynamic region pulse image to optimize the image quality, remove isolated noise points and connect broken target regions, and then performing connected component analysis (CCA) on the filtered image to identify and extract each independent connected region, and finally outputting the boundary information (such as the coordinates of the minimum bounding rectangle) of these regions as the dynamic target detection position [x_min, x_max, y_min, y_max]. The specific process is described as follows.

[0102] Referring to Figure 5 The filtering and connected processing of the dynamic region pulse image to obtain the dynamic target detection position includes the following steps 501 to 502.

[0103] Step 501: performing filtering processing on the dynamic region pulse image to obtain a filtered pulse image.

[0104] The step 501 is described in detail as follows.

[0105] In some embodiments, after obtaining a preliminary dynamic region pulse image B, filtering is performed to optimize the quality of the dynamic region pulse image B for more accurate subsequent target detection. Specifically, a series of filtering operations are performed on the dynamic region pulse image B, including filter → morphological operations (Erode, Dilate) → filter, to ensure that the resulting filtered pulse image has a clearer target outline and less background interference, as described below.

[0106] Reference Figure 6 , filtering the dynamic region pulse image to obtain a filtered pulse image, including the following steps 601 to 604.

[0107] Step 601: Based on the first filtering window and the first filtering pixel value, perform summation filtering processing on the dynamic region pulse image to obtain an initial filtered image.

[0108] Step 602: Based on a preset corrosion element value, the initial filtered image is corroded to obtain a corroded filtered image.

[0109] Step 603: Based on the preset dilation element value, dilation processing is performed on the erosion filter image to obtain a dilation filter image.

[0110] Step 604: Based on the second filtering window and the second filtering pixel value, perform summation filtering processing on the dilated filtered image to obtain a filtered pulse image.

[0111] Steps 601 to 604 are described in detail below.

[0112] In some embodiments, for each dynamic region pulse image B, a summation filtering process is first performed on each pixel in the dynamic region pulse image and its neighborhood based on a predefined first filtering window wf1×wf1 (which is a rectangular area of ​​wf1 x wf1 pixels) and a corresponding preset first filtering pixel value Td1. wf1,Td1 This summation filtering process typically involves calculating the sum of all pixel values ​​within the first filter window (since dynamic region pulse images are typically binary images, this means counting the number of pixels within the window with a value of "1"), and then comparing this sum with the first filtered pixel value. If the sum reaches or exceeds a threshold, the output pixel may be set to "1," otherwise to "0." This operation effectively removes isolated bright spots that are considered noise and performs preliminary smoothing on smaller target areas, resulting in a preliminarily optimized initial filtered image.

[0113] After obtaining the initial filtered image, this step further optimizes the image through morphological erosion operation, the main purpose of which is to remove smaller noise structures and refine the boundaries of the target. Specifically, the initial filtered image is eroded based on a preset erosion element value we×we (also called structure element, which defines the shape and size of the erosion operation). we The effect of the erosion process is that if the structuring element cannot completely match a foreground pixel in the initial filtered image (e.g., a pixel with a value of "1") in the neighborhood of its center pixel, the center pixel will be set to a background pixel (e.g., a value of "0"). This will cause the boundaries of the foreground area in the image to shrink inward, effectively eliminating isolated foreground points or small connections that are smaller than the structuring element, and may disconnect some connected target parts, ultimately resulting in a cleaner eroded filtered image.

[0114] Following the erosion process, the morphological dilation operation is used. Its main purpose is to restore the target area that may have been over-shrunk during the erosion process and fill the small holes that may exist inside the target. The specific operation is to dilate the erosion filtered image based on a preset dilation element value wd×wd (also a structural element, its shape and size can be the same or different from the erosion element) wd Dilation has the opposite effect of erosion. It converts background pixels covered by the structuring element (if the center of the structuring element corresponds to a foreground pixel) into foreground pixels. This causes the boundaries of the foreground region in the image to expand outward, helping to connect adjacent but separated object parts, fill gaps within the object, and partially restore the size of the object that was reduced by erosion, resulting in a more complete and continuous dilated filtered image.

[0115] In the final stage of the filtering process, the dilated filtered image after the morphological operation is finally smoothed and confirmed to obtain the final filtered impulse image. The dilated filtered image is again summed and filtered based on a predefined second filter window wf2×wf2 and the corresponding second filtered pixel value Td2. wf1,Td2 The purpose of this summation filtering process is similar to the first one, namely to further ensure that only those foreground pixels with sufficient support in the neighborhood are retained, thereby further removing noise that may have been introduced or not completely removed in the previous process and making the edges of the target area smoother. After this series of fine-tuning, the final output is the filtered spike image, which will serve as the direct input for the subsequent connected component analysis.

[0116] The specific implementation formula for filtering the dynamic region pulse image B is shown in the following formula (3).

[0117] B p(x,y)=filter wf2,Td2 {Dilate wd {Erode we {filter wf1,Td1 [B(x,y)]}}} (3)

[0118] In one example, the dynamic region pulse image B is first filtered wf1,Td1 The specific steps of filtering are to sum the pixel values ​​in the wf1×wf1==3×3 window around the B(x,y) pixel as the center, and compare it with the threshold Td1 for binarization (that is, when the sum value > Td1, it is 1, otherwise it is 0); then, the pixels that have passed the filter are summed. wf1,Td1 The Erodewe erosion operation is then performed on B, and the eroded structural element is we×we=5×5 in size; similarly, the Dilate operation is then performed in sequence to expand the structure to wd×wd=5×5 wd Dilation operation and filter wf2,Td2 (wf2=7, Td2=25) filtering operation can finally obtain the filtered pulse image Bp. Through filtering processing, the background noise can be eliminated and the dynamic target can be revealed.

[0119] Through the above steps 601 to 604, by sequentially performing preliminary sum filtering, morphological erosion, morphological dilation, and final sum filtering, this solution can effectively remove noise, separate adhered targets, fill target holes, and smooth target edges. This multi-stage filtering strategy, especially the combination of erosion and dilation (usually referred to as part of an opening operation or a closing operation, depending on their order and specific application), is very effective in processing common defects in binary images. It can significantly improve the accuracy and robustness of subsequent connected component analysis, thereby ensuring that the final dynamic target detection position is more reliable and providing a strong guarantee for the performance of the entire target detection.

[0120] Step 502: Based on connected component analysis, the filtered pulse image is connected to obtain the dynamic target detection position.

[0121] Step 502 is described in detail below.

[0122] In some embodiments, after obtaining a higher-quality filtered pulse image Bp, the image is connected to accurately identify and locate each independent dynamic target in the scene, i.e., the dynamic detection target. Specifically, based on the connected component analysis (CCA) algorithm, the filtered pulse image (usually a complete binary image frame) is processed to obtain the bounding box [x_min, x_max, y_min, y_max] of the connected components (dynamic targets) in the filtered pulse image Bp, i.e., the dynamic detection target, as shown in the following formula (4).

[0123] [x_min,x_max,y_min,y_max]=CCA(B p ) (4)

[0124] Connected component analysis is a technique that finds and labels all interconnected pixel regions (i.e., connected components) in a binary image. Each connected component typically corresponds to a potential independent target or part of a target. By analyzing these identified connected components, such as calculating their minimum bounding rectangle, centroid, or other geometric features, the dynamic target detection position can be ultimately obtained. These dynamic target detection positions are typically output in the form of bounding box coordinates, which clearly indicate the spatial extent of each detected dynamic target in the image. CCA can use conventional two-pass traversal methods such as seed filling.

[0125] In one example, connected component analysis is performed on the filtered pulse image Bp to output a dynamic target detection box [x_min, x_max, y_min, y_max]. For example, a two-pass method is used to perform connected component analysis on the filtered pulse image Bp. The specific steps are shown in the following pseudo code example.

[0126] (1) Initialize a label matrix labelBP=0 of the same size as Bp, label variable label=0, and equivalent label lookup table Slabel={};

[0127] (2) The first traversal is from top to bottom and from left to right to traverse the filtered pulse image Bp. If Bp(x,y)==1:

[0128] if label BP (x-1,y)==0AND label BP (x,y-1)==0

[0129] label+=1, label BP (x,y)=label;

[0130] elseif label BP (x-1,y)>0AND label BP (x,y-1)==0

[0131] label BP (x,y)=label BP (x-1,y);

[0132] elseif label BP (x-1,y)==0AND label BP (x,y-1)>0

[0133] label BP (x,y)=label BP (x,y-1);

[0134] else

[0135] label BP (x,y)=min[label BP (x-1,y),label BP (x,y-1)];

[0136] Slabel={Slabel;{label BP (x-1,y),label BP (x,y-1)}}; % record the equality relationship query table between labels, where {label BP (x-1,y),label BP (x,y-1)} represents label BP (x-1,y) and label BP (x,y-1) labels belong to the same connected region;

[0137] end

[0138] (3) The second traversal is from top to bottom and from left to right. BP If label BP (x,y)>1:

[0139] Find the label in the equivalent label lookup table Slabel BP (x,y) belongs to the same connected domain and the minimum label value is assigned to label BP (x,y).

[0140] (4) Label BPThe bounding box [x_min, x_max, y_min, y_max] with the same label area is extracted as the target detection box of the dynamic target, that is, the dynamic detection target.

[0141] Through the above steps 501 and 502, through a series of carefully designed filtering and morphological operations, noise is effectively suppressed and the target signal is enhanced, providing high-quality input for subsequent analysis. Then, using mature connected component analysis technology, each independent dynamic target area is accurately segmented and located on the optimized filtered pulse image, ensuring the accuracy and robustness of the dynamic target detection position, which is a necessary link to achieve reliable target detection, enabling the entire system to robustly extract meaningful target information from the original, possibly noisy pulse data.

[0142] Step 104: Based on the ratio of each stable pulse frame number sequence and the corresponding division number sequence, a reconstructed image is calculated, and based on the reconstructed image and the dynamic target detection position, an observation result of the observation scene is obtained.

[0143] Step 104 is described in detail below.

[0144] In some embodiments, in parallel with dynamic area detection, based on each element Fts(x,y) in the stable pulse frame number sequence of each pixel point and each division element N(x,y) in the corresponding division number sequence, the average light intensity value of the pixel point in the time period is determined by calculating the ratio of each pair of division elements to the stable pulse frame number sequence, as shown in the following formula (5).

[0145] I(x,y)=N(x,y) / Fts(x,y) (5)

[0146] By combining these light intensity values ​​for all pixels, we can calculate a reconstructed image I, which provides a visual representation of the scene. Finally, we combine this reconstructed image I with the dynamic target detection position [x_min, x_max, y_min, y_max] and overlay the bounding box indicated by the dynamic target detection position on the reconstructed image to obtain an observation result that can simultaneously present the scene content and the spatial location of the dynamic target within it.

[0147] The following further describes how to calculate and obtain the reconstructed image based on the ratio of each stable pulse frame number sequence and the corresponding division number sequence.

[0148] Reference Figure 7 , based on the ratio of each stable pulse frame number sequence and the corresponding division number sequence, a reconstructed image is calculated, including the following steps 701 to 702.

[0149] Step 701: Based on the ratio of each element in each stable pulse frame number sequence to each element in the corresponding partition number sequence, a single pixel intensity sequence corresponding to each pixel point is obtained.

[0150] Step 702: Obtain a reconstructed image based on all single-pixel intensity sequences.

[0151] Steps 701 to 702 are described in detail below.

[0152] In some embodiments, for each pixel, the previously obtained stable pulse frame number sequence contains a number of elements Fts(x,y), and the corresponding partition number sequence N(x,y) also contains a corresponding number of elements. The corresponding elements in the two sequences are ratio-calculated, as shown in formula (5) above, to obtain the single-pixel intensity sequence I(x,y) corresponding to the pixel in the stable segment. This single-pixel intensity sequence will directly contribute to the brightness or grayscale of the pixel in the final reconstructed image.

[0153] After calculating a single-pixel intensity sequence I(x,y) for each stable pulse frame sequence (corresponding to a stable segment), this is achieved by sorting and combining all the single-pixel intensity sequences I(x,y) corresponding to a specific pixel point in strict accordance with the order of their elements in the original stable pulse frame sequence (that is, the order of the stable segments in the original pulse interval sequence). Therefore, for each pixel point, a single-pixel intensity sequence is finally obtained, which records the brightness change information of the pixel point during different stable time periods.

[0154] In an example, based on the above-mentioned stable interval sequence S1 = {4 3 4 3} {2 2} {5}, stable pulse frame number sequence Fts(x,y) = {15} {7} {6}, and number of pulse interval elements N(x,y) = {4} {3} {1}, the intensity value of the corresponding pixel point in 28 consecutive frames can be further reconstructed as I(x,y) = {0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.43 0.43 0.43 0.43 0.43 0.43 0.43 0.17 0.17 0.17 0.17 0.17}.

[0155] After generating individual single-pixel intensity sequences for all pixels involved in the imaging, these single-pixel intensity sequences are arranged and combined according to their spatial positions on the physical sensor array (i.e., the two-dimensional coordinates of the pixels) based on the corresponding values ​​in the single-pixel intensity sequences of all pixels at the same time point (or corresponding to the same stable segment index), thereby forming a complete image. If the single-pixel intensity sequence contains values ​​at multiple time points, a series of image frames can be reconstructed, or a specific time point can be selected according to application requirements or the sequence can be aggregated in some way (such as averaging, weighted averaging, etc.) to obtain a single-frame reconstructed image. This reconstructed image converts the original, discrete pulse visual signal into a human-observable, continuous-tone image domain representation.

[0156] Through the above steps 701 to 702, the pixel intensity of each stable segment is quantified by calculating the ratio of the stable pulse frame number sequence to the number of pulse interval elements, and these intensity values ​​are organized in chronological order. Finally, the information of all pixel points is integrated, which can effectively restore the scene illumination information contained in the temporal dynamics of the pulse signal and generate a visually understandable reconstructed image. This is processed in parallel with the dynamic area detection, embodying the idea of ​​joint processing, helping to reduce unnecessary intermediate steps, and providing users with instant visual feedback of the scene, which is mutually verified with the dynamic detection results, thereby improving the overall perception of high-speed dynamic scenes.

[0157] Reference Figure 8 , is a schematic diagram of an example of joint processing of pulse visual signal image reconstruction and target detection provided by an embodiment of the present application. Figure 8As shown in , the key intermediate processes and final results of the pulse visual signal image reconstruction and target detection joint processing method of the present invention are demonstrated. The "original pulse" image in the upper left corner intuitively shows the pulse visual signal sequence of multiple pixel points obtained from the observed scene. The scattered bright spots in the figure represent non-zero pulse elements, reflecting the original, sparse and possibly noisy event information captured by the pulse visual sensor in a short period of time. It is the basic input for the target detection method provided by the present application to perform subsequent pulse interval sequence extraction and stable division calculation. The "dynamic area pulse image B" in the upper right corner is a preliminary result obtained by comparing the stable pulse frame number sequence calculated based on step 102 and the binarization threshold through steps 301 and 302. The area with a relatively high bright spot density in the figure preliminarily identifies the potential dynamic area in the scene, which corresponds to the stage of "obtaining the dynamic area pulse image" in step 103 of the target detection method provided by the present application. At this time, the image may still contain more background noise. The "pre-processed pulse image Bp" in the lower left corner shows the result after the "dynamic area pulse image B" is subjected to filtering processing (including sum filtering, corrosion processing and dilation processing) such as step 501 and steps 601 to 604. The noise in the image is effectively suppressed, and the outline of the dynamic target (the two white areas in the figure) becomes clearer and more complete, laying a good foundation for the subsequent connected component analysis (step 502). The "reconstructed image and dynamic target detection frame" in the lower right corner finally presents the joint processing result of the target detection method provided by this application: the grayscale image of the background is the "reconstructed image" calculated based on the ratio of each stable pulse frame number sequence and the corresponding division number sequence in steps 104 and steps 701 to 703, which clearly shows the scene content; and the two colored rectangular frames superimposed on it (cyan and magenta, with coordinates marked) are the "dynamic target detection positions" obtained after connected component analysis of the "pre-processed pulse image Bp", which accurately mark the two detected dynamic targets. This figure fully reflects the characteristics of parallel output and collaborative work of image reconstruction and target detection.

[0158] Reference Figure 9 , is a schematic flow chart of a method for joint processing of pulse visual signal image reconstruction and target detection provided by an embodiment of the present application. Figure 9As shown in , the process begins with "calculating the total continuous frame rate Fts(x,y) of the stable pulse frame sequence based on the pulse stability judgment principle." This corresponds to step 101 (obtaining a pulse visual signal sequence and obtaining a pulse interval sequence) and step 102 (performing a stable division calculation on the pulse interval sequence to obtain a stable pulse frame sequence and a corresponding division sequence). Fts(x,y) can be understood as a quantity closely related to the stable pulse frame sequence. Subsequently, the method develops two processing paths in parallel: the first path is "obtaining a dynamic region pulse image B through Fts(x,y)," followed by "filtering B to obtain a preprocessed pulse image Bp," and finally "analyzing the connected components of the pulse image Bp and outputting a dynamic target detection frame [x_min, x_max, y_min, y_max]." This path fully describes the implementation process of step 103 (performing dynamic region detection based on the stable pulse frame sequence to obtain a dynamic region pulse image, and filtering and connecting the dynamic region pulse image to obtain the dynamic target detection position). The second parallel path is "using Fts(x,y) to calculate and output the reconstructed image I", which corresponds to the first half of step 104 (calculating the reconstructed image based on the ratio of each stable pulse frame number sequence and the corresponding divided number sequence). Finally, combined with the second half of step 104 "and obtaining the detection result of the observed scene based on the reconstructed image and the dynamic target detection position", it can be understood that the outputs of the two parallel paths (reconstructed image I and dynamic target detection frame) together constitute the final observation result. This flowchart accurately reflects the technical idea of ​​the method of the present invention, which is to first extract the core stable pulse characteristics from the pulse signal, and then perform dynamic target detection and scene image reconstruction in parallel based on this characteristic.

[0159] Different from directly using the difference between adjacent pulses in the original pulse sequence to reconstruct the scene, the embodiment of the present application judges the stability based on the characteristics of the pulse interval sequence S1 sequence based on the maximum absolute difference between any two elements in the pulse interval sequence S1. This process does not limit the adjacency of the elements.

[0160] In addition, unlike using different methods to reconstruct images for static areas and dynamic areas respectively, in the embodiment of the present application, image reconstruction does not distinguish between dynamic areas and static areas. A unified image reconstruction method is used for both static and dynamic areas. The purpose of dividing static and dynamic areas here is to achieve dynamic target detection.

[0161] The embodiment of the present application proposes a joint processing method for pulse visual signal image reconstruction and target detection and related equipment, the method includes: first, obtaining a pulse visual signal sequence of multiple pixel points of the observed scene, and obtaining a pulse interval sequence based on the interval between non-zero pulse elements in each pulse visual signal sequence; then, sorting the interval elements in each pulse interval sequence according to the element order, and selecting multiple interval elements whose maximum absolute difference between any two interval elements does not exceed a preset interval threshold in turn to form a stable interval sequence, and no identical interval elements exist in every two stable interval sequences; all interval elements in each stable interval sequence are accumulated and added to obtain a stable pulse frame number sequence, and all stable pulse frame number sequences are combined to obtain a stable pulse frame number sequence corresponding to each pulse interval sequence; based on the number of interval elements in each stable interval sequence, the number of pulse interval elements is obtained, and the number of pulse interval elements is combined to obtain a partition number sequence corresponding to each pulse interval sequence; next, comparing each element value in the stable pulse frame number sequence with a binarization threshold, and when the comparison result indicates that the stable pulse frame number sequence does not exceed the binarization threshold, obtaining a dynamic area pulse image based on the first value. Pixel value, when the comparison result indicates that the stable pulse frame number sequence exceeds the binarization threshold, the pixel value of the dynamic area pulse image is obtained based on the second value, and the dynamic area pulse image is combined based on the temporal and spatial order of all the dynamic area pulse image pixel values ​​to obtain the dynamic area pulse image, and the dynamic area pulse image is summed and filtered based on the first filter window and the first filter pixel value to obtain an initial filtered image, and the initial filtered image is corroded based on the preset corrosion element value to obtain an corrosion filtered image, and the corrosion filtered image is dilated based on the preset expansion element value to obtain a dilated filtered image, and the dilated filtered image is summed and filtered based on the second filter window and the second filter pixel value to obtain a filtered pulse image, and the filtered pulse image is connected based on the connected component analysis to obtain the dynamic target detection position; finally, based on the ratio of each element value in each stable pulse frame number sequence to the number of corresponding pulse interval elements, a corresponding single pixel intensity sequence is obtained, based on all single pixel intensity sequences and element order order, a single pixel intensity sequence corresponding to each pixel point is combined, and based on the spatial order of all single pixel intensity sequences, a reconstructed image is obtained, and an observation result is obtained based on the reconstructed image and the dynamic target detection position.

[0162] The embodiment of the present application directly extracts pulse intervals from the pulse visual signal sequence, and adaptively segments the pulse sequence in the pulse domain based on pulse stability analysis, thereby using the segmented pulse sequence for dynamic area detection and image reconstruction, avoiding the time-consuming process of having to completely reconstruct each frame of pulse visual signals into an image domain image and cache it before target detection, as well as the resulting repeated conversion between the pulse domain and the image domain. During the calculation process, the pulse image is cached in binary representation, which greatly reduces the demand for cache and computing resources, and significantly reduces redundant calculations and data processing. While ensuring the image reconstruction quality and the accuracy of dynamic target extraction, it can effectively improve the overall processing efficiency and real-time performance of imaging and target detection of high-speed dynamic targets, and better meet the needs of autonomous driving, industrial inspection, security monitoring and other fields for high-speed target observation; and it specifies in detail how to extract the image from the original pulse interval sequence through a stability method based on maximum absolute difference comparison. Based on the fixed division principle, the key parameters characterizing the stability of pulse emission - the stable pulse frame number sequence and the corresponding division number sequence - are accurately extracted. This series of operations ensures a refined analysis of the temporal dynamics of the pulse signal, providing an accurate and structured data basis for subsequent dynamic area detection and image reconstruction. The stable pulse frame number sequence and the number of pulse interval elements defined in this way can more accurately reflect the total number of frames and elements of pulse stability, thereby ensuring the effectiveness of the entire target detection method in pulse domain processing; and, by directly mapping the comparison result to a first numerical value (such as "1", representing dynamic) or a second numerical value (such as "0", representing static), this mechanism realizes a simple and efficient binarization process, ensuring the determinism and consistency of the generation of dynamic area pulse images, so that subsequent image processing steps (such as filtering and connected component analysis) can operate based on clear binary inputs, thereby providing a solid foundation for accurately identifying and locating dynamic targets.This direct binary discrimination at the pulse characteristic analysis level helps to quickly screen information and reduce data dimensions, and is a key link in achieving efficient pulse domain target detection; in addition, by comparing the quantized stable pulse frame number sequence with the preset threshold, it is possible to quickly identify potential dynamic change areas in the pulse domain, and organize these judgment results into structured dynamic area pulse images, directly utilizing the temporal dynamic information of the pulse signal, avoiding complete image reconstruction in the early stage, thereby helping to reduce the amount of calculation and improve the response speed of dynamic detection. The dynamic area pulse image generated provides a clear input for subsequent more refined filtering and connected component analysis, and is a key link in achieving efficient and real-time target detection. In addition, by sequentially performing preliminary sum filtering, morphological erosion, morphological dilation and final sum filtering, the scheme can effectively remove noise, separate adhered targets, fill target holes and smooth target edges. This multi-stage filtering strategy, especially The combination of erosion and dilation (often referred to as part of an opening operation or a closing operation, depending on their order and specific application) is very effective in addressing common defects in binary images. It can significantly improve the accuracy and robustness of subsequent connected component analysis, thereby ensuring that the final dynamic target detection position is more reliable, providing a strong guarantee for the performance of the entire image reconstruction and target detection joint processing system; and, by calculating the ratio of the stable pulse frame number sequence to the number of pulse interval elements to quantify the pixel intensity of each stable segment, and organizing these intensity values ​​in chronological order, and ultimately integrating the information of all pixel points, it can effectively restore the scene illumination information contained in the temporal dynamics of the pulse signal, generating a visually understandable reconstructed image. Parallel processing with dynamic area detection embodies the idea of ​​joint processing, helps to reduce unnecessary intermediate steps, and can provide users with instant visual feedback of the scene, which is mutually confirmed with the dynamic detection results, thereby improving the overall perception of high-speed dynamic scenes.

[0163] The embodiment of the present application also provides a pulse visual signal image reconstruction and target detection joint processing system, which can realize the above-mentioned pulse visual signal image reconstruction and target detection joint processing method, referring to Figure 10 , the system 1000 includes:

[0164] An acquisition module 1010 is configured to acquire a pulse visual signal sequence of a plurality of pixel points of an observed scene, and obtain a pulse interval sequence based on the intervals between non-zero pulse elements in each of the pulse visual signal sequences;

[0165] A stable partitioning module 1020 is configured to perform a stable partitioning calculation on the pulse interval sequence based on the maximum absolute difference between any two elements in each pulse interval sequence to obtain a stable pulse frame number sequence and a corresponding partition number sequence;

[0166] The target detection position calculation module 1030 is used to perform dynamic area detection based on the stable pulse frame number sequence to obtain a dynamic area pulse image, and perform filtering and connectivity processing on the dynamic area pulse image to obtain a dynamic target detection position;

[0167] The image reconstruction module 1040 is used to calculate a reconstructed image based on the ratio of each stable pulse duration frame number sequence to the corresponding divided division number sequence, and obtain the observation result of the observation scene based on the reconstructed image and the dynamic target detection position.

[0168] In some embodiments, the stable partitioning module 1020 is further configured to:

[0169] First, the first interval element is selected from the interval elements of the pulse interval sequence to be added to the temporary stable interval sequence, and then subsequent interval elements are added to the temporary stable interval sequence in sequence. When the maximum absolute difference between the added subsequent interval element and all the interval elements in the temporary stable interval sequence exceeds a preset threshold, the pulse interval sequence is divided and truncated, and the temporary stable interval sequence is used as the stable interval sequence;

[0170] Accumulate all the interval elements in each stable interval sequence and add one to obtain a stable pulse frame number sequence;

[0171] Based on the number of the interval elements in each of the stable interval sequences, the number of pulse interval elements is obtained, and all the numbers of pulse interval elements are combined to obtain the corresponding partition number sequence.

[0172] In some embodiments, the image reconstruction module 1040 is further configured to:

[0173] Obtaining a single pixel intensity sequence corresponding to each pixel point based on a ratio of each stable pulse frame number sequence to the number of corresponding pulse interval elements in each stable pulse frame number sequence;

[0174] The reconstructed image is obtained based on all the single pixel intensity sequences.

[0175] In some embodiments, the detection position calculation module 1030 is further configured to:

[0176] Comparing each stable pulse frame number sequence in the stable pulse frame number sequence with a binarization threshold, and obtaining a dynamic region pulse image pixel value based on the comparison result;

[0177] The dynamic area pulse image is obtained by sorting and combining pixel values ​​of all the dynamic area pulse images.

[0178] In some embodiments, the detection position calculation module 1030 is further configured to:

[0179] When the comparison result indicates that the stable pulse frame number sequence does not exceed the binarization threshold, obtaining the dynamic region pulse image pixel value based on the first value;

[0180] When the comparison result indicates that the stable pulse frame number sequence exceeds the binarization threshold, the dynamic region pulse image pixel value is obtained based on the second value.

[0181] In some embodiments, the detection position calculation module 1030 is further configured to:

[0182] performing filtering processing on the dynamic region pulse image to obtain a filtered pulse image;

[0183] Based on connected component analysis, the filtered pulse images of the plurality of pixel points are connected to obtain the dynamic target detection position.

[0184] In some embodiments, the detection position calculation module 1030 is further configured to:

[0185] performing summation filtering processing on the dynamic region pulse image based on the first filtering window and the first filtering pixel value to obtain an initial filtered image;

[0186] Based on a preset corrosion element value, the initial filtered image is corroded to obtain a corroded filtered image;

[0187] Based on a preset expansion element value, the eroded filter image is expanded to obtain an expanded filter image;

[0188] Based on the second filtering window and the second filtering pixel value, the dilated filtered image is subjected to summation filtering processing to obtain the filtered pulse image.

[0189] In the above embodiments, the description of each embodiment has its own focus. For the part that is not described in detail in a certain embodiment, the specific implementation method of the pulse visual signal image reconstruction and target detection joint processing system is basically the same as the specific implementation method of the above-mentioned pulse visual signal image reconstruction and target detection joint processing method, and will not be repeated here.

[0190] In the embodiments of the present application, the pulse visual signal image reconstruction and target detection joint processing system extracts pulse intervals directly from the pulse visual signal sequence, performs adaptive segmentation of the pulse sequence based on pulse stability analysis in the pulse domain, and then uses the segmented pulse sequence for dynamic region detection and image reconstruction, thereby avoiding the time-consuming process of first reconstructing each frame of pulse visual signal into an image domain image and buffering before target detection, and the repeated conversion between the pulse domain and the image domain. In the calculation process, the binary pulse image is cached, which greatly reduces the demand for cache and computing resources, significantly reduces redundant calculation and data processing, and thus improves the overall processing efficiency and real-time performance of imaging and target detection of high-speed dynamic targets while ensuring image reconstruction quality and dynamic target extraction accuracy, better meeting the needs of automatic driving, industrial detection, security monitoring and other fields for high-speed target observation. In addition, it is specified in detail how to extract the key parameters representing pulse firing stability, i.e. the stable pulse frame number sequence and the corresponding division number sequence, from the original pulse interval sequence based on the stable division principle of maximum absolute difference comparison. This series of operations ensures fine analysis of the temporal dynamics of the pulse signal, providing accurate and structured data foundation for subsequent dynamic region detection and image reconstruction. The stable pulse frame number sequence and pulse interval element number defined in this way can more accurately reflect the total number of stable pulse frames and element numbers, thereby ensuring the effectiveness of the entire joint processing method in the pulse domain processing. Furthermore, by directly mapping the comparison result to a first value (such as "1" representing dynamic) or a second value (such as "0" representing static), this mechanism realizes a simple and efficient binary process, ensuring the certainty and consistency of dynamic region pulse image generation, and enabling subsequent image processing steps (such as filtering and connected component analysis) to operate based on clear binary input, thereby providing a solid foundation for accurate identification and positioning of dynamic targets.This direct binary discrimination at the pulse characteristic analysis level helps to quickly screen information and reduce data dimensions, and is a key link in achieving efficient pulse domain joint processing; in addition, by comparing the quantized stable pulse frame number sequence with the preset threshold, it is possible to quickly identify potential dynamic change areas in the pulse domain, and organize these judgment results into structured dynamic area pulse images, which directly utilizes the temporal dynamic information of the pulse signal and avoids complete image reconstruction in the early stage, thereby helping to reduce the amount of calculation and improve the response speed of dynamic detection. The dynamic area pulse image generated provides a clear input for subsequent more refined filtering and connected component analysis, and is a key link in achieving efficient and real-time target detection. In addition, by sequentially performing preliminary sum filtering, morphological erosion, morphological dilation and final sum filtering, the scheme can effectively remove noise, separate adhesion targets, fill target holes and smooth target edges. This multi-stage filtering strategy The combined use of corrosion and dilation (often called opening or part of closing, depending on their order and specific application) is very effective in processing common defects in binary images, and can significantly improve the accuracy and robustness of subsequent connected component analysis, thereby ensuring that the final dynamic target detection position is more reliable, providing a strong guarantee for the performance of the entire joint processing system; and, by calculating the ratio of the number of stable pulse frame sequences to the number of pulse interval elements to quantify the pixel intensity of each stable segment, and organizing these intensity values ​​in chronological order, and finally integrating the information of all pixel points, it can effectively restore the scene illumination information contained in the temporal dynamics of the pulse signal, generate a visually understandable reconstructed image, and process it in parallel with dynamic area detection, embodying the idea of ​​joint processing, helping to reduce unnecessary intermediate steps, and providing users with instant visual feedback of the scene, which is mutually verified with the dynamic detection results, thereby improving the overall perception of high-speed dynamic scenes.

[0191] An embodiment of the present application further provides an electronic device, including:

[0192] at least one memory;

[0193] at least one processor;

[0194] at least one program;

[0195] The program is stored in the memory, and the processor executes the at least one program to implement the target detection method described above. The electronic device can be any smart terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a car computer, etc.

[0196] See also Figure 11 , Figure 11The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0197] The processor 1101 can be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0198] The memory 1102 can be implemented in the form of ROM (Read Only Memory), static storage device, dynamic storage device or RAM (Random Access Memory). The memory 1102 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1102 and is called by the processor 1101 to execute the target detection method of the embodiments of this application;

[0199] Input / output interface 1103, used to implement information input and output;

[0200] Communication interface 1104, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0201] Bus 1105 , which transmits information between various components of the device (e.g., processor 1101 , memory 1102 , input / output interface 1103 , and communication interface 1104 );

[0202] The processor 1101 , the memory 1102 , the input / output interface 1103 and the communication interface 1104 are connected to each other in communication within the device via a bus 1105 .

[0203] An embodiment of the present application further provides a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, the above-mentioned target detection method is implemented.

[0204] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0205] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0206] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0207] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0208] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0209] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0210] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0211] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0212] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0213] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0214] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0215] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not limited to the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A method for joint processing of pulse visual signal image reconstruction and target detection, characterized in that: The method comprises: Acquire a pulse visual signal sequence of a plurality of pixel points of an observed scene, and obtain a pulse interval sequence based on the interval between non-zero pulse elements in each of the pulse visual signal sequences; Based on the maximum absolute difference between any two elements in each of the pulse interval sequences, performing a stable partition calculation on the pulse interval sequence to obtain a stable pulse frame number sequence and a corresponding partition number sequence; Performing dynamic region detection based on the stable pulse frame number sequence to obtain a dynamic region pulse image, and performing filtering and connectivity processing on the dynamic region pulse image to obtain a dynamic target detection position; Based on the ratio of each stable pulse duration frame number sequence and the corresponding divided division number sequence, a reconstructed image is calculated, and based on the reconstructed image and the dynamic target detection position, an observation result of the observation scene is obtained.

2. The method for combined processing of pulse visual signal image reconstruction and target detection according to claim 1, characterized in that: The method of performing stable division calculation on the pulse interval sequence based on the maximum absolute difference between any two elements in each pulse interval sequence to obtain a stable pulse frame number sequence and a corresponding division number sequence includes: First, the first interval element is selected from the interval elements of the pulse interval sequence to be added to the temporary stable interval sequence, and then subsequent interval elements are added to the temporary stable interval sequence in sequence. When the maximum absolute difference between the added subsequent interval element and all the interval elements in the temporary stable interval sequence exceeds a preset threshold, the pulse interval sequence is divided and truncated, and the temporary stable interval sequence is used as the stable interval sequence; Accumulate all the interval elements in each stable interval sequence and add one to obtain a stable pulse frame number sequence; Based on the number of the interval elements in each of the stable interval sequences, the number of pulse interval elements is obtained, and all the numbers of pulse interval elements are combined to obtain the corresponding partition number sequence.

3. The method for combined processing of pulse visual signal image reconstruction and target detection according to claim 2, characterized in that: The reconstructed image is calculated based on the ratio of each stable pulse duration frame number sequence to the corresponding divided division number sequence, including: Obtaining a single pixel intensity sequence corresponding to each pixel point based on a ratio of each stable pulse frame number sequence to the number of corresponding pulse interval elements in each stable pulse frame number sequence; The reconstructed image is obtained based on all the single pixel intensity sequences.

4. The method for combined processing of pulse visual signal image reconstruction and target detection according to claim 2, characterized in that: The performing dynamic region detection based on the stable pulse frame number sequence to obtain a dynamic region pulse image includes: Comparing each stable pulse frame number sequence in the stable pulse frame number sequence with a binarization threshold, and obtaining a dynamic region pulse image pixel value based on the comparison result; The dynamic area pulse image is obtained by sorting and combining pixel values ​​of all the dynamic area pulse images.

5. The method for combined processing of pulse visual signal image reconstruction and target detection according to claim 4, characterized in that: The step of obtaining the pixel value of the dynamic region pulse image based on the comparison result includes: When the comparison result indicates that the stable pulse frame number sequence does not exceed the binarization threshold, obtaining the dynamic region pulse image pixel value based on the first value; When the comparison result indicates that the stable pulse frame number sequence exceeds the binarization threshold, the dynamic region pulse image pixel value is obtained based on the second value.

6. The method for combined processing of pulse visual signal image reconstruction and target detection according to claim 2, characterized in that: The filtering and connectivity processing of the dynamic region pulse image to obtain the dynamic target detection position includes: performing filtering processing on the dynamic region pulse image to obtain a filtered pulse image; Based on connected component analysis, the filtered pulse images of the plurality of pixel points are connected to obtain the dynamic target detection position.

7. The method for combined processing of pulse visual signal image reconstruction and target detection according to claim 6, characterized in that: The filtering process on the dynamic region pulse image to obtain a filtered pulse image includes: performing summation filtering processing on the dynamic region pulse image based on the first filtering window and the first filtering pixel value to obtain an initial filtered image; Based on a preset corrosion element value, the initial filtered image is corroded to obtain a corroded filtered image; Based on a preset expansion element value, the eroded filter image is expanded to obtain an expanded filter image; Based on the second filtering window and the second filtering pixel value, the dilated filtered image is subjected to summation filtering processing to obtain the filtered pulse image.

8. A pulse visual signal image reconstruction and target detection joint processing system, characterized in that: The system comprises: an acquisition module, configured to acquire a pulse visual signal sequence of a plurality of pixel points of an observed scene, and obtain a pulse interval sequence based on an interval between non-zero pulse elements in each of the pulse visual signal sequences; a stable partitioning module, configured to perform a stable partitioning calculation on the pulse interval sequence based on the maximum absolute difference between any two elements in each pulse interval sequence, to obtain a stable pulse frame number sequence and a corresponding partition number sequence; a target detection position calculation module, configured to perform dynamic region detection based on the stable pulse frame number sequence to obtain a dynamic region pulse image, and perform filtering and connectivity processing on the dynamic region pulse image to obtain a dynamic target detection position; An image reconstruction module is used to calculate a reconstructed image based on the ratio of each stable pulse duration frame number sequence to the corresponding divided division number sequence, and obtain an observation result of the observation scene based on the reconstructed image and the dynamic target detection position.

9. An electronic device, characterized in that: It includes a memory and a processor, the memory stores a computer program, and is characterized in that when the processor executes the computer program, it implements the pulse visual signal image reconstruction and target detection joint processing method described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for jointly processing pulse visual signal image reconstruction and target detection according to any one of claims 1 to 7 is implemented.

Images