Target detection method, electronic device, vehicle, and computer program product

Abstract

The present disclosure provides a target detection method, an electronic device, a vehicle and a computer program product, and relates to the technical field of computers. The target detection method comprises: acquiring state data of a target region in an image frame of an image frame sequence; inputting the state data of the target region into a recurrent spiking neural network for feature extraction to determine a state feature vector of the target region; constructing an utility function based on the state feature vector of the target region; calculating a Nash equilibrium point of the utility function to determine a parameter adjustment strategy of the recurrent spiking neural network; and updating the recurrent spiking neural network based on the parameter adjustment strategy, performing target detection on the target region through the updated recurrent spiking neural network to determine a detection result.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a target detection method, electronic equipment, vehicle, and computer program product. Background Technology

[0002] Traditional object detection methods based on artificial neural networks rely on high-precision floating-point operations and dense data streams, resulting in huge computational overhead and energy consumption in continuous video processing, making it difficult to meet the deployment requirements of edge devices. Although spiking neural networks are considered a potential low-power solution due to their event-driven and sparse computation characteristics, most existing spiking neural networks adopt static network structures, and their parameters are fixed after training, lacking the ability to adaptively adjust according to dynamic scenes. Summary of the Invention

[0003] This disclosure provides a target detection method, electronic device, vehicle, and computer program product.

[0004] According to one aspect of this disclosure, a target detection method is provided, comprising: acquiring state data of a target region in an image frame sequence, the state data being used to describe the visual attributes, motion features, and spatiotemporal context information of the target region; inputting the state data of the target region into a recurrent spiking neural network for feature extraction to determine a state feature vector of the target region; constructing a utility function based on the state feature vector of the target region, with detection accuracy, detection real-time performance, and detection energy consumption as optimization objectives; calculating the Nash equilibrium point of the utility function to determine a parameter adjustment strategy for the recurrent spiking neural network, the parameter adjustment strategy being used to optimize the spiking threshold, synaptic weights, and / or network inference paths of neurons in the recurrent spiking neural network; and updating the recurrent spiking neural network based on the parameter adjustment strategy, performing target detection on the target region using the updated recurrent spiking neural network, and determining a detection result containing the category, confidence level, and location information of the detected target in the target region.

[0005] Based on one aspect of object detection methods, a deep fusion mechanism of recurrent spiking neural networks and multi-objective game optimization is constructed to achieve intelligent adaptive capabilities in complex time-varying environments. This overcomes the limitations of traditional models, such as fixed parameters, difficulty in balancing accuracy, real-time performance, and energy consumption, enabling dynamic and autonomous decision-making of the optimal operating mode according to the scenario. This closed-loop optimization mechanism based on Nash equilibrium not only improves the robustness and energy efficiency of object detection but also effectively avoids oscillations and instability during strategy adjustment, ensuring a consistently high-performance, low-latency, and high-reliability synergistic optimization state during long-term continuous operation.

[0006] According to at least one embodiment of the target detection method of this disclosure, the state data of the target region is input into a recurrent spiking neural network for feature extraction to determine the state feature vector of the target region, including: performing pulse encoding on the state data of the target region to determine the pulse sequence of the neuron node corresponding to the target region; and inputting the pulse sequence into the recurrent spiking neural network to determine the state feature vector of the target region.

[0007] According to at least one embodiment of the target detection method of this disclosure, before inputting the state data of the target region into a recurrent spiking neural network for feature extraction and determining the state feature vector of the target region, the method includes: encoding the spatial adjacency relationship of the target region in the image frame into an adjacency matrix; and converting the adjacency matrix of the target region into the synaptic weights of the corresponding neuron nodes based on the mapping relationship between the target region and the neuron nodes in the recurrent spiking neural network.

[0008] According to at least one embodiment of the target detection method of this disclosure, a utility function is constructed based on the state feature vector of the target region, with detection accuracy, detection real-time performance and detection energy consumption as optimization objectives. The method includes: converting the state feature vector of the target region into a revenue indicator based on the mapping relationship between the state feature vector and the revenue indicator, wherein the revenue indicator is used to quantify the state feature vector; and constructing a utility function based on the revenue indicator, with detection accuracy, detection real-time performance and detection energy consumption as optimization objectives.

[0009] According to at least one embodiment of the target detection method of this disclosure, the recurrent spiking neural network is updated based on the parameter adjustment strategy, and the updated recurrent spiking neural network is used to detect targets in the target region to determine a detection result containing the category, confidence level, and location information of the detected targets in the target region. The method includes: iteratively updating the recurrent spiking neural network based on the parameter adjustment strategy to determine the updated recurrent spiking neural network; calculating the difference between the parameter adjustment strategy and the parameter adjustment strategy of the previous iteration; stopping the iteration when the difference is less than a target threshold twice consecutively; and using the recurrent spiking neural network after the iteration to detect targets in the image frames of the image frame sequence to determine a detection result containing the category, confidence level, and location information of the detected targets in the target region.

[0010] According to at least one embodiment of the target detection method of this disclosure, updating the recurrent spiking neural network based on the parameter adjustment strategy includes: determining priorities corresponding to detection accuracy, detection real-time performance, and detection energy consumption based on the parameter adjustment strategy; and updating the pulse firing threshold of the neuron nodes in the recurrent spiking neural network based on the priorities, wherein the pulse firing threshold is used to control the activation conditions of the neuron nodes.

[0011] According to at least one embodiment of the target detection method of this disclosure, obtaining state data of a target region in an image frame sequence includes: dividing the image frames of the image frame sequence according to a target time step to determine multiple sliding time windows; extracting the target region in each image frame of the sliding time window to determine the state data of the target region.

[0012] According to another aspect of this disclosure, an electronic device is provided, comprising: a memory storing execution instructions; and a processor executing the execution instructions stored in the memory, causing the processor to perform a target detection method according to any embodiment of this disclosure.

[0013] According to another aspect of this disclosure, a vehicle is provided that includes the aforementioned electronic equipment.

[0014] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the target detection method of any embodiment of this disclosure. Attached Figure Description

[0015] The accompanying drawings illustrate exemplary embodiments of the present disclosure and, together with the description thereof, serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0016] Figure 1 This is a schematic diagram of the overall process of a target detection method according to one embodiment of the present disclosure.

[0017] Figure 2 This is a flowchart illustrating the process of determining state data of a target region in a target detection method according to one embodiment of the present disclosure.

[0018] Figure 3 This is a flowchart illustrating the process of determining the state feature vector of a target region in a target detection method according to one embodiment of the present disclosure.

[0019] Figure 4 This is a schematic flowchart illustrating the process of determining the synaptic weights of neuron nodes in a target detection method according to one embodiment of the present disclosure.

[0020] Figure 5 This is a flowchart illustrating the construction of a utility function in a target detection method according to one embodiment of the present disclosure.

[0021] Figure 6 This is a schematic flowchart illustrating the process of determining a detection result in a target detection method according to one embodiment of the present disclosure.

[0022] Figure 7 This is a schematic diagram of the process of updating a recurrent spiking neural network in a target detection method according to one embodiment of the present disclosure.

[0023] Figure 8 This is a schematic diagram of the structure of a target detection method according to one embodiment of the present disclosure.

[0024] Figure 9 This is a schematic flowchart of a target detection method according to one embodiment of the present disclosure.

[0025] Figure 10 This is a schematic diagram of the target detection result in a road video using a target detection method according to one embodiment of the present disclosure.

[0026] Figure 11 This is a schematic diagram of the Pareto optimal solution set obtained by the target detection method and comparison algorithm according to one embodiment of the present disclosure on the multi-objective optimization function of target detection.

[0027] Figure 12 This is a schematic diagram showing the results of the average and standard IGD values ​​of the target detection method and comparison algorithm according to one embodiment of the present disclosure.

[0028] Figure 13 This is a schematic diagram showing the results of the average and standard HV values ​​of the target detection method and the comparison algorithm according to one embodiment of the present disclosure.

[0029] Figure 14 This is a schematic structural block diagram of a target detection device according to one embodiment of the present disclosure.

[0030] Figure 15 This is a schematic structural block diagram of an electronic device according to one embodiment of the present disclosure. Detailed Implementation

[0031] The present disclosure will now be described in further detail with reference to the accompanying drawings and examples. It should be understood that the specific examples described herein are for illustrative purposes only and are not intended to limit the scope of the disclosure. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present disclosure are shown in the accompanying drawings.

[0032] It should be noted that, where there is no conflict, the embodiments and features described in this disclosure can be combined with each other. The technical solutions of this disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0033] When autonomous vehicles navigate complex urban traffic scenarios, they must continuously address dynamic challenges such as sudden changes in target density (e.g., entering an intersection from an open road), large differences in target scale (coexistence of small pedestrians at a distance and large vehicles at close range), and sudden environmental interference (e.g., visual blurring caused by rain or fog). When vehicles enter high-density areas, adhering to a high-precision mode will lead to a surge in computational load, causing response delays and jeopardizing driving safety. Switching to a lightweight mode to ensure real-time performance may result in missing critical small targets. Simultaneously, on battery-powered vehicle platforms, continuous high-performance computing accelerates energy consumption and affects range. Furthermore, traditional models cannot allocate resources based on the most pressing needs (e.g., ensuring the completeness of pedestrian detection rather than pursuing the highest confidence for all targets).

[0034] To address this issue, this disclosure proposes a target detection method that utilizes a recurrent spiking neural network to extract state features rich in spatiotemporal context from the target region, and dynamically constructs a multi-objective utility function based on these state features. A parameter adjustment strategy is generated by solving for the Nash equilibrium point, and this strategy is used to update the recurrent spiking neural network to achieve target detection in the target region. For example, when entering high-density intersections, it can automatically enhance the response sensitivity to critical targets such as small-scale pedestrians to ensure safety; in regular road sections, it actively suppresses redundant calculations to extend the driving range. This achieves a leap from static inference to adaptive control, effectively avoiding performance fluctuations caused by mode switching, and improving detection real-time performance and energy efficiency while ensuring high detection reliability.

[0035] To facilitate description and make the technical solutions of this disclosure easier to understand, the terminology of this disclosure will be explained before describing the technical solutions of this disclosure.

[0036] Recurrent Spike Neural Networks (RSNNs) are a third-generation artificial neural network that simulates the information processing mechanism of biological nervous systems. They combine the event-driven and sparse coding characteristics of Spike Neural Networks (SNNs) with the temporal memory capabilities of Recurrent Neural Networks (RNNs).

[0037] In cloud server scenarios, the method disclosed herein can be used in city-level intelligent traffic monitoring centers to efficiently process massive amounts of camera video streams concurrently, significantly reducing the overall energy consumption and heat dissipation costs of data centers while ensuring high detection accuracy. On edge terminal devices, such as autonomous driving vehicle computing units, drone flight control systems, or mobile robot main control boards, the method disclosed herein can achieve stable and reliable target perception under conditions of limited computing power and energy, extending device battery life and improving operational safety in complex environments. In mobile devices such as smartphones or AR / VR headsets, the method disclosed herein can be used for tasks such as real-time augmented reality navigation and gesture interaction recognition, dynamically adjusting detection sensitivity and power consumption modes according to the user's environment. For example, in strong outdoor light environments, it prioritizes robust recognition of key targets, while switching to energy-saving mode to extend battery life during static browsing. Furthermore, it can be applied to edge intelligence scenarios such as intelligent inspection robots in the Industrial Internet of Things, unmanned plant protection drones in smart agriculture, and wearable health monitoring devices, providing support for various systems requiring long-term, efficient, and adaptive visual perception.

[0038] Figure 1 A schematic diagram illustrating the overall flow of a target detection method according to one embodiment of this disclosure is shown. Figure 1 The method M100 shown includes steps S110 to S150. This method can be executed by a mobile phone, tablet computer, or in-vehicle computing unit.

[0039] In step S110, the state data of the target region in the image frames of the image frame sequence is obtained. The state data is used to describe the visual attributes, motion features and spatiotemporal context information of the target region.

[0040] Image frame sequences are extracted from the video stream, and target regions within these frames are identified. For each target region, static appearance attributes (visual attributes), inter-frame variation patterns (motion features), and interaction relationships with the environment and other target regions (spatiotemporal context information) are extracted in parallel. These heterogeneous information samples are then standardized and fused to form unified state data for the target region.

[0041] Preferably, in some embodiments of this disclosure, it may include, for example: Figure 2 Steps S1101 to S1102 are shown.

[0042] In step S1101, the image frames of the image frame sequence are divided according to the target time step to determine multiple sliding time windows.

[0043] The sequence of image frames arranged chronologically is sliced ​​according to the target time step and window length to generate a series of continuous and partially overlapping time windows. Each sliding time window contains multiple consecutive image frames, serving as the basic processing unit for temporal feature fusion in the recurrent spiking neural network. This ensures that the recurrent spiking neural network can stably estimate the state of the target region based on short-term historical information and provides a temporally consistent basis for subsequent state data extraction and adaptive optimization.

[0044] In step S1102, the target region is extracted from each image frame of the sliding time window to determine the state data of the target region.

[0045] For the target region of the image frame within each sliding time window, a set of state data containing visual, temporal, and environmental features is extracted and used as a direct input source for deep temporal modeling and adaptive optimization of the recurrent spiking neural network, thereby improving the understanding and response decision-making of complex dynamic scenes.

[0046] In step S120, the state data of the target region is input into a recurrent spiking neural network for feature extraction to determine the state feature vector of the target region.

[0047] Structured multidimensional state data is input into a recurrent spiking neural network (RSNN). Leveraging the dynamics and feedback connections of neurons within the RSNN, information accumulation and interaction occur over multiple time steps. Finally, state feature vectors are generated from the hidden layer activities of the neurons. These state feature vectors serve as the foundational input for constructing multi-objective utility functions and solving Nash equilibrium, reflecting a comprehensive understanding of the current target behavior patterns and the environmental situation.

[0048] In step S130, a utility function is constructed based on the state feature vector of the target region, with detection accuracy, detection real-time performance and detection energy consumption as optimization objectives.

[0049] By mapping and quantifying the detection accuracy potential, real-time constraints, and energy consumption level of the target detection method disclosed in this paper under the current scenario using state feature vectors, a comprehensive utility function containing three weighted indices is constructed. The utility function not only reflects the current state of each target region, but also reflects its relative importance in a specific context, providing a clear optimization direction and mathematical foundation for subsequent solutions to the Nash equilibrium point and the generation of parameter adjustment strategies.

[0050] In step S140, the Nash equilibrium point of the utility function is calculated, and the parameter adjustment strategy of the recurrent spiking neural network is determined. The parameter adjustment strategy is used to optimize the spiking threshold, synaptic weights and / or network inference path of the neuron nodes in the recurrent spiking neural network.

[0051] Detection accuracy, real-time performance, and energy consumption are considered as participants in a Nash equilibrium game, each pursuing its own utility maximization. Based on the utility function, a mathematical optimization method is used to search for the stable point in the policy space of each optimization objective, i.e., the Nash equilibrium point. At the Nash equilibrium point, unilaterally changing the policy of any optimization objective cannot further improve its own payoff. The optimal combination determined thus is the parameter adjustment strategy, used to guide the recurrent spiking neural network to dynamically adjust its internal key parameters, achieving targeted resource allocation and adaptive switching of behavioral patterns.

[0052] In step S150, the recurrent spiking neural network is updated based on the parameter adjustment strategy. The updated recurrent spiking neural network is used to detect targets in the target region and determine the detection results, which include the category, confidence level, and location information of the detected targets in the target region.

[0053] The parameter adjustment strategy is used as a control signal to apply to the internal adjustable parameter set of the recurrent spiking neural network, including the spiking threshold of neuron nodes, synaptic connection weights, and the selection of network inference paths. After the recurrent spiking neural network completes the parameter update, it performs forward inference again on the target region in the current or subsequent image frames, and finally outputs the detection results containing the detected target category, confidence level, and bounding box position, enabling performance self-evolution and accurate task response in dynamic environments.

[0054] Therefore, the target detection method disclosed in this paper achieves accurate characterization of the dynamic behavior of the detected target in the target region through sliding time windows and multi-dimensional state modeling. Recurrent spiking neural networks are used to extract temporal features, enhancing the understanding of complex scenes. By constructing a multi-objective utility function and solving for the Nash equilibrium, a parameter adjustment strategy is generated to autonomously balance detection accuracy, real-time performance, and energy consumption. Ultimately, online adaptive optimization is achieved, maintaining a balance between high performance, low latency, and high energy efficiency even under environmental changes or resource constraints, thus improving the intelligence and practicality of target detection.

[0055] Regarding step S120, the state data of the target region is input into a recurrent spiking neural network for feature extraction to determine the state feature vector of the target region. In some embodiments of this disclosure, this may include, for example... Figure 3 Steps S1201 to S1202 are shown.

[0056] In step S1201, pulse encoding is performed on the state data of the target region to determine the pulse sequence of the neuron node corresponding to the target region.

[0057] The state data of the target region is converted into a set of temporal pulse sequences using a specific encoding strategy, and then assigned to the corresponding input layer neurons in the recurrent spiking neural network. These pulse sequences, with their precise temporal distribution or frequency characteristics, carry the original state information of the target region, driving the recurrent spiking neural network to perform subsequent spatiotemporal feature fusion and dynamic inference. This is a prerequisite for starting the entire recurrent spiking neural network processing flow.

[0058] In step S1202, the pulse sequence is input into the recurrent spiking neural network to determine the state feature vector of the target region.

[0059] The encoded target region pulse sequence is input into a recurrent spiking neural network (RSN). Through the accumulation of membrane potentials, pulse firing, and recursive connection feedback mechanisms of the RSN's neurons over multiple time steps, nonlinear transformation and context integration of temporal information are achieved. After the target time period, the activity patterns of the hidden layer neurons are collected and aggregated into a fixed-dimensional state feature vector, serving as a high-level semantic representation reflecting the target's behavioral trends and environmental conditions for subsequent multi-objective optimization.

[0060] Therefore, by converting the multidimensional state data of the target region into a temporal pulse sequence through pulse coding, an efficient mapping from traditional perceptual information to a brain-like computing format is achieved, preserving key temporal dynamic characteristics. Deep spatiotemporal modeling of the pulse sequence is performed using a recurrent spiking neural network, and semantically rich state feature vectors are extracted through membrane potential accumulation and recursive feedback mechanisms, significantly enhancing the understanding of target behavioral trends. The event-driven, sparse computation, and long-term memory advantages of spiking neural networks are fully utilized, reducing redundant energy consumption while ensuring high expressive power, providing a high-quality and interpretable intermediate representation foundation for subsequent multi-objective adaptive optimization.

[0061] Regarding step S120, before inputting the state data of the target region into a recurrent spiking neural network for feature extraction to determine the state feature vector of the target region, in some embodiments of this disclosure, steps such as... Figure 4 Steps S410 to S420 are shown.

[0062] In step S410, the spatial adjacency relationship of the target region in the image frame is encoded into an adjacency matrix.

[0063] Each target region in an image frame is treated as a graph node, and connections are established based on their proximity or interaction potential in the image space. These binary relationships are then organized into a numerical adjacency matrix. The adjacency matrix is ​​used to explicitly model the mutual influence and cooperative behavior between target regions, enhancing the recurrent spiking neural network's ability to understand complex traffic flows or crowd dynamics.

[0064] In step S420, based on the mapping relationship between the target region and the neuron nodes in the recurrent spiking neural network, the adjacency matrix of the target region is converted into the synaptic weights of the corresponding neuron nodes.

[0065] By leveraging the mapping relationship between target regions and neuron nodes in a recurrent spiking neural network (RSN), the adjacency matrix is ​​used as prior knowledge. The element values ​​(0, 1, or weighted values) of the adjacency matrix are mapped to the initial weights or gain coefficients of the synaptic connections between corresponding neuron node pairs. In this way, the pathways of neuron nodes corresponding to spatially adjacent target regions are enhanced, thereby promoting the interaction and integration of relevant information during spike propagation and improving the RSN's ability to model group behavior and contextual dependencies.

[0066] Therefore, by encoding the spatial adjacency relationships of target regions in an image frame into an adjacency matrix and transforming it into synaptic weights between neuron nodes in a recurrent spiking neural network, a dynamic mapping from scene topology to neural network connection strength is achieved. This enables the recurrent spiking neural network to adaptively enhance the connection strength of relevant neural pathways based on the actual spatial relationships between target regions, improving its ability to model complex contextual scenarios such as group behavior or occlusion interactions. By introducing structured prior knowledge, the information integration efficiency and reasoning interpretability of the spiking neural network are enhanced, maintaining stable detection and tracking performance even in high-density target environments, effectively supporting intelligent perception and response to complex dynamic scenes.

[0067] Regarding step S130, based on the state feature vector of the target region, a utility function is constructed with detection accuracy, detection real-time performance, and detection energy consumption as optimization objectives. Prior to step S120, in some embodiments of this disclosure, it may include, for example... Figure 5 Steps S1301 to S1302 are shown.

[0068] In step S1301, based on the mapping relationship between the state feature vector and the revenue index, the state feature vector of the target area is converted into a revenue index, which is used to quantify the state feature vector.

[0069] Based on the mapping relationship between state feature vectors and benefit indicators, the state feature vectors are decoded into a set of specific performance prediction values, i.e., benefit indicators. Benefit indicators typically include detection accuracy potential, real-time guarantee level, and energy consumption cost estimation, which respectively reflect the expected performance of each optimization objective in the current state, and serve as the basic input for constructing the comprehensive utility function and subsequent Nash equilibrium solution.

[0070] In step S1302, a utility function is constructed based on the benefit index, with detection accuracy, detection real-time performance and detection energy consumption as optimization objectives.

[0071] By mapping state feature vectors to quantifiable performance metrics such as detection accuracy, real-time performance, and energy consumption, dynamic evaluation of the system's performance potential is achieved. A multi-objective utility function is constructed based on these metrics, allowing conflicting optimization objectives to be weighed and compared within a unified framework. This provides a clear optimization direction for subsequent Nash equilibrium solutions, enabling the system to adaptively adjust its behavior strategies according to real-world scenarios. In complex and dynamic environments, this achieves an optimal balance between accuracy, response speed, and resource consumption, thereby improving the intelligence and operational efficiency of the target detection system.

[0072] Regarding step S150, the recurrent spiking neural network is updated based on a parameter adjustment strategy. The updated recurrent spiking neural network is then used to detect targets in the target region, determining a detection result that includes the category, confidence level, and location information of the detected targets within the target region. In some embodiments of this disclosure, this may include, for example... Figure 6 Steps S1501 to S1502 are shown.

[0073] In step S1501, the recurrent spiking neural network is iteratively updated based on the parameter adjustment strategy to determine the updated recurrent spiking neural network.

[0074] Guided by a parameter adjustment strategy, key adjustable parameters of the recurrent spiking neural network (such as neuron spiking thresholds, synaptic weights, or network inference paths) are progressively or abruptly updated according to the target time period or event-triggered mechanism. By iteratively applying this parameter adjustment strategy, the recurrent spiking neural network gradually converges to the optimal configuration state under the current environment, ultimately forming a recurrent spiking neural network with stronger scene adaptability for subsequent object detection tasks.

[0075] Preferably, in some embodiments of this disclosure, it may include, for example: Figure 7 Steps S710 to S720 are shown.

[0076] In step S710, based on the parameter adjustment strategy, the priorities corresponding to detection accuracy, detection real-time performance, and detection energy consumption are determined.

[0077] By analyzing the parameter adjustment strategies, the degree to which each optimization objective plays a dominant role in this adjustment is determined, thereby determining their relative priority. For example, if the parameter adjustment mainly involves increasing weights and lowering thresholds, then accuracy is prioritized; if the main focus is on increasing thresholds or switching lightweight paths, then energy saving or real-time performance is prioritized.

[0078] In step S720, the pulse firing threshold of the neuron nodes in the recurrent spiking neural network is updated based on priority. The pulse firing threshold is used to control the activation conditions of the neuron nodes.

[0079] Based on the priorities of detection accuracy, real-time performance, and energy consumption, the pulse firing thresholds of various neurons in the recurrent spiking neural network are dynamically adjusted. When a certain optimization objective is given higher priority, the neuron threshold of the corresponding functional pathway is lowered to enhance the activation probability, or the neuron threshold of non-critical paths is raised to suppress redundant responses. This fine-grained control alters the information flow pattern and computational load distribution of the recurrent spiking neural network, thereby achieving adaptive reconstruction of performance characteristics.

[0080] By analyzing parameter adjustment strategies, the relative priorities of detection accuracy, real-time performance, and energy consumption are dynamically determined, enabling explicit identification and contextualized focusing of optimization targets. Based on this priority, the spiking thresholds of neuron nodes are updated differentially, tilting the resources of the recurrent spiking neural network towards high-priority targets. This enhances responses in key pathways and suppresses activation in unnecessary regions, effectively regulating information flow and computational load. Fine-grained, low-overhead adaptive reconstruction is achieved, balancing performance requirements and resource constraints in complex and variable scenarios, thus improving the intelligence, flexibility, and operational efficiency of the recurrent spiking neural network.

[0081] In step S1502, the difference between the parameter adjustment strategy and the parameter adjustment strategy of the previous iteration is calculated. When the difference is less than the target threshold twice in a row, the iteration is stopped. The target region in the image frame sequence is detected by the recursive spiking neural network after the iteration is completed, and the detection result containing the category, confidence and location information of the detected target in the target region is determined.

[0082] By comparing the difference between the Euclidean distance or Manhattan difference between the current iteration and the parameter adjustment strategy of the previous iteration, if the difference is lower than the target threshold for two consecutive iterations, the recurrent spiking neural network is considered to have entered a convergent state and no further adjustment is needed. Subsequently, the finally stable recurrent spiking neural network is used to perform a standard object detection task on subsequent image frames, outputting complete detection results including category, confidence, and location information.

[0083] A progressive optimization of the recurrent spiking neural network (RSN) is performed based on a parameter tuning strategy, and priority analysis is used to achieve a dynamic trade-off between accuracy, real-time performance, and energy consumption. By fine-grained adjustment of neuron firing thresholds, network resources are allocated on demand, improving the responsiveness of critical task pathways. A convergence criterion based on the difference in policy changes is introduced, stopping iteration when two consecutive changes are less than a threshold, ensuring stable and efficient operation. Ultimately, the RSN achieves adaptive reconstruction and accurate detection in complex environments, ensuring performance while avoiding over-tuning.

[0084] The technical solution disclosed herein will be further explained below using an autonomous driving scenario as an example.

[0085] Multi-objective optimization based on object detection addresses the conflicting core objectives of object detection tasks, such as accuracy, real-time performance, and adaptability. It employs optimization algorithms to find the optimal set of solutions (Pareto front). Instead of pursuing the absolute optimum of a single objective (e.g., improving only accuracy), it seeks a dynamic balance among multiple conflicting objectives. This ensures detection accuracy in complex scenarios (e.g., moving or blurred targets) while meeting the inference speed requirements of real-time applications (e.g., autonomous driving, real-time monitoring), and allows the model to adapt to changes in different scenarios. For complex scenarios, when constructing the multi-objective optimization function, accuracy, real-time performance, and adaptability must first be quantified, and then the priority of each optimization objective is balanced using weight coefficients.

[0086] The detection accuracy is evaluated using the arithmetic mean of the average precision and average recall of the image / video sequences. This approach balances the accuracy and completeness of the detection results. The detection results for the video sequence (i.e., the image frame sequence) and all image frames are compared with the ground truth annotations as follows: (1) in, X Represents an image frame sequence. θ The IoU threshold, The average of true instances (meeting the IoU threshold and matching the class) across all image frames. The average of false positives across all image frames. This is the average of false negatives across all image frames. The function ranges from [0, 1] and directly integrates precision and recall through an arithmetic mean. The closer the value is to 1, the more accurate (fewer false positives) and complete (fewer false negatives) the detection results are.

[0087] The real-time performance of object detection is typically measured by processing speed, i.e., the number of image frames that can be processed per unit of time. For object detection of people or vehicles in road videos, the mathematical function expression for real-time performance can be defined as: (2) in, q The original frame rate of the video (unit: frames per second, FPS). Y This indicates the total number of frames contained in the video. D After the detection algorithm has processed all Y Total frame time (in seconds). This represents the actual frame rate (FPS) processed by the detection algorithm. This is an indicator function used to determine whether the detection algorithm meets real-time standards (i.e., whether the processing speed can keep up with the video playback speed). It returns 1 when the condition is met (processing frame rate ≥ original frame rate), and 0 otherwise. This is the core indicator of the real-time mathematical function. It directly reflects the processing speed of the detection algorithm (the higher the FPS value, the better the real-time performance).

[0088] Adaptability measures the performance stability of the target detection method of this disclosure under scene changes. It is calculated by taking the reciprocal of the cross-scene accuracy fluctuation coefficient. First, 3-5 typical scenes (such as sunny / rainy, day / night, indoor / outdoor) are selected, and the mean accuracy of the target detection method of this disclosure in each scene is calculated. Then, the standard deviation of the mean accuracy is calculated. σ The adaptability measure is defined as: (3) Its value ranges from (0, 1], and the higher the value, the smaller the impact of scene changes on accuracy and the stronger the adaptability.

[0089] Since the goals of detection accuracy, real-time performance, and adaptability are to be maximized, and the priorities of these three aspects need to be balanced, a weighted linear combination function (suitable for scenarios where priorities can be quantified) is adopted, in the following form: Max F ( x )= ω 1 f 1+ ω 2 f 2+ ω 3 f 3 (4) in, x This represents optimization variables (such as the structural parameters, training hyperparameters, and inference acceleration strategies of a recursive spiking neural network). ω 1. ω 2. ω 3 represents the weighting coefficients for each optimization objective. Through weight allocation, improving one optimization objective does not excessively sacrifice other objectives, ultimately finding... F ( x The maximum solution is obtained to achieve multi-objective balance in complex scenarios.

[0090] Furthermore, recurrent spiking neural networks are based on a spiking neuron node model, with the core being the Integrate-and-Fire (IF) logic. When the neuron's membrane potential accumulates to a threshold, it fires a pulse, after which the potential is reset. The dynamic equation for the membrane potential is as follows: (5) in, τ m The membrane time constant describes the rate of potential decay. V i ( t ) is a neuron node i At any moment t The membrane potential;I i ( t ) is a neuron node i The total input current received (including synaptic input, external stimuli, etc.). When V i ( t )≥ V th ( V th When the firing threshold is reached, the neuron fires a pulse, and then... V i ( t Reset to resting potential V rest : (6) Neuron state depends on current input and historical impulse feedback (recursive nature). Postsynaptic currents (triggered by impulses from preceding neurons) are typically modeled using exponential decay. (7) in, ω ji Neuron nodes j To neuron nodes i Synaptic weights; t j f Neuron nodes j The pulse emission timing; τ s is the synaptic current decay time constant.

[0091] The loss function is defined as a measure of the cumulative time difference between the output pulse and the target pulse: (8) in, It is the output layer of a neuron node in a recurrent spiking neural network. k The timing of pulse emission, It is the corresponding moment of the target pulse.

[0092] Synaptic weight ω The update depends on the gradient of the loss function with respect to the parameters, and gradient descent is used: (9) in, This indicates the updated synaptic weights. Indicates the current synaptic weight. Represents the gradient. η This is the learning rate.

[0093] The recursive nature of recurrent spiking neural networks (RSNs) means that the network state depends on historical data, and parameter updates must consider dynamic feedback over time. Therefore, the Backpropagation Through Time (BPTT) method is employed, which requires differentiating the temporal dependency of historical pulses, current state, and loss. The core of this method is to expand the recursion along the time axis, calculate the gradient, and accumulate it. In BPTT, the gradient needs to integrate the contributions from all time steps. (10) in, L ( t (time) t Contribution to total loss, Represents neuron nodes i membrane potential Input current to its synapse The partial derivatives, Indicates synaptic input current Synaptic weights The partial derivatives, Error needs to be calculated through reverse time propagation.

[0094] Furthermore, the essence of Nash equilibrium is to find a set of parameter adjustment strategies (corresponding to the hyperparameters / structural configuration of a recurrent spiking neural network) such that the performance improvement of any one optimization objective cannot be achieved without harming the performance of at least one other optimization objective; that is, all optimization objectives reach an optimal stable state where they mutually restrain each other. The multi-objective optimization in object detection is essentially based on the policy space. S In this process, we seek parameter adjustment strategies that simultaneously bring all optimization objectives close to the optimum. s Its mathematical form is: Max s∈S F ( s )=[ f 1( s )+ f 2( s )+ f 3( s )] T (11) In formula (11), there is no unique global optimal solution, only Pareto optimal solution (i.e., it is impossible to improve the solution of all optimization objectives at the same time), and the Nash equilibrium point is the stable point in the Pareto optimal solution set. At this time, the performance improvement of any optimization objective must be at the cost of the performance decline of at least one other optimization objective.

[0095] For the three-objective optimization problem of object detection, the strategy s ∈ SIt is a Nash equilibrium if and only if for all optimization objectives h ∈{ A , B , R There are no other strategies. s ’ ∈ S , so that: (12) in, F h (·) indicates the first h The performance function of each optimization objective ( F 1 =A, F 2 =B, F 3 =R At the Nash equilibrium point... s To improve accuracy A Then real-time requirements must be accepted. B Decline or adaptability R Decrease; similarly, improve real-time performance. B Sacrificing accuracy A or adaptability R None of the three can be improved simultaneously to achieve a stable balance.

[0096] To further quantify the equilibrium conditions, a target utility function is introduced. U h ( s To describe the strategy s For the first h The degree of benefit of each optimization objective (usually related to the performance function) F h ( s (Positive correlation). For a three-objective optimization, the equivalent condition for Nash equilibrium can be expressed as: (13) in, s -h To fix except the first h All other components except the policy component corresponding to each optimization objective (i.e., keeping the optimal policy of other optimization objectives unchanged). S h For the first h The strategy subspace corresponding to each optimization objective; U h ( s ’ h , s -h Only adjust the first h The strategy for each optimization objective iss ’ h When other optimization objective strategies remain unchanged, the first h The utility value of an optimization objective. At the Nash equilibrium point, adjusting the strategy of only a single optimization objective (such as changing only the anchor frame size to improve accuracy) cannot increase the utility of that optimization objective (in fact, it may lead to a decrease in utility due to mismatch between the strategies of other optimization objectives), further demonstrating the unimprovability of the strategy.

[0097] When using a recurrent spiking neural network (RSN) as the core architecture for distributed resource scheduling, the dynamic characteristics of spiking neurons are utilized to extract time-varying and topological features of the network environment. A fully connected layer composed of spiking neurons then decodes and generates a distributed resource scheduling strategy. Specifically, the RSN features are first processed by a dual-path pulse coding module. The time-varying feature end divides the real-time node state (e.g., CPU utilization) into a sliding window, and through spiking neuron membrane potential accumulation and pulse firing, it is transformed into a pulse sequence, with frequency differences reflecting state trends. The topological feature end transforms node connections into an adjacency matrix, and through mapping between nodes and neurons and synaptic connections, pulse transmission and potential coupling simulate node resource dependence. Then, 2-3 layers of recurrent spiking neurons use feedback connections to capture spatiotemporal dependencies (solving the time lag problem) and dynamically update synaptic weights through temporal backpropagation rules to adapt to topological changes. Finally, the fully connected layer of spiking neurons decodes the fused features into a parameter adjustment strategy. Each neuron node corresponds to a different adjustment action, and binary decisions are output according to a multi-threshold rule (high-priority actions have lower thresholds). A feedback mechanism is also used to propagate the executed RSN state back to optimize the parameter adjustment strategy.

[0098] The detection status data of the target region in the image frame (such as detection accuracy, real-time performance, and adaptability) is converted into a pulse sequence, and a rate coding mechanism is used: (14) (15) in, s i ( t ) represents a neuron node i pulse sequence, f i Neuron nodes i pulse firing rate, N i For time windows T The number of pulses within, x i The feature value (such as detection accuracy) of the input state data is positively correlated with the release rate.

[0099] To preserve the temporal memory of the recurrent spiking neural network's detection state features of the target region (through... (To accumulate historical influence), and can also optimize pulse coding parameters (such as the pulse frequency threshold of rate coding) and recursive connection strength through gradient backpropagation, so that the recursive spiking neural network approaches the Nash equilibrium multi-objective optimization state in dynamic adjustment, and the synaptic weights are updated as follows: (16) in, This indicates the amount of synaptic weight update. η For learning rate, L t for t The loss function at each time step (including loss of detection accuracy, loss of real-time constraint, and adaptive penalty term). ω τ for τ Synaptic weights at time points. The recurrent spiking neural network is expanded along the time axis as follows: τ =0 to τ = t The forward computation chain calculates the gradient of the weight's contribution to the final loss at each time step through backpropagation. And accumulate the gradients of all time steps to complete the weight update.

[0100] Based on this, a Nash equilibrium decision-making mechanism is introduced to handle multi-objective optimization needs. The accuracy, real-time performance, and adaptability to complex scenarios of road target detection are taken as the three core optimization objectives, and a multi-objective utility function as shown in formula (11) is constructed. The optimal parameter adjustment strategy is then selected through Nash equilibrium conditions. s That is, when adjusting s Improving the performance of any one optimization objective will inevitably lead to a decrease in the performance of at least one other optimization objective. For example, increasing the pulse coding time window to improve detection accuracy... A ( s If this happens, the increased processing time per frame will affect real-time performance. B ( s The performance decreases if the number of neurons in the recurrent layer is reduced to improve real-time performance. B ( s If the ability to capture features in complex scenes is weakened, the adaptability will be compromised. R ( s The parameter adjustment strategy at the Nash equilibrium point decreases. s This achieves a stable balance among the three aspects. Simultaneously, the output layer of the recurrent spiking neural network uses fully connected neurons with spiking capabilities, corresponding to decision-making actions in target detection (such as prioritizing large vehicles, enhancing the detection weight of small targets, or switching to a fast inference mode). Its spiking threshold is dynamically adjusted based on the target priority after Nash equilibrium. When the road enters a congested scenario and adaptability needs to be prioritized, the threshold of the neuron node corresponding to the action of enhancing feature extraction in complex scenarios is lowered, ensuring that this decision is executed first. This achieves synergistic optimization of target detection accuracy, real-time performance, and adaptability in complex road environments, providing stable and reliable detection results for the autonomous driving perception system.

[0101] like Figure 8 As shown, the input image frame contains target regions of multiple vehicles. After encoding, these regions are converted into pulse sequences and fed into the input layer of a recurrent spiking neural network (RSN). The pulse sequences represent the state data of the target regions. Temporal features are extracted through the RSN, and neurons emit pulses and transmit information through dynamic membrane potential and threshold judgment mechanisms. The RSN calculates the loss function using supervised signals and employs a temporal backpropagation algorithm combined with gradient descent for weight updates and learning. Furthermore, a utility function is constructed with detection accuracy, real-time performance, and energy consumption as objectives. The Nash equilibrium point (marked with a red star) is solved in a three-dimensional optimization space to obtain the optimal parameter adjustment strategy, enabling adaptive control of the RSN and completing a closed-loop optimization process from perception to decision-making.

[0102] Furthermore, in the multi-objective optimization calculation process of object detection, the target detection method disclosed herein requires the completion of basic configuration of recurrent spiking neural network parameters and model structure during the initialization phase. Specifically, this includes: setting the set of neurons participating in the optimization and the target space topology; initializing the physical parameters of each neuron node (such as membrane potential, resting potential, impulse threshold, refractory period, etc.); constructing the hierarchical structure of the recurrent spiking neural network, defining the number of neurons in the input layer (matching the network state feature dimension), the cluster size of neurons in the hidden layer, and the feature dimension of the output layer; randomly initializing the synaptic weight matrix and setting the core parameters of the neurons; and initializing the key parameters of the Nash equilibrium game, including the maximum number of iterations and the convergence accuracy threshold in the objective utility function. Simultaneously, initial target state data (such as target attributes, spatial information, motion features, environmental associations, etc.) is collected, normalized, and used as the first input to the recurrent spiking neural network, providing an initial benchmark for subsequent feature extraction and strategy optimization.

[0103] The calculation process is as follows: Figure 9As shown, pulse encoding is performed on the state data of the target region, converting the state information of the target region into a time-series pulse signal. Neurons continuously accumulate membrane potential based on input pulses; when the membrane potential reaches or exceeds a threshold, the neuron fires a pulse. After pulse firing, weights are updated through temporal backpropagation. The temporal backpropagation algorithm is used to adjust the neural network parameters to optimize performance, outputting the current spatiotemporal feature representation, and then constructing a target utility function U with multiple objectives as its core. h (s) is used to quantify the overall performance of the system in terms of accuracy, real-time performance, and energy consumption. It determines whether the parameter adjustment strategy has reached Nash equilibrium; if not, it returns to continue iterative optimization until the equilibrium condition is met. After reaching equilibrium, it outputs the final parameter adjustment strategy and executes it. During execution, new state data feedback is collected, and the system re-enters the pulse coding stage to form a closed loop, achieving dynamic adaptive adjustment, and finally completing the entire optimization process and ending the process.

[0104] Therefore, the target detection method disclosed herein achieves global balanced optimization of multiple objectives such as detection accuracy, energy consumption, and real-time performance through the collaborative design of recurrent spiking neural networks and Nash equilibrium. The recursive structure of the recurrent spiking neural network accurately captures the temporal features of the target (such as motion trajectory and inter-frame correlation), providing a high-quality feature foundation for multi-objective optimization. The Nash equilibrium strategy coordinates the conflicts among various optimization objectives through multi-dimensional utility functions and dynamic weight allocation. For dynamic scenarios such as target motion and environmental changes (such as sudden changes in illumination and occlusion), the target detection method disclosed herein has active adaptation and anti-interference capabilities. The recursive layer of the recurrent spiking neural network can update the temporal state of the target in real time, avoiding false detections and missed detections caused by the loss of inter-frame information; the Nash equilibrium strategy can dynamically adjust the optimization logic according to the scenario. Furthermore, the target detection method disclosed herein takes into account both performance and lightweight requirements, and is particularly suitable for the computing power / power consumption limitations of edge devices (such as drones and edge cameras). On the one hand, the event-driven nature of recurrent spiking neural networks inherently possesses the advantage of low power consumption; on the other hand, the lightweight Nash equilibrium solution algorithm decomposes the global game into local computations in sub-regions, relying on the local recursive capability of recurrent spiking neural networks to complete the iteration, without relying on high-performance cloud computing power.

[0105] To comprehensively evaluate the performance of the object detection method disclosed herein, experimental verification was conducted on a multi-objective optimization function for object detection. Figure 10 The target detection results of the disclosed target detection method (RSNN-N) in a road video are shown below. Figure 11 The visualization results of each algorithm in the multi-objective optimization function for object detection are presented. Figure 12 and Figure 13 Detailed experimental results of the target detection method and comparison algorithm disclosed herein on the IGD and HV indices are presented.

[0106] Depend on Figure 10 As can be seen, the object detection method of this disclosure performs excellently in road video object detection. The algorithm can accurately identify different targets such as vehicles, pedestrians, and umbrellas, providing high confidence scores; for example, the confidence score for vehicles reaches 0.90 or higher, and for pedestrians, it reaches 0.89 or higher. This indicates that the object detection method of this disclosure can effectively handle complex road scenes, has high accuracy in detecting moving and variable targets, and can well meet the needs of object detection in road videos, providing a reliable object detection foundation for subsequent multi-target optimization.

[0107] Figure 11 This paper presents the Pareto optimal solution set of the object detection method and comparison algorithm disclosed herein on a multi-objective optimization function for object detection, with three objectives. f 1 (Precision) f 2 (Real-time) f 3. (Adaptability) Mutual constraints. As can be seen from the comparison, the solution set of the object detection method disclosed in this paper is closer to the ideal Pareto front, and the distribution is denser and more uniform. This indicates that it performs better in balancing accuracy, real-time performance and adaptability, and can better take into account the three. While ensuring detection accuracy, it improves real-time processing efficiency and environmental adaptability. Compared with the comparison algorithm, the multi-objective optimization effect is better.

[0108] Depend on Figure 12 and Figure 13 It is evident that, among the multi-objective optimization functions for object detection, the object detection method of this disclosure performs optimally. Figure 12 In this study, the target detection method of this disclosure has an average IGD of 4.635e-2 and a standard value of 1.416e-1, ranking first, indicating that the solution set closely approximates the true Pareto front and has good stability. Figure 13 Among the proposed methods, the HV average of the object detection method is 7.567e-1, and the standard value is 5.011e-1, ranking first, demonstrating its wide coverage of the target space and stable results. Compared with MLTC, CNN, and YOLOv5 algorithms, the proposed object detection method has a greater advantage in multi-target optimization performance.

[0109] Based on any of the above embodiments, this disclosure also provides a target detection device.

[0110] Figure 14 This is a schematic block diagram of the target detection device according to one embodiment of the present disclosure.

[0111] like Figure 14 As shown, the target detection device includes: The data acquisition module 1402 acquires the state data of the target region in the image frames of the image frame sequence. The state data is used to describe the visual attributes, motion features and spatiotemporal context information of the target region. The feature extraction module 1404 inputs the state data of the target region into a recurrent spiking neural network for feature extraction to determine the state feature vector of the target region. The utility determination module 1406 constructs a utility function based on the state feature vector of the target region, with detection accuracy, detection real-time performance and detection energy consumption as optimization objectives. The strategy determination module 1408 calculates the Nash equilibrium point of the utility function and determines the parameter adjustment strategy of the recursive spiking neural network. The parameter adjustment strategy is used to optimize the spiking threshold, synaptic weights and / or network inference path of the neuron nodes in the recursive spiking neural network.

[0112] The target detection module 1410 updates the recurrent spiking neural network based on a parameter adjustment strategy, and performs target detection on the target region through the updated recurrent spiking neural network to determine the detection result containing the category, confidence level and location information of the detected target in the target region.

[0113] The aforementioned target detection device can be in the form of computer software, and each module of the aforementioned target detection device can be implemented through computer software modules.

[0114] The specific implementation process of the functions and roles of each module in the above target detection device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0115] This disclosure also provides an electronic device. Figure 15 A schematic diagram of the hardware implementation using the processing system is shown.

[0116] The hardware architecture of electronic devices can be implemented using a bus architecture. A bus architecture can include any number of interconnect buses and bridges, depending on the specific application and overall design constraints of the hardware. Bus 1100 connects various circuits, including one or more processors 1200, memory 1300, and / or hardware modules. Bus 1100 can also connect various other circuits 1400, such as peripherals, voltage regulators, power management circuits, external antennas, etc. Bus 1100 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one connection line is used in this diagram, but this does not indicate that there is only one bus or one type of bus.

[0117] For ease of explanation, certain steps of the above method are described in relation to modules. It should be understood that the corresponding module performing one or more steps of the above method may be one or more hardware modules specifically configured to perform the corresponding step, or implemented by a processor configured to perform the corresponding step, or stored in a computer-readable medium for implementation by a processor, or implemented by some combination thereof.

[0118] According to an embodiment of this application, a vehicle is also provided. The vehicle includes the aforementioned electronic equipment.

[0119] This disclosure also provides a readable storage medium storing a computer program that, when executed by a processor, is used to implement the methods described above. A "readable storage medium" can be any means capable of containing, storing, communicating, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples of a readable storage medium include: an electrical connection with one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable read-only memory (CDROM), etc.

[0120] This disclosure also provides a computer program product, the methods of which can be implemented wholly or partially through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented wholly or partially as a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed, all or part of the processes or functions of this disclosure are performed.

[0121] Computer programs or instructions can be stored in a readable storage medium or transferred from one readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The readable storage medium can be any available medium capable of access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; an optical medium, such as a digital video optical disc; or a semiconductor medium, such as a solid-state drive. The computer-readable storage medium can be a volatile or non-volatile storage medium, or it can include both volatile and non-volatile types of storage media.

[0122] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0123] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0124] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0125] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0126] In the description of this specification, the references to terms such as "one embodiment / mode," "some embodiments / modes," "example," "specific example," or "some examples," etc., refer to specific features, structures, or characteristics described in connection with that embodiment / mode or example, which are included in at least one embodiment / mode or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment / mode or example. Moreover, the specific features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments / modes or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments / modes or examples described in this specification, as well as the features of different embodiments / modes or examples.

[0127] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0128] Those skilled in the art should understand that the above embodiments are merely for illustrating the present disclosure and are not intended to limit the scope of the disclosure. Those skilled in the art can make other changes or modifications based on the above disclosure, and these changes or modifications still fall within the scope of the present disclosure.

Claims

1. A target detection method, characterized in that, include: Acquire state data of a target region in an image frame sequence, wherein the state data is used to describe the visual attributes, motion features, and spatiotemporal context information of the target region; The state data of the target region is input into a recurrent spiking neural network for feature extraction to determine the state feature vector of the target region. Based on the state feature vector of the target region, a utility function is constructed with detection accuracy, detection real-time performance, and detection energy consumption as optimization objectives. Calculate the Nash equilibrium point of the utility function and determine the parameter adjustment strategy of the recurrent spiking neural network. The parameter adjustment strategy is used to optimize the pulse firing threshold, synaptic weight and / or network inference path of the neuron nodes in the recurrent spiking neural network. as well as The recurrent spiking neural network is updated based on the parameter adjustment strategy. The updated recurrent spiking neural network is then used to detect targets in the target region, and a detection result is determined that includes the category, confidence level, and location information of the detected targets in the target region.

2. The target detection method as described in claim 1, characterized in that, The state data of the target region is input into a recurrent spiking neural network for feature extraction to determine the state feature vector of the target region, including: The state data of the target region is pulse-coded to determine the pulse sequence of the neuron node corresponding to the target region; The pulse sequence is input into a recurrent spiking neural network to determine the state feature vector of the target region.

3. The target detection method as described in claim 1, characterized in that, Before inputting the state data of the target region into a recurrent spiking neural network for feature extraction and determining the state feature vector of the target region, the process includes: The spatial adjacency relationship of the target region in the image frame is encoded into an adjacency matrix; Based on the mapping relationship between the target region and the neuron nodes in the recurrent spiking neural network, the adjacency matrix of the target region is converted into the synaptic weights of the corresponding neuron nodes.

4. The target detection method as described in claim 1, characterized in that, Based on the state feature vector of the target region, a utility function is constructed with detection accuracy, detection real-time performance, and detection energy consumption as optimization objectives, including: Based on the mapping relationship between state feature vectors and revenue indicators, the state feature vectors of the target region are converted into revenue indicators, which are used to quantify the state feature vectors. Based on the aforementioned benefit metrics, a utility function is constructed with detection accuracy, detection real-time performance, and detection energy consumption as optimization objectives.

5. The target detection method as described in claim 1, characterized in that, The recurrent spiking neural network is updated based on the parameter adjustment strategy. The updated recurrent spiking neural network is then used to detect targets in the target region, determining a detection result that includes the category, confidence level, and location information of the detected targets in the target region. Based on the parameter adjustment strategy, the recurrent spiking neural network is iteratively updated to determine the updated recurrent spiking neural network. The difference between the parameter adjustment strategy and the parameter adjustment strategy of the previous iteration is calculated. When the difference is less than the target threshold twice in a row, the iteration is stopped. The target region in the image frame sequence is detected by the recursive spiking neural network after the iteration is completed, and the detection result containing the category, confidence and location information of the detected target in the target region is determined.

6. The target detection method as described in claim 1, characterized in that, The recurrent spiking neural network is updated based on the parameter adjustment strategy, including: Based on the parameter adjustment strategy, the priorities corresponding to detection accuracy, detection real-time performance and detection energy consumption are determined. Based on the priority, the pulse firing threshold of the neuron nodes in the recurrent spiking neural network is updated, and the pulse firing threshold is used to control the activation conditions of the neuron nodes.

7. The target detection method as described in claim 1, characterized in that, Obtain the state data of the target region in the image frames of the image frame sequence, including: The image frames in the image frame sequence are divided according to the target time step to determine multiple sliding time windows; The target region is extracted from each image frame of the sliding time window to determine the state data of the target region.

8. An electronic device, characterized in that, include: The memory stores execution instructions; as well as A processor that executes the execution instructions stored in the memory, causing the processor to perform the target detection method according to any one of claims 1 to 7.

9. A vehicle, characterized in that, The vehicle includes the electronic equipment as described in claim 8.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the target detection method according to any one of claims 1 to 7.

Images