Audio-visual data processing method for target recognition in low-light environments

Abstract

The application relates to the technical field of computer audio-visual software, and discloses an audio-visual data processing method for target recognition in a dark light environment, which comprises the following steps: acquiring audio-visual data to be processed and multi-dimensional environment sensing parameters; reconstructing a reference exposure time and determining local shutter parameters of a target object under a dark light working condition; collecting real-time position feedback readings of a voice coil motor at a moment when the local shutter is closed, calculating physical residual distance of the real-time position feedback readings relative to a preset displacement midpoint; mapping the physical residual distance into a feature offset vector to construct a spatial affine inverse transformation matrix, and performing inverse re-projection operation on a feature vector of the target object by using the spatial affine inverse transformation matrix; and the application eliminates feature topological distortion caused by mechanical response lag of the voice coil motor, and realizes alignment of target features and physical real states.

Description

Technical Field

[0001] This invention belongs to the field of computer audiovisual software technology, and particularly relates to an audiovisual data processing method for target recognition in low-light environments. Background Technology

[0002] Currently, in scenarios such as vehicle-mounted assisted driving, high-altitude inspection, and intelligent monitoring, computer audiovisual software makes decisions through integrated artificial intelligence recognition models. This type of software highly relies on audiovisual data streams with high signal-to-noise ratio and stable geometric features as its operating foundation. Existing image acquisition systems typically employ a combination strategy of extending exposure time, increasing sensitivity gain, and integrating optical or electronic image stabilization components to maintain the brightness and visual smoothness of the output image when dealing with low-light conditions. However, when mobile terminals are in environments with high-frequency vibration and insufficient lighting, the physical response of the voice coil motor exhibits mechanical hysteresis. During the transient process of the system capturing moving targets and shortening the exposure pulse, the mechanical displacement compensation action of the physical image stabilization components is in a non-steady-state process that has not fully returned to its original position. This causes the image signal acquired by the photosensitive component to drift at the pixel level. This physical misalignment caused by hardware response delay manifests as nonlinear deformation of the target edge contour after being converted into audiovisual data streams.

[0003] Faced with the physical limitations of voice coil motor stabilization's mechanical response, attempts to overcome distortion solely through software control methods have shortcomings. For example, Chinese invention patent application CN114519832A discloses a video global motion compensation method based on an inverse affine transform model. While this method aims to eliminate global motion deviation, its underlying logic heavily relies on extracting and matching transformation matrices from surface pixels between frames. In low-light conditions with high-frequency micro-vibrations, the physical lag of the voice coil motor causes smearing of the target's feature contours within the transient exposure cycle. A purely electronic compensation approach, based solely on feature comparison of surface-distorted pixel sets and detached from underlying physical constraints, suffers from a fundamental mismatch between core presuppositions and actual boundary conditions: the input feature source used to calculate the inverse transform matrix itself... The distortion is a product of physical lag, leading to distorted mathematical solutions for affine parameters. Simple electronic compensation cannot penetrate the physical dimension error caused by mechanical misalignment, causing the front-end feature extraction stage to be deeply mired in topological distortion. The algorithm structure of traditional audiovisual software is based on the preset foundation of instantaneous synchronization of physical compensation actions, which cannot respond to the local feature coordinate offset caused by mechanical lag. This results in nonlinear topological distortion in the front-end feature extraction stage of the software. Simply increasing the brightness gain will introduce shot noise, which will destroy the underlying data quality required for the feature extraction process. The general electronic cropping method lacks the perception of the real-time status of the physical actuator and is difficult to correct the target feature tensor distortion caused by mechanical lag. This causes deviations when the downstream recognition node extracts edge feature parameters, inducing misjudgments by the decision module or alarm module.

[0004] Therefore, the technical problem to be solved by this invention is how to offset the feature misalignment caused by hardware mechanical lag in the audiovisual data processing link based on the real-time operating status feedback of the physical compensation component, so as to push the feature flow that aligns the geometric topology with the physical reality to the recognition model. Summary of the Invention

[0005] This invention aims to solve the problem of declining recognition rate caused by target feature tensor distortion due to mechanical lag of physical image stabilization components.

[0006] In this technical solution, an audiovisual data processing method for target recognition in low-light environments includes the following steps: Step S1: Obtain the audio-visual data stream to be processed and collect multi-dimensional environmental perception parameters, including ambient light index, global motion state parameters and real-time position feedback reading of voice coil motor. Step S2: Determine the current working condition type through the ambient light illuminance index, and trigger the dark light processing mode when the ambient light illuminance index is lower than the preset brightness threshold. Adjust the baseline exposure time according to the global motion state parameters, and use the depth feature recognition model to lock the motion trajectory vector of the target object in the audiovisual data stream to be processed and the original extracted features of the target object. Step S3: Calculate the local shutter parameters for the target object based on the motion trajectory vector, and use the local shutter parameters to cover the reference exposure time; Step S4: Collect the real-time position feedback reading of the voice coil motor at the moment of partial shutter closure, and calculate the physical residual distance of the real-time position feedback reading relative to the preset displacement midpoint. Step S5: Convert the physical residual distance into a feature offset vector in the image coordinate system according to the preset optical conversion ratio, and construct the spatial affine inverse transformation matrix for the target object using the feature offset vector as the input parameter. Step S6: Based on the spatial affine inverse transformation matrix, the original extracted features, and the preset background low-frequency noise tolerance parameter, perform inverse reprojection operation on the edge feature vector of the target object to eliminate feature topological distortion caused by the mechanical response lag of the voice coil motor and generate target recognition data stream.

[0007] Preferably, the specific steps for triggering the low-light processing mode in step S2 include: step S21, under the condition that the ambient light measurement data continues to decrease, reducing the sensitivity reference value according to the collaborative compensation strategy; step S22, while the sensitivity reference value decreases, extending the reference exposure time using global motion state parameters to acquire the first-level noise reduction image matrix stripped of global noise.

[0008] Preferably, the specific steps for performing the reverse reprojection operation in step S6 include: step S61, extracting the position feedback data of the Hall sensor inside the voice coil motor; step S62, converting the position feedback data into a mechanical hysteresis parameter characterizing the physical response delay of the underlying optical motor; and step S63, performing frame-by-frame edge reprojection on the continuous sequence frames in the audiovisual data stream to be processed based on the mechanical hysteresis parameter.

[0009] Preferably, the method for determining the current working condition type in step S2 is as follows: by judging whether the ambient light illuminance index is less than a preset brightness threshold and whether the global motion state parameter is greater than a preset frequency threshold, the current working condition type is determined to be a low-light high-frequency micro-vibration working condition.

[0010] Preferably, step S6 specifically includes: calculating the coordinate correction value of the feature tensor of the target object based on the electronic algorithm correction weight allocated by the collaborative compensation strategy, and using the coordinate correction value to eliminate the spatial position drift between consecutive sequence frames in the audiovisual data stream to be processed.

[0011] Preferably, step S3 specifically includes: reconstructing the exposure integration time within the area where the target object is located, and performing local gain compensation on the target object based on the feature offset vector during the integration period when the voice coil motor is in an unsteady state.

[0012] Preferably, the method further includes the following steps: Step S7, establishing a characteristic description of the motion feedback parameters of the voice coil motor, injecting the characteristic description as a deterministic compensation variable into the image processing link, and implementing data dimension hedging against high-frequency composite vibration interference.

[0013] Preferably, the multidimensional environmental perception parameters are captured through a data perception layer, which includes a light sensor, a motion vector sensing unit, and a real-time position monitoring module integrated inside the voice coil motor.

[0014] Preferably, the method further includes the following steps: Step S8, inputting the target recognition data stream into the subsequent artificial intelligence recognition model, and using the artificial intelligence recognition model to determine the category of the target object.

[0015] Compared with existing technologies, the audiovisual data processing method for target recognition in low-light environments of this invention has the following advantages: 1. In the audiovisual data processing for target recognition in low-light environments, the real-time physical displacement residual of the optical compensation component during the shutter integration period is mapped into a feature offset vector in the pixel space. This vector is then used to drive the inverse affine transformation of the target feature matrix, thereby achieving dynamic correction of the topological distortion of the feature tensor caused by the mechanical lag of the voice coil motor. This processing method, which transforms the low-level sensor feedback readings into upper-level feature-level data alignment parameters, enables the system to eliminate the nonlinear coordinate displacement between the edge contour of the moving target and the background texture from the data operation dimension without reducing the physical shutter response delay. This provides a low-level audiovisual data stream with geometric rigidity consistency for subsequent visual recognition models.

[0016] 2. By utilizing a multi-dimensional correlation triggering mechanism between ambient light indicators and global motion parameters, the system actively lowers the ambient light reference when the ambient brightness is detected to be below a preset threshold. Simultaneously, it reconstructs exposure parameters by combining local target offset vectors. This local adaptive adjustment logic based on real-time perception feedback optimizes the underlying data preprocessing architecture of computer audiovisual software. It changes the traditional approach of simply relying on increasing the perceived brightness, which introduces destructive noise. By covering the directional parameters of the core subject of interest, it strengthens the acquisition of high-frequency characteristic information of moving targets while maintaining the background signal-to-noise ratio. This improves the feature extraction accuracy of the artificial intelligence recognition model integrated into the audiovisual software at the decision node under complex low-light and micro-vibration conditions.

[0017] 3. Construct a deep parallel closed loop for physical displacement compensation and electronic pixel cropping correction. Combine Hall sensor position feedback data with frame-by-frame edge reprojection algorithm to achieve closed-loop erasure of subtle coordinate drift between consecutive frames. This cross-level data flow and correction strategy integrates the originally separate physical execution components and software processing levels into a unified phase alignment pipeline. On the physical basis of ensuring sufficient light intake in dark areas, coordinate recalibration in the logical dimension ensures a rigorous mapping relationship between the target outline in the output audiovisual stream and its physical reality, avoiding misjudgments by the back-end decision module. Attached Figure Description

[0018] Figure 1 This is a flowchart of the target recognition and inverse reprojection method in low-light environment of the present invention; Figure 2 This is a data flow architecture diagram for multi-dimensional environmental perception and target recognition in this invention. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0020] It should be noted that all directional and positional terms used in this invention, such as: up, down, left, right, front, back, vertical, horizontal, inner, outer, top, bottom, transverse, longitudinal, center, etc., are only used to explain the relative positional relationship and connection between components in a specific state (as shown in the accompanying drawings). They are only for the convenience of describing this invention and do not require that this invention be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention. In addition, the descriptions of "first," "second," etc., in this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated.

[0021] In the description of this invention, unless otherwise explicitly specified and limited, the terms installation, connection, and linking should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections; they can refer to direct connections or indirect connections through an intermediate medium; they can refer to the internal connection of two components. For those skilled in the art, the specific meaning of the above terms in this invention can be understood according to the specific circumstances.

[0022] In the description of this specification, references to the terms "an embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example, and the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0023] An audiovisual data processing method for target recognition in low-light environments includes the following steps: Step S1: Obtain the audio-visual data stream to be processed and collect multi-dimensional environmental perception parameters, including ambient light index, global motion state parameters and real-time position feedback reading of voice coil motor. Step S2: Determine the current working condition type through the ambient light illuminance index, and trigger the dark light processing mode when the ambient light illuminance index is lower than the preset brightness threshold. Adjust the baseline exposure time according to the global motion state parameters, and use the depth feature recognition model to lock the motion trajectory vector of the target object in the audiovisual data stream to be processed and the original extracted features of the target object. Step S3: Calculate the local shutter parameters for the target object based on the motion trajectory vector, and use the local shutter parameters to cover the reference exposure time; Step S4: Collect the real-time position feedback reading of the voice coil motor at the moment of partial shutter closure, and calculate the physical residual distance of the real-time position feedback reading relative to the preset displacement midpoint. Step S5: Convert the physical residual distance into a feature offset vector in the image coordinate system according to the preset optical conversion ratio, and construct the spatial affine inverse transformation matrix for the target object using the feature offset vector as the input parameter. Step S6: Based on the spatial affine inverse transformation matrix, the original extracted features, and the preset background low-frequency noise tolerance parameter, perform inverse reprojection operation on the edge feature vector of the target object to eliminate feature topological distortion caused by the mechanical response lag of the voice coil motor and generate target recognition data stream.

[0024] Preferably, the specific steps for triggering the low-light processing mode in step S2 include: step S21, under the condition that the ambient light measurement data continues to decrease, reducing the sensitivity reference value according to the collaborative compensation strategy; step S22, while the sensitivity reference value decreases, extending the reference exposure time using global motion state parameters to acquire the first-level noise reduction image matrix stripped of global noise.

[0025] Preferably, the specific steps for performing the reverse reprojection operation in step S6 include: step S61, extracting the position feedback data of the Hall sensor inside the voice coil motor; step S62, converting the position feedback data into a mechanical hysteresis parameter characterizing the physical response delay of the underlying optical motor; and step S63, performing frame-by-frame edge reprojection on the continuous sequence frames in the audiovisual data stream to be processed based on the mechanical hysteresis parameter.

[0026] Preferably, the method for determining the current working condition type in step S2 is as follows: by judging whether the ambient light illuminance index is less than a preset brightness threshold and whether the global motion state parameter is greater than a preset frequency threshold, the current working condition type is determined to be a low-light high-frequency micro-vibration working condition.

[0027] Preferably, step S6 specifically includes: calculating the coordinate correction value of the feature tensor of the target object based on the electronic algorithm correction weight allocated by the collaborative compensation strategy, and using the coordinate correction value to eliminate the spatial position drift between consecutive sequence frames in the audiovisual data stream to be processed.

[0028] Preferably, step S3 specifically includes: reconstructing the exposure integration time within the area where the target object is located, and performing local gain compensation on the target object based on the feature offset vector during the integration period when the voice coil motor is in an unsteady state.

[0029] Preferably, the method further includes the following steps: Step S7, establishing a characteristic description of the motion feedback parameters of the voice coil motor, injecting the characteristic description as a deterministic compensation variable into the image processing link, and implementing data dimension hedging against high-frequency composite vibration interference.

[0030] Preferably, the multidimensional environmental perception parameters are captured through a data perception layer, which includes a light sensor, a motion vector sensing unit, and a real-time position monitoring module integrated inside the voice coil motor.

[0031] Preferably, the method further includes the following steps: Step S8, inputting the target recognition data stream into the subsequent artificial intelligence recognition model, and using the artificial intelligence recognition model to determine the category of the target object.

[0032] Example 1: In the operation of an in-vehicle autonomous driving assistance system, when the ambient light intensity is below 10 Lux and the road vibration frequency is between 40Hz and 60Hz, the system identifies a moving object in the dark. Since the ambient light intensity is below a preset brightness threshold, the system triggers a low-light processing mode and extends the baseline exposure time. Simultaneously, it uses a depth feature recognition model to lock the object's motion trajectory vector. When the object's motion trajectory vector exceeds a preset static threshold, the system calculates local shutter parameters based on the motion trajectory vector and covers the baseline exposure time. At this time, the voice coil motor performs lens displacement compensation. Due to the 3ms to 5ms mechanical response delay of the voice coil motor, it is in a non-steady-state process at the moment the local shutter closes. The system uses a real-time position monitoring module integrated within the voice coil motor to collect the real-time position feedback reading from the Hall sensor at the moment the local shutter closes. It calculates the physical residual distance of the real-time position feedback reading relative to the preset displacement midpoint and converts the physical residual distance into a feature offset vector in the image coordinate system according to a preset optical conversion ratio, thereby constructing a spatial affine inverse transformation matrix. While executing the aforementioned instruction to extend the reference exposure time, the underlying image processor simultaneously invokes a collaborative compensation strategy. This strategy extracts the incremental ratio parameter of the extended exposure time compared to the original reference time, and uses the reciprocal of this ratio coefficient as an attenuation factor to automatically perform a down-multiplication operation on the system's initially set sensitivity electronic amplification reference value. This achieves quantitative cutoff of background shot noise induced by excessive sensitivity enhancement through source control, while maintaining the physical premise of maintaining the overall photon collection balance of the sensor.

[0033] The system extracts the original features of the target. In conjunction with preset background low-frequency noise tolerance parameters Perform the inverse reprojection operation to obtain the aligned feature vectors. The calculation process follows the formula ,in, The aligned feature vectors, Let be the inverse affine transformation matrix in space. For the original extracted features, Using the low-frequency noise tolerance parameter as a background, the system eliminates the feature topological distortion caused by mechanical response lag in the feature tensor space by converting the physical execution delay of the voice coil motor into a quantized feature offset vector in the data dimension. This aligns the edge feature vector of the target with the physical reality, enabling feature extraction of the moving target.

[0034] Example 2: Verifying the recognition accuracy and feature geometric fidelity of the method of the present invention in a low-light, high-frequency micro-vibration superimposed environment. The experiment was conducted on a hardware-in-the-loop simulation platform based on a six-degree-of-freedom vibration table and a high-precision illuminance meter. A luminance sensor with a resolution of 0.1 Lux was used to monitor the ambient light illuminance index in real time, and a voice coil motor Hall displacement monitoring unit with a sampling frequency of 1000 Hz was used to collect physical feedback data. The audiovisual data stream to be processed in the experiment originated from a sequence of frames captured by a 1 / 1.8-inch photosensitive component in a simulated nighttime road scene. To simulate real industrial conditions, Gaussian white noise with a signal-to-noise ratio of 20 dB was actively superimposed in the video acquisition link to characterize the effects of sensor shot noise and circuit thermal noise. The sampling period was set as a key parameter. Its value is limited by the highest frequency component in the global motion state parameters output by the vibration table and the closed-loop control bandwidth of the voice coil motor. To balance the real-time performance of motion vector extraction and the computational load of the processor, the sampling period is... The value is set to 1ms, which enables the system to reconstruct waveforms at 20 sampling points per cycle under 50Hz vibration conditions.

[0035] The experiment was divided into a control group using existing global electronic image stabilization and an experimental group using the method of this invention. Under initial conditions of an ambient light level of 8 Lux and a vibration frequency of 50 Hz, the control group extended the exposure time to 33 ms. Due to the 4.2 ms mechanical response delay of the voice coil motor within the 50 Hz vibration cycle, the object edge experienced nonlinear drift within the integration period, leading to changes in the original extracted features output by the artificial intelligence recognition model. An average topological deviation of 3.5 pixels was generated, reducing the recognition hit rate to 65.4%. The experimental group used the method of this invention to obtain global motion state parameters through the data perception layer and calculate the local shutter parameters as 8ms. At this time, the control center collected the real-time position feedback reading of the voice coil motor at the moment of local shutter closure, and measured the physical residual distance as 0.085mm. According to the preset optical conversion ratio, the physical residual distance was converted into a feature offset vector in the image coordinate system, and the spatial affine inverse transformation matrix was constructed based on this. According to the formula Perform inverse reprojection on the pixel coordinates within the target region to obtain the aligned feature vector. ,in, The aligned feature vectors, Let be the inverse affine transformation matrix in space. For the original extracted features, The background low-frequency noise tolerance parameter is set to 0.05. The aligned feature vector after actual processing is measured. The geometric alignment error with the physical reality was reduced from 3.5 pixels to 0.4 pixels, the classification confidence of the AI ​​recognition model was improved from 0.62 to 0.94, and the average accuracy of the generated target recognition data stream was 96.2%.

[0036] To verify the performance of the method under extreme boundary conditions, the vibration frequency was further increased to 70Hz and the ambient light intensity was reduced to 3Lux in a deep dark light condition. It was observed that as the vibration frequency increased, the physical displacement compensation action of the voice coil motor entered the nonlinear saturation region. In the control group, the target contour was blurred due to mechanical return failure, resulting in a recognition rate decrease to 18.5%. In the experimental group, real-time position feedback readings of the voice coil motor were collected to offset the mechanical response lag. Although the physical residual distance increased to 0.142mm under the 70Hz condition, the recognition accuracy remained at 88.7% after performing inverse reprojection operations on the feature vector using the spatial affine inverse transformation matrix. Furthermore, this experiment… The impact of the severity gradient of the problem on the recognition rate was examined. When the vibration frequency increased at 40Hz, 50Hz, and 60Hz, the recognition accuracy of the control group was 72.1%, 65.4%, and 48.8%, respectively, showing a rapid downward trend. In contrast, the recognition accuracy of the experimental group at the corresponding gradients was 97.4%, 96.2%, and 93.5%, respectively, showing a stable linear correlation. The data results demonstrate that the method of this invention utilizes the characteristic description of the motion feedback parameters of the voice coil motor and eliminates the feature topological distortion caused by mechanical response lag by implementing data dimension offsetting for high-frequency composite vibration interference, thereby achieving a rigorous geometric mapping alignment between the target features and the physical reality.

[0037] Example 3: In the deployment of the intelligent security monitoring terminal, the system initializes and aligns the parameters of the built-in optical sensing module. Due to manufacturing tolerances in the flux linkage constant and spring stiffness of the voice coil motor, fluctuations occur in the mechanical response delay. The system then enters the background noise reference extraction stage. Under conditions of lens obscuring and no light signal input, the system uses the photosensitive component to acquire 100 frames of original grayscale image tensors. The processor calculates the amplitude distribution of each pixel in the spatial domain and extracts the mean parameter of the background noise distribution. With standard deviation parameter And according to the formula Determine the background low-frequency noise tolerance parameter This calculation process ensures that the parameter values ​​cover 99.7% of the system's random thermal noise.

[0038] The deep feature recognition model employs a feature extraction network consisting of five residual convolutional layers. The audiovisual data stream to be processed is converted into an input tensor with dimensions of 1×3×640×640. After sliding convolutional kernel operations, the original extracted features containing object edge information are extracted. When the model detects the motion trajectory vector of the target object When the preset static threshold is exceeded, the system calculates the local shutter parameters. Obtain the baseline exposure time corresponding to the ambient light intensity index. Calculate the motion trajectory vector With the system's critical displacement velocity The ratio of the critical displacement velocity To determine the minimum speed constant that results in a 1-pixel motion blur, the local shutter parameters are finally determined based on proportional relationships. ,Right now To suppress dynamic blur of the target object and enhance the perception capability across the temporal domain based on static extraction networks, the processor synchronously allocates a circular sequence buffer queue in system memory containing five consecutive frames. The centroid coordinates of the target edge captured by the residual convolutional layer in the current frame, along with their corresponding centroid parameters in historical frames, are input to a temporal matching layer with differential comparison logic. By calculating the translation difference of the centroid physical coordinates between two frames and superimposing the inverse of the sampling period on the base, the processor accurately locks down and calculates the motion trajectory vector with dual attributes of velocity scalar and phase direction. The system triggers an exposure time baseline verification procedure. Based on the photoelectric effect, the accumulated charge of the photosensitive component is positively linearly correlated with the exposure integration time. This prevents the exposure cycle from being excessively shortened, causing the feature information to fall below the hardware detection limit. The system extracts the photosensitive component's baseline photoelectric conversion efficiency constant and the current data sensing layer's ambient light index. Based on these two physical quantities, the system calculates the shortest integration time constant required to maintain the pixel signal-to-noise ratio of the target area at a preset characterization threshold. The local shutter parameters calculated based on the aforementioned proportional relationship are then used to determine the optimal shutter speed. When the value is less than the shortest integration time constant, the shortest integration time constant is extracted and the local shutter parameters are directly overwritten. Establish safe operating boundaries for physical light intake.

[0039] The control center collects real-time position feedback readings of the voice coil motor at the moment of partial shutter closure via a Hall sensor, and calls upon the optical conversion ratio determined through online calibration. This method involves driving a voice coil motor to generate a known physical displacement and monitoring the pixel offset generated by a reference point, then determining the ratio of the pixel offset to the physical displacement as the optical conversion ratio. The system calculates the physical residual distance and optical conversion ratio between the real-time position feedback reading and the preset displacement midpoint. The product of these components generates a feature offset vector, which is then injected into the spatial affine inverse transformation matrix. The system enforces a bounded procedure based on a homogeneous coordinate system matrix. In this physical substitution logic, since the working cycle of the underlying photosensitive component has been strictly limited to an extremely short integration time window of a few milliseconds by the preceding procedure, the displacement trajectory of the voice coil motor during this extremely short unsteady damping recovery process is physically and mechanically approximated as a first-order uniform linear drift. Therefore, the system only needs to extract the residual reading of the single transient endpoint of shutter closure, and use linear equivalent calculation to represent the average spatial stretch bias of the target object during the entire millisecond-level exposure integration period. This resolves the dimensional contradiction between the single-point discrete sampling parameters and the continuous-time trailing convolution. Based on the fundamental principles of two-dimensional translational geometry, the pure displacement image in the planar space is the translation vector component of a third-order transformation matrix. The spatial affine inverse transformation matrix is ​​then used to... The architecture is initialized as a 3x3 identity matrix. The scaling factor constant on the main diagonal is locked to 1, while the remaining rotation and shearing factor constants are locked to 0. The horizontal and vertical coordinate displacement values ​​of the extracted feature offset vectors are assigned to the corresponding translation parameter positions of the identity matrix, thus forcibly decoupling the unmeasurable and complex 3D perturbations. This ensures that the model input only receives explicit 2D physical displacements. The core computing unit is called to perform coordinate mapping. Customized tensor operation expressions are used to perform the plus operator as a feature-level noise thresholding action. The original extracted features are compared one by one. Mid-pixel amplitude and background low-frequency noise tolerance parameters Remove pixels with amplitudes below the tolerance parameter; perform the division operator as an inverse spatial mapping operation, using the already constructed third-order identity matrix. Inverse bilinear interpolation coordinates are calculated for the remaining feature tensors after noise removal, using the formula... Perform feature-level reprojection on the pixel coordinates within the target region to eliminate feature topological distortion caused by mechanical response hysteresis, resulting in a higher output feature vector. To align with the physical reality and ensure the mathematical rigor and physical executability of the above formula in the feature tensor space, the division operator in the aforementioned formula is defined as a coordinate mapping operator. It does not perform scalar division, but rather refers to a preset inverse coordinate mapping query transformation. In actual operation, the system uses the aligned target coordinate system of the desired output as the reference grid. Based on the translation parameter vector in the inverse affine transformation matrix, it inversely infers the floating-point real coordinates of each standard grid point in the original distorted feature matrix. Then, it performs bilinear weighted interpolation on the amplitudes of the four source pixels adjacent to the real coordinates and reassigns the obtained interpolation amplitudes to the reference grid, thereby completing the dimensionality reduction and reconstruction mapping of the feature data from the distorted physical state to the geometrically stable state.

[0040] Example 4: In multi-batch hardware deployment scenarios, the processor extracts the reference parameters of the photosensitive module, acquires multiple sets of raw image tensors in a controlled darkroom environment with the light inlet blocked, calculates the average level fluctuation of pixel sites in the time domain sequence, and determines the background low-frequency noise tolerance parameter characterizing the hardware noise floor level. The external stepper drive mechanism moves the photosensitive module, and the processor synchronously monitors the pixel migration of the reference point at the center of the image, establishes a linear correlation between physical spatial displacement and changes in image tensor coordinates, and determines the critical displacement velocity corresponding to the physical displacement threshold per unit pixel offset. .

[0041] During the inspection of the zoom optical system, the processor calibrates the feedback characteristics of the voice coil motor, the real-time position monitoring module records the output reading of the Hall sensor at the end of the lens compensation travel, and calculates the optical conversion ratio between the physical residual distance and the image pixel displacement by combining the lens focal length parameters. The processor bases its decisions on the motion trajectory vector. The step value corrects the local shutter parameters. Relative reference exposure time The response curves and aligned feature vectors generated at different zoom levels. Through spatial affine inverse transformation matrix The feature topology distortion caused by mechanical response delay is offset, and the target feature is aligned with the physical reality.

[0042] Example 5: In an intelligent security terminal installation scenario involving multi-scene adaptation requirements, the processor determines hardware baseline parameters through pre-calibration, acquires 200 frames of initial audiovisual data streams in a controlled darkroom environment with the light inlet blocked, calculates the pixel amplitude distribution of the image background texture region, and extracts the mean of the amplitude distribution. with standard deviation Then according to the formula Determine the background low-frequency noise tolerance parameter The drive mechanism applies a pulse excitation with an amplitude of 0.1 mm to the voice coil motor, and the real-time position monitoring module records the level response of the Hall sensor. The processor determines the mechanical response delay based on the time difference between the pulse start time and the response peak time, and simultaneously monitors the pixel displacement of feature points in the imaging image within a 1 ms time step, and determines this displacement as the critical displacement velocity that satisfies the physical displacement threshold corresponding to the unit pixel offset. To address the parameter drift phenomenon caused by internal reed fatigue and magnetic deterioration in voice coil motors during long-term service, a built-in timeliness guarantee and baseline reconstruction procedure is implemented. During the non-operational idle period of the image signal acquisition module, a preset hourly working cycle is used as the trigger condition to control the drive mechanism to sequentially inject standard frequency sweep excitation electrical signals of three discrete frequencies corresponding to low-frequency, mid-frequency, and high-frequency vibration spectra into the voice coil motor. The real-time position monitoring module synchronously acquires the actual steady-state physical displacement amplitude of the voice coil motor at each frequency point, calculates the attenuation deviation ratio of the actual steady-state physical displacement amplitude relative to the factory preset initial steady-state displacement amplitude, and when the attenuation deviation ratio at any frequency point exceeds the preset mechanical aging tolerance limit, the deviation ratio is extracted as a correction coefficient and directly multiplied into the critical displacement velocity. In the mechanical response delay basic parameter library, the preset mechanical aging tolerance limit is essentially a percentage safety constant embedded in the processor firmware at the factory, representing the limit of hardware fatigue life. In conventional implementations in this field, its value is set to 5% of the nominal steady-state displacement amplitude. In the background comparison logic, the system compares whether the percentage of attenuation deviation obtained from the above deduction is greater than this 5% hard boundary in absolute value. Once this tolerance red line is crossed, the parameter library product self-update mechanism is activated to update the underlying physical mapping reference data to avoid hardware performance degradation leading to distortion of the input source of the front-end calculation model.

[0043] When the system is applied to inspection scenarios with zoom optical systems and faces fluctuations in ambient light levels, the control center initiates an online fault-tolerant and parameter fine-tuning process, and the processor extracts the original features containing edge information of the target object. And use a deep feature recognition model to calculate the motion trajectory vector. ,like When the residual value of the real-time position feedback reading deviates from the preset displacement midpoint by more than 0.2 mm, the system determines that the voice coil motor is in an unsteady state, and the processor calls the spatial affine inverse transformation matrix. Extracting features from the original source Perform feature-level reprojection operations to generate aligned feature vectors. The calculation process follows the formula ,in, The aligned feature vectors, Let be the inverse affine transformation matrix in space. For the original extracted features, As a background low-frequency noise tolerance parameter, by injecting the physical feedback readings of the Hall sensor into the algorithm logic, the system cancels the feature topological distortion caused by mechanical response hysteresis and pushes the target recognition data stream with the target contour aligned with the physical reality to the backend.

[0044] The embodiments of this application have been described above with reference to the accompanying drawings. Unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other. This application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit of this application and the scope of protection of this invention, and all of these forms are within the protection scope of this application.

Claims

1. An audiovisual data processing method for target recognition in low-light environments, characterized in that, Includes the following steps: Step S1: Obtain the audio-visual data stream to be processed and collect multi-dimensional environmental perception parameters, including ambient light index, global motion state parameters and real-time position feedback reading of voice coil motor. Step S2: Determine the current working condition type through the ambient light illuminance index, and trigger the dark light processing mode when the ambient light illuminance index is lower than the preset brightness threshold. Adjust the baseline exposure time according to the global motion state parameters, and use the depth feature recognition model to lock the motion trajectory vector of the target object in the audiovisual data stream to be processed and the original extracted features of the target object. Step S3: Calculate the local shutter parameters for the target object based on the motion trajectory vector, and use the local shutter parameters to cover the reference exposure time; Step S4: Collect the real-time position feedback reading of the voice coil motor at the moment of partial shutter closure, and calculate the physical residual distance of the real-time position feedback reading relative to the preset displacement midpoint. Step S5: Convert the physical residual distance into a feature offset vector in the image coordinate system according to the preset optical conversion ratio, and construct the spatial affine inverse transformation matrix for the target object using the feature offset vector as the input parameter. Step S6: Based on the spatial affine inverse transformation matrix, the original extracted features, and the preset background low-frequency noise tolerance parameter, perform inverse reprojection operation on the edge feature vector of the target object to eliminate feature topological distortion caused by the mechanical response lag of the voice coil motor and generate target recognition data stream.

2. The audiovisual data processing method for target recognition in low-light environments according to claim 1, characterized in that, The specific steps for triggering the low-light processing mode in step S2 include: step S21, under the condition that the ambient light measurement data continues to decrease, reducing the sensitivity reference value according to the collaborative compensation strategy; step S22, while the sensitivity reference value decreases, extending the reference exposure time using global motion state parameters to acquire the first-level noise reduction image matrix stripped of global noise.

3. The audiovisual data processing method for target recognition in low-light environments according to claim 1, characterized in that, The specific steps for the reverse reprojection operation in step S6 include: step S61, extracting the position feedback data of the Hall sensor inside the voice coil motor; step S62, converting the position feedback data into a mechanical hysteresis parameter that characterizes the physical response delay of the underlying optical motor; and step S63, performing frame-by-frame edge reprojection on the continuous sequence frames in the audiovisual data stream to be processed based on the mechanical hysteresis parameter.

4. The audiovisual data processing method for target recognition in low-light environments according to claim 1, characterized in that, The method for determining the current working condition type in step S2 is as follows: by judging whether the ambient light illuminance index is less than the preset brightness threshold and whether the global motion state parameter is greater than the preset frequency threshold, the current working condition type is determined to be a low-light high-frequency micro-vibration working condition.

5. The audiovisual data processing method for target recognition in low-light environments according to claim 1, characterized in that, Step S6 specifically includes: calculating the coordinate correction value of the feature tensor of the target object based on the electronic algorithm correction weight allocated by the collaborative compensation strategy, and using the coordinate correction value to eliminate the spatial position drift between consecutive frames in the audiovisual data stream to be processed.

6. The audiovisual data processing method for target recognition in low-light environments according to claim 1, characterized in that, Step S3 specifically includes: reconstructing the exposure integration time within the area where the target object is located, and performing local gain compensation on the target object based on the feature offset vector during the integration period when the voice coil motor is in an unsteady state.

7. The audiovisual data processing method for target recognition in low-light environments according to claim 1, characterized in that, It also includes the following steps: Step S7: Establish a characteristic description of the motion feedback parameters of the voice coil motor, and inject the characteristic description as a deterministic compensation variable into the image processing link to implement data dimension hedging against high-frequency composite vibration interference.

8. The audiovisual data processing method for target recognition in low-light environments according to claim 1, characterized in that, Multidimensional environmental perception parameters are captured through a data perception layer, which includes a light sensor, a motion vector sensing unit, and a real-time position monitoring module integrated inside the voice coil motor.

9. The audiovisual data processing method for target recognition in low-light environments according to claim 1, characterized in that, It also includes the following steps: Step S8: Input the target recognition data stream into the subsequent artificial intelligence recognition model, and use the artificial intelligence recognition model to determine the category of the target object.

Images