Intelligent awakening method based on PIR and image fusion and AOV camera

Abstract

The invention relates to the technical field of AOV cameras, and discloses an intelligent awakening method based on PIR and image fusion and an AOV camera. The method comprises the following steps: acquiring a first wake-up signal through a dual-element pyroelectric unit, and synchronously starting a plurality of fisheye image sensors in an AOV camera to acquire a panoramic circular spliced image; and when the AOV camera is in a non-trigger signal state, turning off the plurality of fisheye image sensors to enter a standby state, keeping the lookup table through a DDR self-refreshing mode, and when a new second wake-up signal is received, recovering the power supply of the AOV camera and reading the lookup table to complete rapid wake-up. The problems of false triggering and missing detection of a traditional fixed threshold PIR in a temperature change environment are solved, and an energy-saving and efficient intelligent awakening scheme is provided for battery power supply panoramic monitoring equipment.

Description

Technical Field

[0001] This invention relates to the field of AOV camera technology, and in particular to an intelligent wake-up method based on PIR and image fusion, and an AOV camera. Background Technology

[0002] Traditional AOV cameras use PIR sensors with fixed thresholds as wake-up triggers, which are prone to false triggers or missed detections when the ambient temperature changes. In particular, the increased thermal radiation background noise in high-temperature environments leads to a false trigger rate as high as 15%-22%, while the decreased detection sensitivity in low-temperature environments causes missed detections, affecting the reliability of monitoring.

[0003] Existing fisheye panoramic cameras typically perform global distortion correction and feature extraction on the entire circular image during image stitching. The high computational complexity results in a stitching delay of 200-450ms. Furthermore, each wake-up requires recalculation of the homography transformation matrix, consuming a significant amount of processing time and failing to meet the requirements for rapid response. In addition, traditional SIFT or ORB feature detection algorithms do not consider the differences in imaging quality from the center to the edge of the circular image, leading to unstable matching accuracy.

[0004] Battery-powered AOV cameras still need to maintain high power consumption in standby mode to maintain the system's ability to wake up quickly. The standby power consumption of traditional solutions is usually in the range of 80-120mW, which seriously restricts the battery life. In addition, the loss of calibration data during standby means that the stitching parameters need to be recalculated after waking up, which further prolongs the response delay. Summary of the Invention

[0005] The main objective of this invention is to provide an intelligent wake-up method and AOV camera based on PIR and image fusion. This invention solves the problems of false triggering and missed detection of traditional fixed threshold PIR under temperature change environment, and provides an energy-saving and efficient intelligent wake-up solution for battery-powered panoramic monitoring equipment.

[0006] To achieve the above objectives, this invention provides an intelligent wake-up method based on PIR and image fusion, comprising the following steps: The first wake-up signal is collected by the dual-element pyroelectric unit, and multiple fisheye image sensors in the AOV camera are simultaneously activated to collect panoramic circular stitched images. When the AOV camera is in a no-trigger signal state, the multiple fisheye image sensors are turned off and enter standby mode. The lookup table is maintained through DDR self-refresh mode. When a new second wake-up signal is received, the power supply to the AOV camera is restored and the lookup table is read to complete the fast wake-up.

[0007] Optionally, in a first implementation of the first aspect of the present invention, a first wake-up signal is acquired through a dual-element pyroelectric unit, simultaneously activating multiple fisheye image sensors in the AOV camera to acquire a panoramic circular stitched image, including: Acquire the first voltage signal and the second voltage signal output by the dual pyroelectric unit, and generate a differential voltage signal based on the first voltage signal and the second voltage signal; The amplitude of the differential voltage signal is compared with the dynamic trigger threshold. When the amplitude is greater than or equal to the dynamic trigger threshold, the timer is started to accumulate the duration. If the duration reaches the preset trigger duration and the amplitude is continuously greater than or equal to the dynamic trigger threshold during the period, a rising edge pulse signal is sent to the main control chip as the first wake-up signal. Based on the first wake-up signal, multiple fisheye image sensors in the AOV camera are simultaneously activated to acquire panoramic circular stitched images.

[0008] Optionally, in a second implementation of the first aspect of the present invention, multiple fisheye image sensors in the AOV camera are synchronously activated based on the first wake-up signal to acquire panoramic circular stitched images, including: Based on the first wake-up signal, the power management chip is activated to power on multiple fisheye image sensors in the AOV camera in stages, configure the working parameters of the first sensor, and acquire multiple circular distortion images. Each of the circular distorted images is divided into fan-shaped regions, and bilinear interpolation resampling is performed on each fan-shaped region to obtain multiple first fan-shaped narrowband images; Extract matching point pairs from adjacent first sector narrowband images, and transform and map the multiple first sector narrowband images according to the matching point pairs to obtain multiple second sector narrowband images; The multiple second sector narrowband images are cosine-weighted fused to obtain a panoramic circular stitched image.

[0009] Optionally, in a third implementation of the first aspect of the present invention, each of the circular distorted images is equally divided into fan-shaped regions, and bilinear interpolation resampling is performed on each fan-shaped region to obtain multiple first fan-shaped narrowband images, including: Obtain the geometric center coordinates of each of the circular distorted images, and calculate the radial coordinates and angular coordinates based on the pixel coordinates in each of the circular distorted images and the geometric center coordinates; Divide the preset angle range by the preset number to obtain the sector angle width, calculate the start angle and end angle for each sector area, and extend half of the preset overlap angle on both sides of the start angle and the end angle to obtain the sector angle range. Based on the radial coordinates and the angular coordinates, a polar coordinate grid is constructed within the fan-shaped angle range. The coordinates and gray values ​​of the four surrounding pixels of the grid point position in the polar coordinate grid are obtained and bilinear interpolation is performed to obtain multiple first fan-shaped narrowband images.

[0010] Optionally, in a fourth implementation of the first aspect of the present invention, a set of matching point pairs is extracted from adjacent first sector narrowband images, and a transformation mapping is performed on the plurality of first sector narrowband images according to the set of matching point pairs to obtain a plurality of second sector narrowband images, including: Within the overlapping angle region of adjacent first sector narrowband images, obtain the first radial gradient value of the previous radial position and the second radial gradient value of the next radial position of each pixel in the overlapping angle region. Obtain the grayscale value of the corresponding pixel in the adjacent first sector narrow band image and calculate the absolute value of the grayscale difference. Calculate the radial adaptive gradient threshold based on the reference gradient threshold and the radial coordinate. Select the pixel points whose absolute value of the grayscale difference is less than the radial adaptive gradient threshold as the initial matching points. Calculate the gradient intensity value based on the first and second radial gradient values ​​of the initial matching point, filter the initial matching points whose gradient intensity values ​​are greater than a preset intensity threshold, and generate a set of matching point pairs according to the radial coordinates; The homography transformation matrix is ​​read from a preset lookup table, and the multiple first sector narrowband images are transformed and mapped according to the set of matching point pairs and the homography transformation matrix to obtain multiple second sector narrowband images.

[0011] Optionally, in a fifth implementation of the first aspect of the present invention, a homography transformation matrix is ​​read from a preset lookup table, and the plurality of first sector narrowband images are transformed and mapped according to the set of matching point pairs and the homography transformation matrix to obtain a plurality of second sector narrowband images, including: The corresponding homography transformation matrix is ​​read from the preset lookup table stored in DDR according to the number index of the adjacent first sector narrowband image; Construct homogeneous coordinate vectors for each pixel in the first sector narrowband image, perform matrix multiplication between the homography transformation matrix and the homogeneous coordinate vector to obtain the transformed coordinate vector, and normalize the first two components of the transformed coordinate vector by the third component to obtain the mapped pixel coordinates. The mapped pixel coordinates are rounded down to obtain integer coordinates and the fractional part is calculated. The gray values ​​of the four pixels surrounding the integer coordinates are obtained. The gray values ​​of the four pixels are weighted and summed according to the fractional part to generate multiple second sector narrowband images.

[0012] Optionally, in a sixth implementation of the first aspect of the present invention, cosine-weighted fusion is performed on the plurality of second sector narrowband images to obtain a panoramic circular stitched image, including: Establish a panoramic coordinate system. For each pixel point in the overlapping angle region, calculate the angle coordinate in the panoramic coordinate system and subtract the starting angle of the current fan-shaped narrow band to obtain the first local angle coordinate. Calculate the angle coordinate and subtract the starting angle of the adjacent fan-shaped narrow band to obtain the second local angle coordinate. The angle offset is calculated based on the first local angle coordinate and half the width of the sector angle. The product of pi and the angle offset is divided by the half width of the overlapping angle to obtain the first weighting coefficient. The first weighting coefficient is calculated and subtracted from the first weighting coefficient to obtain the second weighting coefficient. The first average gray value and the second average gray value are obtained by calculating the sum of the gray values ​​of the current narrow fan band and the adjacent narrow fan band in the overlapping angle region and dividing them by the number of pixels in the overlapping angle region. The first average gray value is calculated and divided by the second average gray value to obtain the brightness ratio. The gray value of the adjacent narrow fan band is multiplied by the brightness ratio to obtain the corrected gray value. The first weighting coefficient is multiplied by the gray value of the current narrow sector to obtain the first weighted gray value. The second weighting coefficient is multiplied by the corrected gray value to obtain the second weighted gray value. The first weighted gray value and the second weighted gray value are summed to obtain the fused gray value and filled into the corresponding position of the panoramic circular stitched image.

[0013] Optionally, in the seventh implementation of the first aspect of the present invention, a panoramic coordinate system is established, and the angular coordinates of each pixel point in the overlapping angular region are calculated in the panoramic coordinate system, minus the starting angle of the current fan-shaped narrow band, to obtain a first local angular coordinate. The angular coordinates are then calculated, and the starting angles of adjacent fan-shaped narrow bands are subtracted to obtain a second local angular coordinate. This includes: Obtain the polar coordinate origin of the circular image, and set the radial coordinate range of the panoramic coordinate system from zero to the radius of the circular image, and the angular coordinate range from zero degrees to 360 degrees; The starting angle of the current sector narrow band is obtained by subtracting one from the current sector narrow band number and multiplying it by the sector angle width. The starting angle of the adjacent sector narrow band is obtained by subtracting one from the adjacent sector narrow band number and multiplying it by the sector angle width. For each pixel within the overlapping angle region, obtain its angular coordinates in the panoramic coordinate system. Calculate the angular coordinates and subtract the starting angle of the current fan-shaped narrow band to obtain the first local angular coordinates. Calculate the angular coordinates and subtract the starting angle of the adjacent fan-shaped narrow band to obtain the second local angular coordinates.

[0014] Optionally, in the eighth implementation of the first aspect of the present invention, when the AOV camera is in a no-trigger signal state, the plurality of fisheye image sensors are turned off and enter a standby state, and the lookup table is maintained through DDR self-refresh mode. When a new second wake-up signal is received, the power supply to the AOV camera is restored and the lookup table is read to complete the fast wake-up, including: Monitor the amplitude of the differential voltage signal. When the amplitude is less than the preset ratio of the dynamic trigger threshold for a continuous preset time period, determine that the AOV camera is in a no-trigger signal state. Sequentially turn off the analog and digital power supplies of the multiple fisheye image sensors, stop the video encoder and communication module, reduce the main control chip system clock frequency, and turn off the peripheral clock. The refresh cycle and number of rows to be refreshed are set through DDR self-refresh mode, and the address space of the lookup table is refreshed periodically. When the dual pyroelectric unit detects a new differential voltage signal and generates a second wake-up signal, it sequentially restores the main control chip system clock frequency, configures the DDR memory controller to exit self-refresh mode and restore read / write response, powers on the multiple fisheye image sensors in stages, configures the second sensor operating parameters, and reads the lookup table from the preset starting address of the DDR memory to complete the fast wake-up.

[0015] The present invention also provides an AOV camera, comprising: The acquisition module is used to acquire the first wake-up signal through the dual pyroelectric unit and simultaneously start multiple fisheye image sensors in the AOV camera to acquire panoramic circular stitched images. The fast wake-up module is used to shut down the multiple fisheye image sensors and enter standby mode when the AOV camera is in a state without trigger signal. It maintains the lookup table through DDR self-refresh mode. When a new second wake-up signal is received, it restores the power supply to the AOV camera and reads the lookup table to complete the fast wake-up.

[0016] In summary, this invention solves the problems of false triggering and missed detection in traditional fixed-threshold PIR under temperature-changing environments by combining differential voltage detection of dual-element pyroelectric units with a dynamic threshold adjustment mechanism based on ambient temperature. It adaptively adjusts the trigger sensitivity and sets duration judgment conditions according to real-time ambient temperature, effectively filtering out transient interference signals. By dividing the fisheye circular image into ordered fan-shaped narrowband structures according to polar coordinates and combining a radial position adaptive gradient threshold feature extraction method, it accurately matches the imaging quality variation characteristics of the circular image from the center to the edge, avoiding the high computational complexity of global processing. A fast mapping mechanism using a DDR pre-stored homography transformation matrix lookup table is employed to... The coordinate transformation time has been reduced from 120ms in traditional calculations to 8ms. The DDR self-refresh mode maintains the integrity of the lookup table data during standby, enabling direct reading after wake-up without recalibration. The sector overlap angle region fusion strategy based on the cosine weighted function is specifically designed for the polar coordinate characteristics of circular images, achieving a smooth and continuous grayscale transition at the stitching seam. Through hierarchical power management and DDR self-refresh technology, standby power consumption has been reduced to 38mW. Combined with a millisecond-level fast wake-up and recovery process, the technical problem of real-time stitching of PIR physical trigger and fisheye circular images under ultra-low power conditions has been solved, providing an energy-saving and efficient intelligent wake-up solution for battery-powered panoramic monitoring equipment. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the steps of an intelligent wake-up method based on PIR and image fusion in one embodiment of the present invention; Figure 2 This is a structural block diagram of the AOV camera in an embodiment of the present invention.

[0018] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0020] Reference Figure 1 This embodiment provides a smart wake-up method based on PIR and image fusion, including the following steps: S1, the first wake-up signal is collected through the dual pyroelectric unit, and multiple fisheye image sensors in the AOV camera are simultaneously activated to collect panoramic circular stitched images; S2, when the AOV camera is in a state without trigger signal, multiple fisheye image sensors are turned off and enter standby mode. The lookup table is maintained through DDR self-refresh mode. When a new second wake-up signal is received, the power supply to the AOV camera is restored and the lookup table is read to complete the fast wake-up.

[0021] In one example, a first wake-up signal is acquired via a dual-element pyroelectric unit, simultaneously activating multiple fisheye image sensors in the AOV camera to acquire a panoramic circular stitched image, including: Acquire the first voltage signal and the second voltage signal output by the dual pyroelectric unit, and generate a differential voltage signal based on the first voltage signal and the second voltage signal; The amplitude of the differential voltage signal is compared with the dynamic trigger threshold. When the amplitude is greater than or equal to the dynamic trigger threshold, the timer is started to accumulate the duration. If the duration reaches the preset trigger duration and the amplitude is continuously greater than or equal to the dynamic trigger threshold during the period, a rising edge pulse signal is sent to the main control chip as the first wake-up signal. Based on the first wake-up signal, multiple fisheye image sensors in the AOV camera are simultaneously activated to acquire panoramic circular stitched images.

[0022] In this example, the dual-element pyroelectric detection unit is kept continuously operational. This unit consists of two symmetrically arranged pyroelectric sensing elements, outputting a first analog voltage signal and a second analog voltage signal, both within the weak infrared voltage response range of microvolts to millivolts. These two voltage signals are input to a differential amplifier circuit. The differential amplifier circuit employs an instrumentation amplifier topology to achieve high common-mode rejection ratio and low-noise differential amplification. The amplification gain is set to 45 dB, corresponding to a linear gain coefficient of 178. The output differential voltage signal is expressed as ΔV = 178 × (V1 - V2), where V1 and V2 are the first and second pyroelectric signals, respectively. The differential voltage signal is then passed through a second-order active low-pass filter for high-frequency noise suppression. The filter's cutoff frequency is set to 25 Hz to effectively filter out environmental electromagnetic interference and sudden disturbances. The filtered signal ΔV... 滤波 The high-precision analog-to-digital converter then performs digital acquisition at a sampling rate of 100 Hz, and combines this with the real-time sampling results from the thermistor temperature sensor equipped in the system to calculate the current ambient temperature value T. 环境 To ensure the system maintains stable and reliable trigger sensitivity under different ambient temperatures, the formula V is used. 阈值 = V0 + β × (T 环境 - T0) Dynamically calculate the differential trigger voltage threshold, where V0 is the initial threshold under the temperature reference, β is the temperature compensation coefficient, T0 is the reference standard temperature, and V 阈值The amplitude increases with increasing temperature to compensate for thermal noise errors and decreases with decreasing temperature to improve sensitivity. The absolute amplitude of the current differential voltage signal is compared with a real-time dynamic threshold. When the differential amplitude meets the condition of being greater than or equal to the dynamic threshold, a hardware timer is started to record the duration of the trigger condition. The hardware timer is driven by a 10 kHz clock with a minimum timing accuracy of 0.1 milliseconds. During the timer's accumulation period, the amplitude of the differential voltage is continuously monitored to ensure it always meets the requirement of being greater than or equal to the threshold. When the total duration reaches the preset trigger threshold of 150 milliseconds and the signal does not fall below the threshold condition, it is considered a valid pyroelectric infrared dynamic target event. A set of 3.3V rising-edge digital pulse signals with a pulse width of 5 milliseconds is output to the main control chip via the external GPIO controller as the first wake-up signal to wake up the system and execute the image acquisition process. After receiving the first wake-up signal, the main control chip starts the panoramic perception process of the AOV camera with an interrupt response delay of less than 2 milliseconds. The power management module sequentially restores the AVDD, DVDD, and IOVDD power supplies to multiple fisheye image sensors, and then... 2 The C-bus configures image acquisition parameters and sends synchronous acquisition commands to ensure that multiple sensors start up synchronously and complete the first frame image exposure within milliseconds, thus obtaining the original circular distortion image.

[0023] Before comparing the amplitude of the differential voltage signal with the dynamic trigger threshold, the process includes a step of calculating the dynamic trigger threshold in real time based on the ambient temperature: obtaining the current ambient temperature value output by the ambient temperature sensor, calculating the difference between the current ambient temperature value and the preset standard temperature to obtain the temperature deviation value; multiplying the temperature deviation value by the preset temperature correction coefficient to obtain the temperature compensation voltage, and adding the preset reference trigger voltage to the temperature compensation voltage to obtain the dynamic trigger threshold, wherein the temperature correction coefficient is determined by offline calibration based on the thermal radiation response characteristics of the dual pyroelectric unit at different temperatures; storing the dynamic trigger threshold in a register and recalculating and updating it every preset update cycle, triggering an immediate update when the absolute value of the temperature deviation value is greater than the preset temperature change threshold, ensuring that the dynamic trigger threshold can be adjusted in real time with changes in ambient temperature to reduce the false trigger rate and the missed detection rate.

[0024] In one example, multiple fisheye image sensors in an AOV camera are synchronously activated based on a first wake-up signal to acquire panoramic circular stitched images, including: Based on the first wake-up signal, the power management chip is started to power on multiple fisheye image sensors in the AOV camera in stages, configure the working parameters of the first sensor, and acquire multiple circular distortion images. Each circular distorted image is divided into a sector region, and bilinear interpolation is performed on each sector region to resample it, resulting in multiple first sector narrowband images. Extract matching point pairs from adjacent first sector narrowband images, and transform and map multiple first sector narrowband images based on the matching point pairs to obtain multiple second sector narrowband images; Cosine-weighted fusion of multiple second-sector narrowband images yields a panoramic circular stitched image.

[0025] In this example, upon receiving the first wake-up signal, the main control chip sends a power-on command to the power management chip. The power management chip then controls a tiered power-on process for the multiple fisheye image sensors in the AOV camera, ensuring that the voltage timing and signal stability meet the sensor specifications. Specifically, for each sensor, the analog power supply AVDD is activated sequentially, followed by the digital power supply DVDD after a 1-millisecond delay, and then the I / O power supply IOVDD after another 1-millisecond delay, completing the activation of the full power supply chain. The main control chip activates the I / O power supply via I... 2 The C-bus sends configuration commands to each image sensor, setting operating parameters including exposure time, gain factor, image format, and resolution. After configuration, it uniformly sends a synchronization frame trigger signal, enabling multiple fisheye image sensors to simultaneously initiate image acquisition with millisecond-level errors, outputting a raw circular distortion image with a 360° field of view. The effective region of the acquired image exhibits a typical circular structure, with a center coordinate of (cx, cy) and a radius of R. By constructing a polar coordinate system and dividing each circular image into several overlapping fan-shaped regions along the angular direction at fixed angular intervals Δφ, a polar coordinate system is constructed. For each fan-shaped region, the irregular polar coordinate fan-shaped image is mapped to a regular rectangular first fan-shaped narrowband image through polar coordinate transformation and bilinear interpolation resampling. For adjacent first fan-shaped narrowband image pairs, a set of matching point pairs with positional consistency and grayscale continuity is extracted within the overlapping angular region. The radial gradient is calculated using central difference, and the matching points are filtered using an adaptive threshold function to obtain corresponding pixel pairs with high precision and structural consistency. Based on the matching point pairs, a homography transformation lookup table pre-stored in the DDR self-refresh memory region is invoked to extract the transformation matrix parameters corresponding to each sector pair. These parameters are then used to perform homogeneous coordinate transformation and normalization operations on the first sector narrowband image, mapping the image pixels to a unified transformed polar coordinate domain, generating multiple second sector narrowband images. Since the transformed coordinates are non-integer values, bilinear interpolation is used for resampling to ensure image quality and edge continuity. Between adjacent second sector narrowband images, a cosine weighting function is applied to the overlapping pixels to calculate the fusion weight. Based on the angular offset characteristics of the weighting function, each pixel is weighted and superimposed. Brightness equalization is performed before fusion to achieve a smooth transition and brightness consistency between the multiple second sector narrowband images. After all narrowband images are fused, they are stitched together to form a panoramic circular stitched image.

[0026] In one example, each circularly distorted image is divided into equal fan-shaped regions, and bilinear interpolation resampling is performed on each fan-shaped region to obtain multiple first fan-shaped narrowband images, including: Obtain the geometric center coordinates of each circular distorted image, and calculate the radial and angular coordinates based on the pixel coordinates and geometric center coordinates of each circular distorted image; Divide the preset angle range by the preset number to obtain the sector angle width, calculate the start angle and end angle for each sector area, and extend half of the preset overlap angle on both sides of the start angle and end angle to obtain the sector angle range. A polar coordinate grid is constructed within the sector angle range based on radial and angular coordinates. The coordinates and gray values ​​of the four surrounding pixels of the grid point are obtained and bilinear interpolation is performed to obtain multiple first sector narrowband images.

[0027] In this example, for each frame of the circularly distorted image acquired by the fisheye image sensor, its geometric center coordinates are determined based on the image's fixed geometric features, denoted as (cx, cy), corresponding to the exact center point of the image. For example, for a 2048×2048 resolution image, the geometric center of its effective circular imaging area is (1024, 1024), which serves as the origin reference for polar coordinate transformation. For each effective pixel in the circular image, its two-dimensional Cartesian coordinates (x, y) are extracted, and based on its relative position to the geometric center, the radial coordinate r and angular coordinate φ of that pixel are calculated using geometric formulas, where the radial coordinate r is given by r = √[(x - cx)]. 2 + (y - cy) 2 The angle coordinate φ is obtained by using the arctangent function, which is φ = arctan2(y - cy, x - cx) × 180° / π. After adjustment, the angle range is ensured to be between 0° and 360°, with counterclockwise direction as the positive angle. To achieve structured image segmentation and subsequent sector resampling, the 360° circular image angle range is divided into a preset number N equal parts. The angle width Δφ of each sector is calculated as Δφ = 360° / N. For example, when N is set to 30, Δφ is 12°. The theoretical starting angle φ is calculated for the i-th sector. 起始 = (i - 1) × Δφ and termination angle φ 终止 = i × Δφ, and based on the set angle overlap strategy, at the initial angle φ 起始 Left side and termination angle φ 终止 The right side is extended by half of the preset overlap angle δφ, i.e., 10°, to obtain the actual sector angle range [φ]. 起始 - 10°, φ 终止[+10°]. Within the sector angle range, a two-dimensional polar coordinate grid is constructed using polar coordinates. The grid is defined by the angular sampling interval Δθ and the radial sampling interval Δr, typically set to 160 sampling points in the angular direction and 1024 sampling points in the radial direction, corresponding to a regular rectangular narrowband image structure in the polar coordinate domain. For each sampling point position (r, φ) in the polar coordinate grid, it is back-projected to the (x, y) position in the Cartesian coordinate system. A bilinear interpolation operation is then performed using the gray values ​​of the four nearest integer pixels around that position. That is, the gray values ​​of the four points are weighted and averaged according to the relative position ratio of the grid point among the four integer pixels to obtain the gray value corresponding to the current polar coordinate grid point. After the interpolation operation is completed, all polar coordinate grids within each sector angle range are converted into a regular first sector narrowband image. The image data structure is uniformly 160 columns (angular direction) × 1024 rows (radial direction), and each image is stored separately.

[0028] The coordinates and grayscale values ​​of the four surrounding pixels are obtained from the grid point positions in the polar coordinate grid, and bilinear interpolation is performed to obtain multiple first sector narrowband images. This includes: mapping the positions of each grid point in the polar coordinate grid back to the Cartesian coordinate system of the original circular distorted image; obtaining the horizontal coordinate component by adding the product of the radial coordinate and the cosine of the angle to the center coordinate; obtaining the vertical coordinate component by adding the product of the radial coordinate and the sine of the angle; calculating the integer and fractional parts of the mapped coordinates, where the integer part is obtained by rounding down and the fractional part is obtained by subtracting the integer part; determining the coordinate positions of the four surrounding pixels based on the integer part as the integer coordinate position, the position of the integer coordinate x-coordinate plus one, the position of the integer coordinate y-coordinate plus one, and the integer coordinate y-coordinate ... For positions where both the horizontal and vertical coordinates are incremented by one, the grayscale values ​​of four pixels are read from the original circularly distorted image. The product of the opposite of the horizontal decimal part plus one and the opposite of the vertical decimal part plus one is used as the first interpolation weight for the integer coordinate position. The product of the opposite of the horizontal decimal part plus one and the vertical decimal part plus one is used as the second interpolation weight for the horizontal coordinate incremented by one. The product of the opposite of the horizontal decimal part plus one and the vertical decimal part plus one is used as the third interpolation weight for the vertical coordinate incremented by one. The product of the horizontal decimal part plus one and the vertical decimal part is used as the fourth interpolation weight for the position where both the horizontal and vertical coordinates are incremented by one. The four interpolation weights are multiplied by the corresponding pixel grayscale values ​​and summed to obtain the interpolated grayscale values ​​of the grid points, which are then filled into the first fan-shaped narrow band image.

[0029] In one example, a set of matching point pairs is extracted from adjacent first sector narrowband images, and a transformation mapping is performed on multiple first sector narrowband images based on the set of matching point pairs to obtain multiple second sector narrowband images, including: Within the overlapping angle region of adjacent first sector narrow band images, obtain the first radial gradient value of the previous radial position and the second radial gradient value of the next radial position for each pixel in the overlapping angle region. Obtain the grayscale value of the corresponding pixel in the adjacent first sector narrow band image and calculate the absolute value of the grayscale difference. Calculate the radial adaptive gradient threshold based on the baseline gradient threshold and the radial coordinate. Select the pixel points with the absolute value of the grayscale difference less than the radial adaptive gradient threshold as the initial matching points. The gradient intensity value is calculated based on the first and second radial gradient values ​​of the initial matching point. Initial matching points with gradient intensity values ​​greater than a preset intensity threshold are selected and a set of matching point pairs is generated according to the radial coordinates. The homography transformation matrix is ​​read from the preset lookup table, and multiple first sector narrowband images are transformed and mapped according to the matching point pair set and the homography transformation matrix to obtain multiple second sector narrowband images.

[0030] In this example, for any two adjacent first sector narrowband images i and i+1, the image regions within the overlapping angle range are extracted. The overlapping region is set to a 20° angle band to provide sufficient pixel intersection. Within the overlapping angle region, all pixels are traversed using a polar coordinate grid structure, and the image grayscale values ​​at the previous position r-1 and the next position r+1 of the radial coordinate r of the current pixel are obtained sequentially. These grayscale values ​​are used to calculate the first radial gradient value Grad1(r,φ) = [I] of the first sector image i at that position. i (r+1,φ) - I i (r-1,φ)] / 2, and the second radial gradient value of image i+1, Grad2(r,φ) = [I i+1 (r+1,φ) - I i+1 [(r-1,φ)] / 2, where I i (r,φ) and I i+1 (r, φ) represent the gray values ​​of image i and image i+1 at the polar coordinates (r, φ), respectively. Performing a difference operation on the gray values ​​of image i and image i+1 at the current pixel (r, φ) yields the absolute value of the gray-level difference ΔG(r, φ) = |I i (r,φ) - I i+1 (r,φ)|, and calculate the corresponding radial adaptive gradient threshold G based on the preset central region reference gradient threshold G0 and the radial position r of the pixel. 阈值 (r) = G0× (1 + r / R max )^0.6, where R max The function dynamically adjusts the threshold based on the change in the distance of each pixel from the center to compensate for the degradation in image quality at the edges. This is where ΔG(r,φ) is less than G.阈值 (r) is used as the judgment condition. Pixels that meet the condition are regarded as initial matching points, and the gradient intensity value S(r,φ) = √[Grad1(r,φ)] is calculated for the initial matching points based on their first and second radial gradient values. 2 + Grad2(r,φ) 2 Then, using a gradient intensity value greater than a set minimum intensity threshold as a filtering condition, pixels that meet the double-judgment criteria are retained as valid matching points. All valid matching points are organized into a set of matching point pairs {(r j , φ j )}, and according to the radial coordinate r j Arranged in ascending order. After constructing the set of matching point pairs, the homography transformation matrix H(i) corresponding to image i and image i+1 is read from the data lookup table maintained in the self-refreshing state in DDR. The homography transformation matrix is ​​a 3×3 homogeneous transformation matrix containing 8 degrees of freedom, which can realize coordinate system transformation between polar coordinate images. Each point (r, φ) in the set of matching point pairs is input into the transformation matrix in homogeneous coordinates, and the new coordinates (r′, φ′) after mapping are calculated. Bilinear interpolation is performed on all mapped coordinate points to obtain the resampled gray values, thereby performing spatial transformation on image i and generating the transformed second sector narrowband image. After completing the transformation mapping operation on all first sector narrowband images, a set of spatially aligned second sector narrowband images is constructed.

[0031] The radial adaptive gradient threshold is calculated based on the baseline gradient threshold and the radial coordinates. This includes: obtaining the radial coordinates of a pixel and dividing them by the radius of the circular image to obtain a normalized radial coordinate value, where the normalized radial coordinate value ranges from zero to one; adding one to the normalized radial coordinate value and calculating a preset exponent to obtain a radial weighting factor, where the preset exponent is determined by fitting and optimizing the imaging quality attenuation curve of the circular image from the center to the edge, and is used to compensate for gradient attenuation caused by imaging blurring in the edge region; multiplying the preset baseline gradient threshold by the radial weighting factor to obtain the radial adaptive gradient threshold, so that a lower threshold is used at the center position to preserve detailed features, and a higher threshold is used at the edge position to filter out blur noise, thereby accurately extracting matching points at different radial positions.

[0032] In one example, a homography transformation matrix is ​​read from a preset lookup table, and multiple first sector narrowband images are transformed and mapped according to the set of matching point pairs and the homography transformation matrix to obtain multiple second sector narrowband images, including: The corresponding homography transformation matrix is ​​read from the preset lookup table stored in DDR according to the number index of the adjacent first sector narrowband image; Construct homogeneous coordinate vectors for each pixel in the first sector narrowband image. Perform matrix multiplication on the homography transformation matrix and the homogeneous coordinate vector to obtain the transformed coordinate vector. Then, normalize the first two components of the transformed coordinate vector by dividing them by the third component to obtain the mapped pixel coordinates. The mapped pixel coordinates are rounded down to obtain integer coordinates and the fractional part is calculated. The gray values ​​of the four pixels surrounding the integer coordinates are obtained. The gray values ​​of the four pixels are weighted and summed based on the fractional part to generate multiple second sector narrowband images.

[0033] In this example, based on the index of the adjacent first sector narrowband image pair, the corresponding homography transformation matrix is ​​read from the DDR memory. The lookup operation relies on a pre-established two-dimensional index table (LUT), where the first dimension is the image number, the second dimension is the narrowband pair number, and the table entries contain the 3×3 homography matrix parameters between each pair of sector images. The standard form of the homography transformation matrix H is H = [[h 11 , h 12 , h 13 ], [h 21 , h 22 , h 23 ], [h 31 , h 32 The vector p = [r, φ, 1] is a homogeneous coordinate vector constructed for each pixel in the first narrow-band sector image. It contains eight floating-point transformation parameters and a normalization factor, which describes the mapping relationship between the source image sector coordinates and the target polar coordinate domain. T Let r represent the radial position r and the angular position φ in polar coordinates. Perform matrix multiplication between the homogeneous coordinate vector and the read homography matrix H, calculate the transformed coordinate vector p′ = H × p, and obtain the new polar coordinate vector p′ = [r′, φ′, w′]. T Where r′ and φ′ are the transformed unnormalized coordinate components, and w′ is the normalization factor. To map the transformation result back to the standard image coordinate system, p′ is normalized, and r′ / w′ and φ′ / w′ are used as the precise position coordinates of the final transformed pixel. Since the result is a floating-point value, it is rounded down to obtain the integer part r. int = floor(r′ / w′) and φ int = floor(φ′ / w′), and calculate the decimal part Δr = r′ / w′ - r int With Δφ = φ′ / w′ - φ int This is used to obtain the offset ratio of the current transformed coordinates relative to the four surrounding integer pixels. int , φ intStarting from a given point, obtain the grayscale values ​​of its four neighboring integer pixels, which are the top left and bottom right pixels. 00 Top right I 01 Lower left I 10 With the bottom right I 11 The final pixel value I is obtained by performing a weighted calculation according to the bilinear interpolation principle. interp = (1 - Δr)(1 - Δφ) × I 00 + (1 -Δr)Δφ × I 01 + Δr(1 - Δφ) × I 10 + ΔrΔφ × I 11 The final pixel value is stored as the resampled grayscale value of the current transformed coordinate point in the second sector narrowband image. The entire coordinate transformation and interpolation process is performed on each first sector narrowband image, and the processing results are written to the corresponding second sector narrowband image buffer one by one.

[0034] Before retrieving the corresponding homography transformation matrix from the preset lookup table stored in DDR memory based on the index of the adjacent first sector narrowband image, the process includes the following steps: constructing the preset lookup table during the system's offline calibration phase. This involves acquiring multiple sets of circular calibration images of a standard checkerboard calibration board from several fixedly installed fisheye image sensors, dividing each circular calibration image into sector narrowbands and extracting corner features, and calculating nine parameters of the homography transformation matrix between adjacent sector narrowbands using the least squares method. A two-dimensional index structure is constructed based on the fisheye image sensor number and the sector narrowband number. The nine parameters of the calculated homography transformation matrix are then sequentially stored in the continuous address space of the DDR memory according to the two-dimensional index structure, forming the preset lookup table. The starting address of the continuous address space is preset to a fixed value. The DDR memory controller is configured to mark the address space storing the preset lookup table as a self-refresh protection area, setting the self-refresh period and refresh row number parameters to ensure the integrity of the preset lookup table data is maintained through periodic refresh operations by the DDR internal refresh controller in standby mode, avoiding increased latency caused by recalibration calculations after wake-up.

[0035] The angular offset is calculated based on the first local angular coordinates and half the width of the sector angle. The product of pi and the angular offset is divided by the half-width of the overlapping angle to obtain the first weighting coefficient. This includes: calculating the absolute value of the difference between the first local angular coordinates and the sector angle width divided by two to obtain the angular offset of the pixel from the center line of the sector narrow band, where the sector angle width divided by two corresponds to the center angular position of the sector narrow band; multiplying the preset value of pi by the angular offset to obtain the angular product; and calculating the preset overlapping angle and dividing by two to obtain the half-width of the overlapping angle. The normalized angle parameter is obtained by dividing the angle product by half the overlap angle width. The normalized angle parameter is zero at the center line of the fan-shaped narrow band and equal to pi at the boundary of the overlapping region. The cosine function value is calculated for the normalized angle parameter. The cosine function value is added by one and then divided by two or multiplied by 0.5 to obtain the first weighting coefficient. The first weighting coefficient is one at the center line of the fan-shaped narrow band, zero at the boundary of the overlapping region, and presents a smooth cosine curve transition in the middle position, ensuring that the weighted fusion of adjacent fan-shaped narrow bands in the overlapping region presents a continuous change in gray level rather than an abrupt change.

[0036] In one example, cosine-weighted fusion is performed on multiple second-sector narrowband images to obtain a panoramic circular stitched image, including: Establish a panoramic coordinate system. Calculate the angle coordinates of each pixel within the overlapping angle region in the panoramic coordinate system and subtract the starting angle of the current fan-shaped narrow band to obtain the first local angle coordinates. Calculate the angle coordinates and subtract the starting angle of the adjacent fan-shaped narrow band to obtain the second local angle coordinates. The angle offset is calculated based on the first local angle coordinate and half the width of the sector angle. The product of pi and the angle offset is divided by the half width of the overlapping angle to obtain the first weighting coefficient. The first weighting coefficient is calculated and then subtracted from the first weighting coefficient to obtain the second weighting coefficient. The first average gray value and the second average gray value are obtained by calculating the sum of the gray values ​​of the current sector narrow band and the adjacent sector narrow band in the overlapping angle region and dividing them by the number of pixels in the overlapping angle region. The first average gray value is calculated and divided by the second average gray value to obtain the brightness ratio. The gray value of the adjacent sector narrow band is multiplied by the brightness ratio to obtain the corrected gray value. Multiply the first weighting coefficient by the gray value of the current narrow sector to obtain the first weighted gray value. Multiply the second weighting coefficient by the corrected gray value to obtain the second weighted gray value. Summate the first weighted gray value and the second weighted gray value to obtain the fused gray value and fill it into the corresponding position of the panoramic circular stitched image.

[0037] In this example, a unified panoramic coordinate system is established. The panoramic coordinate system adopts a polar coordinate structure, with angular coordinates ranging from 0° to 360° and radial coordinates ranging from 0 to the image radius R. maxIn the panoramic polar coordinate system, each pixel (r, φ) within the overlapping angle region is processed. Its global angle coordinate φ is subtracted from the starting angle of the current i-th fan-shaped narrowband image to obtain the first local angle coordinate of the pixel in the current narrowband. Simultaneously, the pixel's angle coordinate φ is subtracted from the starting angle of the adjacent (i+1)-th fan-shaped narrowband image to obtain the second local angle coordinate. To calculate the offset of the pixel's angular position relative to the center line within the current narrowband, half the width of the fan-shaped angle is used as a reference. The first local angle coordinate is subtracted from half the angle width Δφ / 2 to obtain the angle offset. This offset is then multiplied by pi (π) and divided by half the width of the overlapping angle δφ / 2 to calculate the first weighting coefficient w1 = 0.5 × [1 + cos(π × Δφ′ / (δφ / 2))]. The second weighting coefficient w2 is obtained by subtracting the first weighting coefficient from 1, i.e., w2 = 1 - w1. Simultaneously, to achieve brightness consistency between two adjacent images during the fusion process, the grayscale values ​​of all pixels in the overlapping angle region of the current and adjacent narrow fan-shaped images are summed and divided by the number of pixels in that region to obtain the first and second average grayscale values. Then, the brightness ratio k = first average grayscale / second average grayscale is calculated. Brightness correction is performed on the pixel values ​​of the adjacent narrow fan-shaped images using the brightness ratio k. The original grayscale value at the corresponding position in image i+1 is multiplied by k to obtain the corrected grayscale value. Weighting coefficients are used to weight the grayscale values ​​of the current pixel in the two fan-shaped images. The grayscale value of the current narrow fan-shaped image is multiplied by the first weighting coefficient to obtain the first weighted grayscale value. The corrected grayscale value is multiplied by the second weighting coefficient to obtain the second weighted grayscale value. These two weighted grayscale values ​​are summed to obtain the fused grayscale value. The fused grayscale value is written to the (r, φ) position of the corresponding panoramic circular stitched image, completing the pixel-level fusion operation. By traversing all pixels within the overlapping area, a local gradient fusion based on a weighted function and brightness balance is achieved, avoiding abrupt grayscale changes and visual discontinuities at image boundaries.

[0038] In one example, a panoramic coordinate system is established. For each pixel within the overlapping angle region, the angular coordinates in the panoramic coordinate system are calculated, and the starting angle of the current narrow fan-shaped band is subtracted to obtain the first local angular coordinates. The angular coordinates are then calculated, and the starting angles of adjacent narrow fan-shaped bands are subtracted to obtain the second local angular coordinates, including: Obtain the polar coordinate origin of the circular image, and set the radial coordinate range of the panoramic coordinate system from zero to the radius of the circular image, and the angular coordinate range from zero degrees to 360 degrees; Get the starting angle of the current sector narrow band by subtracting one from the current sector narrow band number and multiplying it by the sector angle width. Get the starting angle of the adjacent sector narrow band by subtracting one from the adjacent sector narrow band number and multiplying it by the sector angle width. For each pixel within the overlapping angle region, obtain its angular coordinates in the panoramic coordinate system. Calculate the angular coordinates and subtract the starting angle of the current fan-shaped narrow band to obtain the first local angular coordinates. Calculate the angular coordinates and subtract the starting angle of the adjacent fan-shaped narrow band to obtain the second local angular coordinates.

[0039] In this example, a polar coordinate panoramic coordinate system is established. The origin of the coordinate system is taken as the geometric center of each frame of the circular image, labeled (cx, cy). This origin corresponds to the center pixel position of the image. If the image resolution is 2048×2048, then (cx, cy) is (1024, 1024). The radial coordinate range of the panoramic coordinate system is set from zero to the maximum effective radius R of the image. max , where R max The resolution is 1024 pixels, and the angular coordinate range is set from 0 degrees to 360 degrees, with counterclockwise as the positive angle direction to form a completely closed circular angular domain. To obtain the corresponding angular position of each sector image in the panoramic coordinate system, the starting angle of the current sector narrowband image is calculated as (i - 1) × Δφ, which is calculated by subtracting one from the number of the i-th sector narrowband image and multiplying it by the sector angle width Δφ. Correspondingly, the starting angle of its adjacent (i+1)-th sector narrowband image is i × Δφ. When processing overlapping angular regions, all pixels within the overlapping region are traversed, and the angular coordinate φ of each pixel in the panoramic polar coordinate system is obtained. Specifically, the circular image pixel position (x, y) is converted to polar coordinate form (r, φ), where φ = arctan2(y - cy, x - cx) × 180° / π. If φ is negative, 360° is added to normalize it to the interval [0°, 360°]. For each pixel's angle coordinate φ, the relative angle of the angle coordinate φ with respect to the starting angle of the current sector narrow band is calculated in sequence, namely the first local angle coordinate, and the second local angle coordinate with respect to the starting angle of the adjacent sector narrow band.

[0040] In one example, when the AOV camera is in a no-trigger signal state, multiple fisheye image sensors are turned off and enter standby mode. The lookup table is maintained through DDR self-refresh mode. When a new second wake-up signal is received, the AOV camera is powered on and the lookup table is read to complete the fast wake-up, including: Monitor the amplitude of the differential voltage signal. When the amplitude is less than the preset ratio of the dynamic trigger threshold for a continuous preset time, determine that the AOV camera is in a no-trigger signal state. Then, sequentially turn off the analog and digital power supplies of multiple fisheye image sensors, stop the video encoder and communication module, reduce the system clock frequency of the main control chip, and turn off the peripheral clock. The refresh cycle and number of rows to be refreshed are set through DDR self-refresh mode, and the address space of the lookup table is refreshed periodically. When the dual-element pyroelectric unit detects a new differential voltage signal and generates a second wake-up signal, it sequentially restores the main control chip system clock frequency, configures the DDR memory controller to exit self-refresh mode and restore read / write response, powers on multiple fisheye image sensors in stages, configures the second sensor operating parameters, and reads the lookup table from the preset starting address of the DDR memory to complete the rapid wake-up.

[0041] In this example, the differential voltage signal output by the PIR differential amplifier circuit is continuously monitored, and its absolute amplitude is calculated in real time. The differential voltage signal amplitude remains below the current dynamic trigger threshold V for multiple consecutive sampling periods. 阈值 A certain preset ratio (e.g., 0.5 × V) 阈值When the duration reaches a set time threshold, such as 5 seconds or the equivalent of 500 100Hz sampling cycles, it is determined that there is no valid infrared dynamic signal in the current scene, and the AOV camera enters a no-trigger signal state. At this time, the main control chip issues a standby control command, which sequentially controls multiple fisheye image sensors to shut down their power supply channels. First, the I / O power supply IOVDD is shut down, then the digital power supply DVDD is shut down, and finally the analog power supply AVDD is shut down, with a 0.5ms delay between each stage. After the image sensors are powered off, their power consumption drops from 127mW / each during operation to 0mW. The main control chip stops the video encoder and shuts down the DMA data path, while disconnecting the RF transmission part of the WiFi or 4G communication module to release peripheral resources and reduce total power consumption. The main control chip reduces the system main frequency from 1.2GHz to 24MHz in low-power mode through the internal clock control module, and shuts down the clock signals of all unnecessary peripherals, retaining only the GPIO controller and internal timer used for interrupt response. The SoC core enters a shallow sleep or interrupt waiting state, at which point the main control chip power consumption drops to less than 20mW. Meanwhile, to ensure the homography transformation lookup table used for image stitching is safely retained even when power is off, the DDR memory controller is configured in self-refresh mode. This self-refresh mode is automatically executed by the DDR chip's built-in refresh control logic, independent of the main controller's access. Before entering self-refresh mode, the refresh cycle is set to 64 milliseconds, the single row refresh time to 50 nanoseconds, and the total number of refreshed rows to 8192. The address space containing the lookup table (e.g., the 3132 consecutive bytes starting from 0x80000000) is ensured to be periodically accessed during self-refresh to maintain data validity. During standby, the power consumption for maintaining DDR self-refresh alone is approximately 12mW, plus the continuous operation power consumption of the PIR sensor and its preceding differential circuit (approximately 26mW), keeping the total power consumption in standby mode below 56mW. When the PIR sensor detects a dynamic pyroelectric signal in the infrared scene again, the differential voltage amplitude again meets the wake-up condition and the duration exceeds the trigger threshold, generating a second wake-up signal. After receiving the second wake-up signal via GPIO, the main control chip responds via the interrupt controller and exits the low-frequency operating state. The system clock frequency recovers from 24MHz to 1.2GHz within 0.5 milliseconds, simultaneously activating the system bus and peripheral interfaces. The DDR memory controller exits self-refresh mode and returns to normal operating mode, restoring external read / write operations to a latency of 10 nanoseconds. After the system bus and memory regain normal access capabilities, the main control chip controls the power management module to sequentially perform a graded power-on process on multiple fisheye image sensors, in the reverse order of power-off: first, the analog power supply AVDD is powered on, then the digital power supply DVDD, and finally the I / O power supply IOVDD, maintaining a 1-millisecond interval between adjacent power supplies to ensure power-on reliability. After the image sensors are powered on, the main control chip, via I... 2The C interface configures its operating parameters, including exposure time, gain settings, resolution, and output format, and sends a frame synchronization trigger signal. It quickly reads lookup table data (3132 bytes) from the preset starting address space of DDR (e.g., 0x80000000), completing data loading and writing to the SoC's cache area within 100 microseconds using DMA. The lookup table contains homography transformation matrix parameters between multiple sets of sector images. After all hardware units have recovered and the lookup table is ready, the image acquisition, polar coordinate sector division, matching point extraction, homography transformation mapping, and weighted fusion processing flow is initiated, outputting a panoramic circular stitched image.

[0042] The DDR self-refresh mode sets the refresh cycle and refresh row count, periodically refreshing the address space of the lookup table. This includes: sending a self-refresh mode entry command to the DDR memory controller; configuring the refresh cycle register to a preset millisecond time interval and the refresh row count register to the total number of rows in the DDR chip; the DDR memory controller calculating the time for a single refresh operation based on the refresh cycle and refresh row count and starting the internal refresh controller; obtaining the starting address and data length of the lookup table in DDR memory; calculating the address range occupied by the lookup table; and the DDR internal refresh controller performing read, amplify, and write-back operations on the memory cells containing the address range within each refresh cycle to compensate for charge leakage and maintain the integrity of the lookup table data during standby. During self-refresh mode, the DDR memory controller blocks read / write commands from the main control chip and only responds to exit self-refresh mode commands. Upon receiving a second wake-up signal, the main control chip sends an exit self-refresh mode command. The DDR memory controller completes the last refresh operation within a preset microsecond time and resumes normal read / write response. The main control chip reads the lookup table data from the starting address without recalculation.

[0043] Reference Figure 2 This embodiment provides an AOV camera, including: Acquisition module 1 is used to acquire the first wake-up signal through the dual pyroelectric unit and simultaneously start multiple fisheye image sensors in the AOV camera to acquire panoramic circular stitched images. The fast wake-up module 2 is used to shut down multiple fisheye image sensors and enter standby mode when the AOV camera is in a state without trigger signal. It maintains the lookup table through DDR self-refresh mode. When a new second wake-up signal is received, it restores the power supply to the AOV camera and reads the lookup table to complete the fast wake-up.

[0044] In this embodiment, the specific implementation of each unit in the above device embodiment is described in the above method embodiment, and will not be repeated here.

[0045] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0046] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A smart wake-up method based on PIR and image fusion, characterized in that, include: The first wake-up signal is collected by the dual-element pyroelectric unit, and multiple fisheye image sensors in the AOV camera are simultaneously activated to collect panoramic circular stitched images. When the AOV camera is in a no-trigger signal state, the multiple fisheye image sensors are turned off and enter standby mode. The lookup table is maintained through DDR self-refresh mode. When a new second wake-up signal is received, the power supply to the AOV camera is restored and the lookup table is read to complete the fast wake-up.

2. The intelligent wake-up method based on PIR and image fusion according to claim 1, characterized in that, The first wake-up signal is acquired through a dual-element pyroelectric unit, simultaneously activating multiple fisheye image sensors in the AOV camera to acquire panoramic circular stitched images, including: Acquire the first voltage signal and the second voltage signal output by the dual pyroelectric unit, and generate a differential voltage signal based on the first voltage signal and the second voltage signal; The amplitude of the differential voltage signal is compared with the dynamic trigger threshold. When the amplitude is greater than or equal to the dynamic trigger threshold, the timer is started to accumulate the duration. If the duration reaches the preset trigger duration and the amplitude is continuously greater than or equal to the dynamic trigger threshold during the period, a rising edge pulse signal is sent to the main control chip as the first wake-up signal. Based on the first wake-up signal, multiple fisheye image sensors in the AOV camera are simultaneously activated to acquire panoramic circular stitched images.

3. The intelligent wake-up method based on PIR and image fusion according to claim 2, characterized in that, Based on the first wake-up signal, multiple fisheye image sensors in the AOV camera are synchronously activated to acquire panoramic circular stitched images, including: Based on the first wake-up signal, the power management chip is activated to power on multiple fisheye image sensors in the AOV camera in stages, configure the working parameters of the first sensor, and acquire multiple circular distortion images. Each of the circular distorted images is divided into fan-shaped regions, and bilinear interpolation resampling is performed on each fan-shaped region to obtain multiple first fan-shaped narrowband images; Extract matching point pairs from adjacent first sector narrowband images, and transform and map the multiple first sector narrowband images according to the matching point pairs to obtain multiple second sector narrowband images; The multiple second sector narrowband images are cosine-weighted fused to obtain a panoramic circular stitched image.

4. The intelligent wake-up method based on PIR and image fusion according to claim 3, characterized in that, Each of the circularly distorted images is divided into equal fan-shaped regions, and bilinear interpolation resampling is performed on each fan-shaped region to obtain multiple first fan-shaped narrowband images, including: Obtain the geometric center coordinates of each of the circular distorted images, and calculate the radial coordinates and angular coordinates based on the pixel coordinates in each of the circular distorted images and the geometric center coordinates; Divide the preset angle range by the preset number to obtain the sector angle width, calculate the start angle and end angle for each sector area, and extend half of the preset overlap angle on both sides of the start angle and the end angle to obtain the sector angle range. Based on the radial coordinates and the angular coordinates, a polar coordinate grid is constructed within the fan-shaped angle range. The coordinates and gray values ​​of the four surrounding pixels of the grid point position in the polar coordinate grid are obtained and bilinear interpolation is performed to obtain multiple first fan-shaped narrowband images.

5. The intelligent wake-up method based on PIR and image fusion according to claim 4, characterized in that, Extract matching point pairs from adjacent first sector narrowband images, and perform transformation mapping on the multiple first sector narrowband images based on the matching point pairs to obtain multiple second sector narrowband images, including: Within the overlapping angle region of adjacent first sector narrowband images, obtain the first radial gradient value of the previous radial position and the second radial gradient value of the next radial position of each pixel in the overlapping angle region. Obtain the grayscale value of the corresponding pixel in the adjacent first sector narrow band image and calculate the absolute value of the grayscale difference. Calculate the radial adaptive gradient threshold based on the reference gradient threshold and the radial coordinate. Select the pixel points whose absolute value of the grayscale difference is less than the radial adaptive gradient threshold as the initial matching points. Calculate the gradient intensity value based on the first and second radial gradient values ​​of the initial matching point, filter the initial matching points whose gradient intensity values ​​are greater than a preset intensity threshold, and generate a set of matching point pairs according to the radial coordinates; The homography transformation matrix is ​​read from a preset lookup table, and the multiple first sector narrowband images are transformed and mapped according to the set of matching point pairs and the homography transformation matrix to obtain multiple second sector narrowband images.

6. The intelligent wake-up method based on PIR and image fusion according to claim 5, characterized in that, The homography transformation matrix is ​​read from a preset lookup table, and the multiple first sector narrowband images are transformed and mapped according to the set of matching point pairs and the homography transformation matrix to obtain multiple second sector narrowband images, including: The corresponding homography transformation matrix is ​​read from the preset lookup table stored in DDR according to the number index of the adjacent first sector narrowband image; Construct homogeneous coordinate vectors for each pixel in the first sector narrowband image, perform matrix multiplication between the homography transformation matrix and the homogeneous coordinate vector to obtain the transformed coordinate vector, and normalize the first two components of the transformed coordinate vector by the third component to obtain the mapped pixel coordinates. The mapped pixel coordinates are rounded down to obtain integer coordinates and the fractional part is calculated. The gray values ​​of the four pixels surrounding the integer coordinates are obtained. The gray values ​​of the four pixels are weighted and summed according to the fractional part to generate multiple second sector narrowband images.

7. The intelligent wake-up method based on PIR and image fusion according to claim 6, characterized in that, Cosine-weighted fusion of the multiple second-sector narrowband images yields a panoramic circular stitched image, including: Establish a panoramic coordinate system. For each pixel point in the overlapping angle region, calculate the angle coordinate in the panoramic coordinate system and subtract the starting angle of the current fan-shaped narrow band to obtain the first local angle coordinate. Calculate the angle coordinate and subtract the starting angle of the adjacent fan-shaped narrow band to obtain the second local angle coordinate. The angle offset is calculated based on the first local angle coordinate and half the width of the sector angle. The product of pi and the angle offset is divided by the half width of the overlapping angle to obtain the first weighting coefficient. The first weighting coefficient is calculated and subtracted from the first weighting coefficient to obtain the second weighting coefficient. The first average gray value and the second average gray value are obtained by calculating the sum of the gray values ​​of the current narrow fan band and the adjacent narrow fan band in the overlapping angle region and dividing them by the number of pixels in the overlapping angle region. The first average gray value is calculated and divided by the second average gray value to obtain the brightness ratio. The gray value of the adjacent narrow fan band is multiplied by the brightness ratio to obtain the corrected gray value. The first weighting coefficient is multiplied by the gray value of the current narrow sector to obtain the first weighted gray value. The second weighting coefficient is multiplied by the corrected gray value to obtain the second weighted gray value. The first weighted gray value and the second weighted gray value are summed to obtain the fused gray value and filled into the corresponding position of the panoramic circular stitched image.

8. The intelligent wake-up method based on PIR and image fusion according to claim 7, characterized in that, Establish a panoramic coordinate system. For each pixel within the overlapping angle region, calculate its angular coordinates in the panoramic coordinate system and subtract the starting angle of the current fan-shaped narrow band to obtain the first local angular coordinates. Then, calculate the angular coordinates and subtract the starting angle of the adjacent fan-shaped narrow band to obtain the second local angular coordinates, including: Obtain the polar coordinate origin of the circular image, and set the radial coordinate range of the panoramic coordinate system from zero to the radius of the circular image, and the angular coordinate range from zero degrees to 360 degrees; The starting angle of the current sector narrow band is obtained by subtracting one from the current sector narrow band number and multiplying it by the sector angle width. The starting angle of the adjacent sector narrow band is obtained by subtracting one from the adjacent sector narrow band number and multiplying it by the sector angle width. For each pixel within the overlapping angle region, obtain its angular coordinates in the panoramic coordinate system. Calculate the angular coordinates and subtract the starting angle of the current fan-shaped narrow band to obtain the first local angular coordinates. Calculate the angular coordinates and subtract the starting angle of the adjacent fan-shaped narrow band to obtain the second local angular coordinates.

9. The intelligent wake-up method based on PIR and image fusion according to claim 8, characterized in that, When the AOV camera is in a no-trigger signal state, the multiple fisheye image sensors are turned off and enter standby mode. The lookup table is maintained through DDR self-refresh mode. When a new second wake-up signal is received, the power supply to the AOV camera is restored and the lookup table is read to complete the fast wake-up, including: Monitor the amplitude of the differential voltage signal. When the amplitude is less than the preset ratio of the dynamic trigger threshold for a continuous preset time period, determine that the AOV camera is in a no-trigger signal state. Sequentially turn off the analog and digital power supplies of the multiple fisheye image sensors, stop the video encoder and communication module, reduce the main control chip system clock frequency, and turn off the peripheral clock. The refresh cycle and number of rows to be refreshed are set through DDR self-refresh mode, and the address space of the lookup table is refreshed periodically. When the dual pyroelectric unit detects a new differential voltage signal and generates a second wake-up signal, it sequentially restores the main control chip system clock frequency, configures the DDR memory controller to exit self-refresh mode and restore read / write response, powers on the multiple fisheye image sensors in stages, configures the second sensor operating parameters, and reads the lookup table from the preset starting address of the DDR memory to complete the fast wake-up.

10. An AOV camera, characterized in that, The steps for implementing the intelligent wake-up method based on PIR and image fusion as described in any one of claims 1 to 9 include: The acquisition module is used to acquire the first wake-up signal through the dual pyroelectric unit and simultaneously start multiple fisheye image sensors in the AOV camera to acquire panoramic circular stitched images. The fast wake-up module is used to shut down the multiple fisheye image sensors and enter standby mode when the AOV camera is in a state without trigger signal. It maintains the lookup table through DDR self-refresh mode. When a new second wake-up signal is received, it restores the power supply to the AOV camera and reads the lookup table to complete the fast wake-up.

Images