Panoramic monitoring method, system and equipment based on main and distribution models, and medium

Abstract

The invention relates to the technical field of panoramic monitoring, and discloses a panoramic monitoring method, system and device based on a main and distribution model, and a medium, and the method comprises the steps: carrying out the reasonable model selection and layout of the main and distribution model, processing an image through the combination of an SIFT algorithm and a wavelet decomposition algorithm, and solving the problem of poor image splicing quality; the feature points are detected and positioned through the SIFT algorithm, the homography matrix is calculated to complete registration, and the image registration precision is improved; by setting different sampling frequencies, automatically adjusting exposure parameters and applying GPU parallel computing and compression encryption technologies, the risk of data delay and frame loss is reduced, and the real-time performance of data transmission and processing is improved; the sensor redundancy switching unit is used for monitoring in real time, rapidly switching standby sensors and recording logs, sensor faults are effectively handled, monitoring blind areas are prevented from being formed, and stable operation of the system is ensured.

Description

Technical Field

[0001] This invention relates to the field of panoramic monitoring technology, and in particular to a panoramic monitoring method, system, device and medium based on a master-slave model. Background Technology

[0002] In today's rapidly developing digital and intelligent society, panoramic surveillance, with its comprehensive and intuitive characteristics, demonstrates indispensable value and widespread application demand in numerous fields. In the security field, panoramic surveillance is widely deployed in densely populated areas with high security requirements, such as airports, banks, and large shopping malls. It can monitor abnormal behavior and track suspicious individuals in real time, effectively preventing criminal activities and building a strong defense for the safety of people and property. In traffic management, panoramic surveillance systems can monitor overall road traffic conditions in real time, accurately capturing traffic accidents and congestion, assisting traffic management departments in timely guidance and decision-making, and improving road traffic efficiency. In industrial production, panoramic surveillance is applied in production workshops, warehousing, and logistics scenarios, enabling comprehensive monitoring of production line operation status and equipment operation, timely detection of production faults and safety hazards, and ensuring the efficient and safe operation of production activities.

[0003] Traditional panoramic surveillance solutions mainly include two methods: single high-resolution camera surveillance and simple stitching of multiple cameras. While a single high-resolution camera can provide clear local images, its limited field of view makes it difficult to cover large areas, failing to meet the needs of full-area scene monitoring. While simple stitching of multiple cameras expands the monitoring range, it also has certain problems: ① Differences in imaging parameters between different cameras (such as resolution, exposure, and focal length) lead to uneven brightness and color deviation in the stitched images; ② During image registration, traditional methods often use region- or edge-based matching algorithms, which are easily affected by changes in lighting and similar textures in complex scenes, resulting in low registration accuracy and phenomena such as object misalignment and stitching gaps; ③ The transmission and processing of large amounts of raw image data places extremely high demands on hardware computing resources and network bandwidth, making real-time processing difficult for existing systems and posing risks of data delays and frame drops; ④ Traditional surveillance systems lack effective mechanisms to deal with sensor failures, easily creating blind spots if a camera fails. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a panoramic monitoring method, system, device and medium based on a master-slave model to solve the following problems: poor image stitching quality caused by differences in imaging parameters of different cameras in the prior art, low image registration accuracy in complex scenes, insufficient real-time data transmission and processing, and easy occurrence of monitoring blind spots when sensors fail.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides a panoramic monitoring method based on a master-slave model, comprising:

[0008] A main model is set up in the monitoring center, and multiple auxiliary models are distributed in a ring or array around the main model to form a monitoring coverage without blind spots.

[0009] The main model and the auxiliary model respectively acquire images of the main model and the auxiliary model in real time according to the first preset parameters, and monitor the working status of the sensor in real time. In case of failure, the sensor is switched to the backup sensor.

[0010] The acquired main model image and auxiliary model image are preprocessed separately;

[0011] The preprocessed image is registered based on the first matching algorithm, and the registered master model image and the matching model image are fused using the first decomposition algorithm to generate a panoramic image.

[0012] The panoramic image is subjected to target detection, and the target is tracked and its behavior is analyzed in combination with the first association algorithm to generate target trajectory data and alarm information;

[0013] The panoramic image, target trajectory data, and alarm information are stored in a distributed storage system, compressed and encrypted, and then transmitted to the monitoring center.

[0014] As a preferred embodiment of the panoramic monitoring method based on a master-slave model described in this invention, the registration of the preprocessed image based on the first matching algorithm includes:

[0015] Feature points were extracted from the main model image and the auxiliary model image respectively;

[0016] Calculate the Euclidean distance between feature points in the main model image and the matching model image, set a first distance threshold, and retain feature point pairs with a distance less than the first distance threshold as initial matching pairs;

[0017] The initial matching pairs are subjected to bidirectional matching verification to obtain the final matching pairs;

[0018] The homography matrix is ​​calculated based on the final matching pair, and the geometric transformation relationship between the master model image and the matching model image is established.

[0019] Using the homography matrix, a perspective transformation is performed on the matching model image to align the matching model image with the main model image in the spatial coordinate system, thus completing image registration.

[0020] The advantages of this preferred technical solution are: compared with traditional region- or edge-based matching algorithms, it is more robust to changes in lighting and interference from regions with similar textures, effectively avoids phenomena such as object misalignment and splicing gaps, and improves image registration accuracy.

[0021] As a preferred embodiment of the panoramic monitoring method based on a master-slave model described in this invention, the generation of panoramic images includes:

[0022] The registered master model image and the registered model image are decomposed to obtain low-frequency sub-band image and high-frequency sub-band image, respectively.

[0023] The high-frequency subband images of the main model image and the auxiliary model image are fused using the first fusion rule;

[0024] The low-frequency sub-band images of the main model image and the auxiliary model image are fused using the second fusion rule;

[0025] The fused high-frequency sub-band image and low-frequency sub-band image are subjected to inverse wavelet transform to obtain the panoramic image.

[0026] The beneficial effects of this preferred technical solution are: it effectively addresses issues such as uneven brightness and color deviation caused by differences in camera imaging parameters, thereby improving the quality of the stitched images.

[0027] As a preferred embodiment of the panoramic monitoring method based on a master-supplementary model according to the present invention, the master model uses a narrow field-of-view sensor with a first resolution to monitor key areas; the supplementary model uses a wide field-of-view sensor with a second resolution to monitor areas not covered by the master model, and the monitoring ranges of adjacent supplementary models overlap.

[0028] As a preferred embodiment of the panoramic monitoring method based on a master-slave model described in this invention, the master model and the slave model respectively acquire images of the master model and the slave model in real time according to a first preset parameter, and the exposure parameters of the master model and the slave model are automatically adjusted according to the ambient light intensity, with an adjustment range of ISO 100-3200.

[0029] As a preferred embodiment of the panoramic monitoring method based on a master-slave model according to the present invention, the preprocessing of the acquired master model image and slave model image includes:

[0030] The preprocessing of the main model image includes noise reduction and geometric correction.

[0031] The preprocessing of the model image includes brightness equalization and motion detection.

[0032] As a preferred embodiment of the panoramic monitoring method based on the master-slave model described in this invention, the method involves combining a first association algorithm to perform target tracking and behavior analysis, generating target trajectory data and alarm information; the first association algorithm includes a Kalman filter algorithm and a Hungarian algorithm.

[0033] The Kalman filter algorithm is used to predict the position and velocity of the target in the next frame, and the Hungarian algorithm is used to calculate the Mahalanobis distance between targets and perform trajectory correlation.

[0034] Secondly, the present invention provides a panoramic monitoring system based on a master-slave model, comprising:

[0035] The model setting module is used to set up the main model at the monitoring center and distribute multiple auxiliary models in a ring or array around the main model to form a monitoring coverage without blind spots.

[0036] The image acquisition module is used to acquire images of the main model and the auxiliary model in real time according to the first preset parameters, and to monitor the working status of the sensor in real time. In case of failure, it switches to the backup sensor.

[0037] The image processing module is used to preprocess the acquired main model image and the auxiliary model image respectively;

[0038] The registration and fusion module is used to register the preprocessed image based on the first matching algorithm, and to fuse the registered master model image and the matching model image using the first decomposition algorithm to generate a panoramic image.

[0039] The target analysis module is used to detect targets in the panoramic image, and combine it with the first association algorithm to perform target tracking and behavior analysis, generating target trajectory data and alarm information;

[0040] The data transmission module is used to store the panoramic image, target trajectory data and alarm information into a distributed storage system, and then transmit them to the monitoring center after compression and encryption.

[0041] Thirdly, the present invention provides an electronic device, including a memory and a processor; the memory is used to store computer-executable instructions, and the processor, when executing the computer-executable instructions, implements the steps of a panoramic monitoring method based on a master-slave model.

[0042] Fourthly, the present invention provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of a panoramic monitoring method based on a master-slave model.

[0043] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0044] ① This invention uses a high-resolution main model to focus on key areas, a wide-area model with overlapping ranges, and utilizes the SIFT algorithm for feature point matching and wavelet decomposition algorithm to fuse images. This effectively addresses issues such as uneven brightness and color deviation caused by differences in camera imaging parameters, thereby improving the quality of the stitched images.

[0045] ② This invention filters initial matching pairs by calculating Euclidean distance, verifies final matching pairs through bidirectional matching, establishes geometric transformation relationships by calculating homography matrix using least squares method, and completes registration by perspective transformation and matching model images. Compared with traditional matching algorithms based on regions or edges, it is more robust to interference from lighting changes and textured similar regions, effectively avoids phenomena such as object misalignment and splicing gaps, and improves image registration accuracy.

[0046] ③ This invention accelerates processing by employing a GPU parallel computing architecture and compresses and encrypts data before transmission, reducing the requirements for hardware computing resources and network bandwidth, reducing data latency and frame loss risks, and improving the real-time performance of data transmission and processing.

[0047] ④ This invention monitors the working status of the main model and the backup model sensors in real time. When a sensor fails, it automatically switches to the backup sensor to avoid creating a monitoring blind spot and ensure the stable operation of the system. Attached Figure Description

[0048] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a schematic diagram of the overall process logic of a panoramic monitoring method based on a master-slave model according to an embodiment of the present invention. Detailed Implementation

[0050] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0051] Example 1, referring to Figure 1 As one embodiment of the present invention, a panoramic monitoring method based on a master-slave model is provided, such as... Figure 1 The specific steps shown are as follows:

[0052] S100: A main model is set up in the monitoring center, and multiple auxiliary models are distributed in a ring or array around the main model to form a monitoring coverage without blind spots;

[0053] S200: The main model and the auxiliary model respectively acquire images of the main model and the auxiliary model in real time according to the first preset parameters, and monitor the working status of the sensor in real time. In case of failure, it switches to the backup sensor.

[0054] S300: Preprocess the acquired main model image and auxiliary model image respectively;

[0055] S400: Based on the first matching algorithm, the preprocessed image is registered, and the first decomposition algorithm is used to fuse the registered master model image and the matching model image to generate a panoramic image;

[0056] S500: Performs target detection on panoramic images, and combines the first correlation algorithm to perform target tracking and behavior analysis, generating target trajectory data and alarm information;

[0057] S600: Stores panoramic images, target trajectory data, and alarm information in a distributed storage system, and transmits them to the monitoring center after compression and encryption.

[0058] It should be noted that existing technologies suffer from poor image stitching quality, low image registration accuracy in complex scenes, insufficient real-time data transmission and processing, and the potential for blind spots in sensor malfunctions due to differences in imaging parameters between different cameras. Steps S100–S600 address these issues by rationally selecting and arranging the primary and secondary models, and by combining SIFT and wavelet decomposition algorithms to process images. They improve image registration accuracy by using the SIFT algorithm to detect and locate feature points and calculate the homography matrix. Furthermore, by setting different sampling frequencies, automatically adjusting exposure parameters, and employing GPU parallel computing and compression encryption techniques, they reduce data latency and frame loss risks, enhancing the real-time performance of data transmission and processing. Finally, the sensor redundancy switching unit monitors in real-time, quickly switches to backup sensors, and records logs, effectively addressing sensor malfunctions, preventing blind spots, and ensuring stable system operation.

[0059] Example 2, based on the previous example, provides a specific implementation of a panoramic monitoring method based on a master-slave model, to illustrate the technical solution of the present invention.

[0060] S100: A main model is set up in the monitoring center, and multiple auxiliary models are distributed in a ring or array around the main model to form a monitoring coverage without blind spots;

[0061] In this application embodiment, it is necessary to use professional tools such as total station and GPS positioning device to accurately measure the range of the monitoring area, analyze environmental factors such as lighting, obstruction and electromagnetic interference, and identify key monitoring areas such as entrances and exits and important facilities.

[0062] In this embodiment, based on the monitoring accuracy requirements of key areas, the main model uses a narrow field-of-view sensor with a first resolution (resolution ≥ 4K) to monitor the key areas; the auxiliary model uses a wide field-of-view sensor with a second resolution (resolution ≥ 1080P) to monitor the areas not covered by the main model, and the monitoring ranges of adjacent auxiliary models overlap, and a reasonable number is determined to ensure that the overlap rate of the monitoring ranges of adjacent auxiliary models is 10%-20%, while the device parameters are pre-configured.

[0063] Specifically, by selecting a narrow field-of-view sensor with a resolution of ≥4K, such as in areas with extremely high requirements for detail recognition, like bank cash counters or jewelry store display areas, 4K resolution can ensure clear capture of facial features, object details, and other information. Narrow field-of-view sensors typically have a smaller field of view, generally between 20° and 30°, allowing focus on key areas, reducing unnecessary background interference, and improving image quality in critical areas. Conversely, selecting a wide field-of-view sensor with a resolution of ≥1080P, typically with a field of view between 120° and 150°, can cover a larger area; for example, in shopping mall lobbies, etc. In open areas such as parking lots, wide-field-of-view sensors can reduce the number of sensors needed, lowering costs. A 1080P resolution is sufficient for basic identification of people and vehicles. Furthermore, by employing a ring or array distribution, the monitoring ranges of adjacent sensors can overlap to a certain extent, typically between 10% and 20%. This avoids blind spots, and image fusion and other techniques can be used to improve the accuracy and reliability of monitoring within the overlapping areas. For example, in a circular plaza, sensors can be distributed in a ring around the perimeter, with each sensor's monitoring range overlapping with that of its neighbors.

[0064] In this embodiment, a main model is installed at the center of the area, and its position is determined by a laser positioning device. A suitable bracket is used to fix the model and adjust the shooting angle. Supporting models are installed around the main model in a ring or array layout. The equipment is debugged to ensure normal startup, clear images, and coverage without blind spots. Coordinate calibration is performed, equipment parameters are optimized, and the linkage performance of the main and supporting models is tested. Thus, a main model is set at the center of the monitoring area, and multiple supporting models are distributed around it to form a monitoring coverage without blind spots.

[0065] It should be noted that step S100 optimizes the blind spot problem of traditional multi-camera systems. The main model acts as the core node to coordinate the overall situation, while the auxiliary models supplement the details, forming spatial complementarity. When selecting models, the matching of parameters such as focal length and field of view is taken into account, laying a hardware foundation for subsequent high-quality image stitching.

[0066] S200: The main model and the auxiliary model respectively acquire images of the main model and the auxiliary model in real time according to the first preset parameters, and monitor the working status of the sensor in real time. In case of failure, it switches to the backup sensor.

[0067] In this embodiment, the main model uses a high-resolution narrow field-of-view sensor to acquire real-time images of key areas with a sampling frequency of ≥30 frames / second and exposure parameters that are adaptively adjusted within the ISO 100-3200 range; the auxiliary model uses a wide field-of-view sensor to acquire images of areas not covered by the main model with a sampling frequency of ≥15 frames / second and the same range of exposure parameter adjustments.

[0068] In an optional embodiment, the first preset parameter can also be set such that both the main model and the auxiliary model are configured with wide dynamic range (WDR) or HDR mode, with a dynamic adjustment range of ≥120dB, to support local overexposure suppression in backlit scenes.

[0069] In another optional embodiment, the first preset parameter can also be set as follows: the main model sets a dynamic ROI detection region, allocates a higher bitrate to the region, and reduces the bitrate in non-ROI regions; the matching model adopts full-scene balanced encoding.

[0070] It should be noted that the main model is used to monitor key areas. To ensure clear capture of dynamic changes of targets within these key areas, a sampling frequency of ≥30 frames per second is set. For example, in airport security checkpoints, where people and luggage move quickly, a higher sampling frequency ensures that the acquired images are coherent and clear, facilitating accurate identification of potentially dangerous items by security personnel. The secondary model is used to monitor areas not covered by the main model. Taking into account the movement speed of targets within these areas and the overall performance of the system, a sampling frequency of ≥15 frames per second is set. For example, in public areas of large shopping malls, where the flow of people is relatively slow, a sampling frequency of 15 frames per second can reduce the pressure on data processing and storage while ensuring image quality.

[0071] Specifically, the automatic adjustment of the exposure parameters of the main model and the auxiliary model based on the ambient light intensity is implemented as follows:

[0072] Light intensity sensors are integrated into the sensors of the main model and the auxiliary model to detect the ambient light intensity in real time; for example, photoresistors or digital light sensors are used to accurately measure the brightness value of the ambient light and convert it into a digital signal for transmission to the image acquisition device.

[0073] In the software of the image acquisition device, an automatic exposure parameter adjustment algorithm is designed. Based on the detected light intensity value and combined with the preset ISO adjustment range (ISO 100-3200), the exposure parameters of the main model and the auxiliary model are automatically adjusted. For example, when the ambient light intensity is low, the ISO value is appropriately increased to increase the brightness of the image; when the light intensity is high, the ISO value is decreased to avoid overexposure of the image.

[0074] During system operation, changes in light intensity are continuously monitored, and exposure parameters are adjusted in real time according to fluctuations in light intensity. At the same time, the quality of the acquired images is evaluated through image quality assessment algorithms to further optimize the adjustment strategy of exposure parameters. For example, when an image is overexposed or underexposed, the exposure time and ISO value are automatically adjusted to obtain the best image effect.

[0075] In this embodiment, the sensor status monitoring unit in the system monitors the working status of the main and auxiliary model sensors in real time. With the help of the sensor's built-in status feedback signals (such as power, data transmission, and hardware fault indicator information) and data acquisition quality detection (such as image integrity and frame rate stability judgment), when a sensor fault is detected (such as hardware damage, network interruption, or abnormal loss of image data), the sensor redundancy switching unit automatically starts the backup sensor within 500 milliseconds to ensure uninterrupted image acquisition and generates a fault log, which records in detail the fault sensor number, fault occurrence time, fault type, and switching operation information for subsequent maintenance and troubleshooting.

[0076] It should be noted that step S200 above ensures the continuity and stability of data acquisition, solves the problem of monitoring blind spots caused by hardware failures in traditional systems, and at the same time, adaptive parameter adjustment optimizes the imaging quality in complex lighting scenarios.

[0077] S300: Preprocess the acquired main model image and auxiliary model image respectively;

[0078] In this embodiment of the application, the preprocessing of the main model image includes noise reduction processing and geometric correction processing;

[0079] Specifically, the noise reduction process includes:

[0080] Given that the main model image requires high resolution and detail, an adaptive median filtering algorithm can be used. This algorithm can dynamically adjust the size of the filtering window according to the noise density of local areas of the image, effectively removing noise while preserving image details to the greatest extent.

[0081] The initial size of the filtering window is generally set to 3×3. This size allows for fine processing of local areas and avoids excessive smoothing of image details by larger windows. Each pixel in the main model image is traversed to calculate the noise density of its local area. If the noise density is low, the 3×3 window size is maintained for filtering. If the noise density is high, the window is gradually increased to 5×5 or 7×7 to enhance the noise reduction effect.

[0082] Based on a determined window size, median filtering is applied to the pixels, which means taking the median value of all pixels within the window and replacing the current pixel value, thereby achieving noise reduction.

[0083] Specifically, the steps of geometric correction processing include:

[0084] Using professional camera calibration methods such as Zhang Zhengyou's calibration method, we can obtain the camera's intrinsic parameters (such as focal length and principal point coordinates) and extrinsic parameters (such as rotation matrix and translation vector). In actual operation, we can use a calibration board (such as a checkerboard calibration board) to take multiple images from different angles and positions, and calculate the various parameters of the camera through algorithms.

[0085] Common distortion models include radial distortion and tangential distortion; radial distortion will cause straight lines in the image to bend, while tangential distortion will cause local tilting of the image; corresponding distortion models are established based on the parameters obtained from camera calibration.

[0086] For each pixel in the main model image, its correct position in the absence of distortion is calculated based on the distortion model and camera parameters; the value of the pixel is mapped to the correct position through a resampling method (such as bilinear interpolation) to complete the geometric correction.

[0087] In this embodiment of the application, the preprocessing of the model image includes brightness equalization processing and motion detection processing;

[0088] Specifically, brightness equalization processing includes:

[0089] The histogram equalization algorithm is adopted, which adjusts the gray-level histogram of the image to make the gray-level distribution of the image more uniform, thereby improving the overall brightness and contrast of the image.

[0090] The number of pixels at each gray level in the matching model image is counted to obtain the gray-level histogram of the image. The gray levels are usually 0-255, and each gray level corresponds to one pixel.

[0091] Based on the grayscale histogram, calculate the cumulative distribution function for each grayscale level. The cumulative distribution function represents the proportion of pixels less than or equal to that grayscale level out of the total number of pixels.

[0092] Based on the cumulative distribution function, each gray level of the original image is mapped to a new gray level, making the new gray level distribution more uniform; for example, a pixel with a gray level of 100 in the original image becomes 150 after mapping.

[0093] Based on the grayscale mapping relationship, the grayscale value of each pixel in the original image is replaced to generate an image with balanced brightness.

[0094] Specifically, motion detection processing includes:

[0095] Using the background subtraction method, this algorithm establishes a stable background model, compares the current frame image with the background model, and extracts moving targets.

[0096] In the first few frames of the video sequence, the background value of each pixel is calculated through statistical analysis (such as the mean method) to construct an initial background model; for example, the average gray value of the corresponding pixels in the first 10 frames is taken as the background value.

[0097] In subsequent processing, as the environment changes, the background model also needs to be continuously updated. An adaptive background update method can be used to dynamically adjust the background value based on the difference between the current frame and the background model.

[0098] The current frame image is compared with the background model to obtain a difference image. The difference image is then thresholded, and pixels with a difference greater than the threshold are marked as moving target regions. For example, if the threshold is set to 30, when the difference of a pixel in the difference image is greater than 30, it is considered as part of the moving target.

[0099] Morphological processing (such as dilation and erosion) is performed on the extracted moving target region to remove noise and small interference areas, making the outline of the moving target clearer.

[0100] S400: Based on the first matching algorithm, the preprocessed image is registered, and the first decomposition algorithm is used to fuse the registered master model image and the matching model image to generate a panoramic image;

[0101] In this embodiment of the application, the registration of the preprocessed image based on the first matching algorithm includes the following sub-steps A1 to A5:

[0102] In A1: feature points are extracted from the main model image and the auxiliary model image respectively;

[0103] Specifically, the master model image and the matching model image are subjected to Gaussian blurring at different scales to generate multiple sets of images at different scales; for example, the original images are sequentially blurred using a standard deviation of... Convolution is performed using Gaussian kernels to obtain image layers with different degrees of blur. Generally, 3-5 groups are constructed, each containing 4-5 image layers, forming a Gaussian pyramid structure.

[0104] Specifically, in each group of adjacent layers of the Gaussian pyramid, local extreme points are found by comparing the gray values ​​of each pixel with its 8 neighboring pixels and the corresponding 9×2 pixels in the upper and lower layers. These extreme points are candidate feature points. For example, if a pixel has the largest or smallest gray value among the 26 neighboring pixels in the same layer and the upper and lower layers, it is considered a candidate feature point.

[0105] Specifically, candidate feature points are located at the sub-pixel level by fitting a three-dimensional quadratic function to determine the precise location of the feature points and remove unstable feature points with low contrast and located at the edges. Specifically, the stability of feature points is evaluated by calculating the Hessian matrix and points with a Hessian matrix determinant less than a threshold (such as an empirical value of 0.04) are removed to improve the quality of feature points.

[0106] Specifically, centered on the feature point, the gradient direction and magnitude of pixels within a 6σ radius neighborhood (σ being the standard deviation of the scale space where the feature point is located) are calculated. By constructing a direction histogram (generally divided into 36 direction intervals), the direction corresponding to the peak of the histogram is determined as the main direction of the feature point, while the direction with more than 80% of the peak is regarded as the auxiliary direction to enhance the robustness of the feature description. Centered on the feature point, a 16×16 window is taken and divided into 4×4 sub-regions. Gradient histograms of 8 directions are calculated for each sub-region, finally obtaining a 4×4×8=128-dimensional feature descriptor. The position, scale, direction, and descriptor information of the feature point are recorded to complete the feature point extraction of the main model image and the matching model image.

[0107] In A2: Calculate the Euclidean distance between feature points in the master model image and the matching model image, set a first distance threshold, and retain feature point pairs with a distance less than the first distance threshold as initial matching pairs;

[0108] Specifically, the feature point descriptors extracted from the main model image and the auxiliary model image are normalized, and the vector length of each feature point descriptor is normalized to 1 to avoid the distance calculation results being affected by the difference in vector length.

[0109] Specifically, the Euclidean distance formula is used:

[0110]

[0111] Where, x 1i and x 2iLet i be the i-th dimension component of the two feature point descriptors in the main model image and the matching model image respectively. Calculate the Euclidean distance between each feature point in the main model image and all feature points in the matching model image in turn to obtain the distance matrix.

[0112] Specifically, a first distance threshold is set based on experience or experimental results. If the distance between a feature point in the main model image and a feature point in the matching model image is less than the threshold, then these two feature points are paired as an initial matching pair.

[0113] It should be noted that the first distance threshold can generally be set to a small value, such as 0.6-0.8 (this is the threshold for the feature point descriptor distance ratio, that is, the distance ratio between a feature point and the nearest and second nearest feature points in another image).

[0114] In A3: Perform bidirectional matching verification on the initial matching pairs to obtain the final matching pairs;

[0115] Specifically, for matching from the main model image to the matching model image, each initial matching feature point in the main model image is traversed, and the feature point in the matching model image that is closest to it and meets the distance threshold condition is found, and the matching relationship is recorded; for matching from the matching model image to the main model image, each initial matching feature point in the matching model image is traversed, and the feature point in the main model image that is closest to it and meets the threshold condition is found, and the matching relationship is recorded.

[0116] Specifically, by comparing the results of forward and reverse matching, (A,B) is considered the final matching pair only when feature point A in the main model image matches feature point B in the matching model image, and feature point B in the matching model image also matches feature point A in the main model image. This eliminates feature point pairs with inconsistent bidirectional matching and improves matching accuracy.

[0117] In A4: Calculate the homography matrix based on the final matching pair and establish the geometric transformation relationship between the master model image and the matching model image;

[0118] Specifically, construct a system of equations and calculate the homography matrix:

[0119]

[0120] For each final matching pair, let the coordinates of the feature point in the master model image be (x1, y1), and the coordinates of the corresponding feature point in the matching model image be (x2, y2). According to the homography transformation formula:

[0121]

[0122] Two equations can be obtained:

[0123]

[0124] Transforming it into a system of linear equations, for n final matching pairs, we obtain 2n equations; solving the overdetermined system of linear equations using the least squares method, by minimizing the error function:

[0125]

[0126] The values ​​of each element of the homography matrix H are obtained, thereby establishing the geometric transformation relationship between the master model image and the matching model image.

[0127] In A5: Using the homography matrix, perspective transformation is performed on the matching model image to align the matching model image with the main model image in the spatial coordinate system, thus completing image registration;

[0128] Specifically, for each pixel (x, y) in the model image, a perspective transformation is performed based on the homography matrix H, and the new coordinates (x′, y′) are calculated using the following formula:

[0129]

[0130] Right now:

[0131]

[0132] Based on the calculated new coordinates, the matching model image is resampled. Since the transformed coordinates are non-integers, bilinear interpolation or bicubic interpolation is used to calculate the pixel values ​​corresponding to the new coordinates, generating the transformed matching model image. The matching model image after perspective transformation is aligned with the main model image in the spatial coordinate system, completing the image registration process.

[0133] In an optional embodiment, the first matching algorithm may also be a semantic matching algorithm based on deep learning, which uses a convolutional neural network or Transformer model to extract high-level semantic features of the image (such as object edges and texture regions) and achieves pixel-level matching through feature similarity calculation.

[0134] In another alternative embodiment, the first matching algorithm can also be a phase-correlation-based frequency domain matching algorithm, which converts the image to the frequency domain and uses phase correlation to calculate the displacement, rotation, and scaling parameters of the two images to achieve fast global registration.

[0135] In this embodiment of the application, the first decomposition algorithm is used to fuse the registered master model image and the matching model image to generate a panoramic image, including the following sub-steps B1 to B4;

[0136] In B1: The registered master model image and the registered model image are decomposed to obtain low-frequency sub-band image and high-frequency sub-band image respectively; among them, the low-frequency sub-band image reflects the overall contour information of the image, and the high-frequency sub-band image reflects the details and edge information of the image;

[0137] In an optional embodiment, a suitable wavelet basis function is selected based on the image characteristics and processing requirements, such as the commonly used Haar wavelet, Daubechies wavelet (db series), Symlets wavelet (sym series), etc. For example, the Haar wavelet is simple to calculate and has a fast operation speed, making it suitable for scenarios with high processing efficiency requirements; while the Daubechies wavelet has better smoothness and vanishing moment characteristics, and can capture image details more accurately, and can be selected according to the actual situation.

[0138] In this embodiment, the registered master model image and the registered model image are subjected to a first wavelet decomposition using selected wavelet basis functions. The image is then decomposed in the horizontal and vertical directions using a two-dimensional discrete wavelet transform (DWT) to obtain four sub-band images: a low-frequency sub-band (LL1), a horizontal high-frequency sub-band (LH1), a vertical high-frequency sub-band (HL1), and a diagonal high-frequency sub-band (HH1). The low-frequency sub-band LL1 retains the main low-frequency information of the image, i.e., the overall contour. LH1, HL1, and HH1 respectively contain high-frequency detail information in the horizontal, vertical, and diagonal directions of the image.

[0139] In this embodiment, the low-frequency sub-band LL1 obtained from the first-level decomposition is used as input and wavelet decomposition is performed again to obtain four sub-band images (LL2, LH2, HL2, HH2) of the second level; similarly, the low-frequency sub-band LL2 of the second level is decomposed a third time to obtain four sub-band images (LL3, LH3, HL3, HH3) of the third level; after three-level wavelet decomposition, the main model image and the auxiliary model image are each decomposed into 10 sub-band images (1 low-frequency sub-band and 9 high-frequency sub-bands), thus completing the multi-scale decomposition of the image.

[0140] In B2: The high-frequency subband images of the main model image and the auxiliary model image are fused using the first fusion rule;

[0141] In this embodiment, the high-frequency sub-band images obtained by three-level wavelet decomposition of the main model image and the auxiliary model image are matched one-to-one according to the same decomposition level and sub-band type; for example, the first-level horizontal high-frequency sub-band LH1 of the main model image corresponds to the first-level horizontal high-frequency sub-band LH1 of the auxiliary model image, and so on.

[0142] In this embodiment, for each group of corresponding high-frequency sub-band images, the absolute values ​​of pixels at the same position in the two sub-band images are compared pixel by pixel, and the pixel value with the larger absolute value is selected as the pixel value of the fused high-frequency sub-band image at that position. For example, if the pixel value of a certain pixel in the high-frequency sub-band of the main model image is -15, and the pixel value of the corresponding pixel in the matching model image is 10, then the pixel value of that position after fusion is -15 (because |-15|>|10|). In this way, all corresponding high-frequency sub-band images are processed to obtain the fused high-frequency sub-band image, ensuring that the fused image can retain more details and edge information.

[0143] In an optional embodiment, the first fusion rule may also be a weighted fusion rule based on local energy, which dynamically allocates fusion weights by calculating the energy distribution within a local window of the image, thereby achieving a smooth transition while preserving salient features.

[0144] In another alternative embodiment, the first fusion rule may also be a gradient-based fusion rule that prioritizes retaining pixels with clearer edges by comparing the gradient magnitudes of the primary and secondary model images, thereby significantly enhancing the clarity of the target contour.

[0145] In B3: The low-frequency sub-band images of the main model image and the auxiliary model image are fused using the second fusion rule;

[0146] In this embodiment, weighting coefficients are set for the low-frequency sub-bands of the main model image and the supporting model image based on their importance in monitoring. The weighting coefficients for the low-frequency sub-bands of the main model image range from 0.6 to 0.8, and the weighting coefficients for the low-frequency sub-bands of the supporting model image range from 0.2 to 0.4. For example, if more emphasis is placed on the main model's representation of key regions, the weighting coefficients for the low-frequency sub-bands of the main model image can be set to 0.7, and the weighting coefficients for the low-frequency sub-bands of the supporting model image can be set to 0.3.

[0147] In this embodiment, the low-frequency sub-band images (such as LL3) of the main model image and the auxiliary model image are calculated by weighted averaging according to a set weighting coefficient; assuming the pixel value of the low-frequency sub-band of the main model image is I. M (x,y), the low-frequency subband pixel value of the model image is I. P (x,y), the fused low-frequency sub-band pixel value I F The formula for calculating (x,y) is:

[0148] I F (x,y)=0.7×I M (x,y)+0.3×I P (x,y)

[0149] By traversing all pixels of the low-frequency sub-band image and performing a weighted average calculation, the fused low-frequency sub-band image is obtained. This allows the fused image to retain the overall outline while highlighting the information expression of key regions by the main model image.

[0150] In an optional embodiment, the second fusion rule may also be a region-variance-based fusion rule, which dynamically adjusts the fusion weights by calculating the local region variance, maintaining uniformity in smooth regions while retaining more information in detailed regions.

[0151] In another optional embodiment, the second fusion rule can also be a fusion rule based on principal component analysis (PCA), which achieves adaptive fusion by extracting the main component features of the image, and is particularly suitable for monitoring scenarios with uneven lighting or large contrast differences.

[0152] In B4: Inverse wavelet transform is performed on the fused high-frequency sub-band image and low-frequency sub-band image to obtain the panoramic image;

[0153] In this embodiment of the application, the fused low-frequency subband image and high-frequency subband image are organized according to the inverse process of wavelet decomposition; that is, the fused low-frequency subband image LL3 is placed in the innermost layer, and the fused second-layer high-frequency subband images (LH2, HL2, HH2) and the first-layer high-frequency subband images (LH1, HL1, HH1) are placed outward in sequence.

[0154] In this embodiment, the same wavelet basis functions as those used during decomposition are used to perform a three-level two-dimensional discrete wavelet inverse transform (IDWT) on the organized sub-band images. Starting from the innermost low-frequency sub-band image, reconstruction calculations are performed step by step with the outermost high-frequency sub-band images. Through the inverse transform operation, the sub-band images after multi-scale decomposition are restored to the original size images. After the wavelet inverse transform, a panoramic image that integrates the information of the main model image and the auxiliary model image is obtained, thus completing the panoramic image generation process.

[0155] In an optional embodiment, the first decomposition algorithm can also be a multi-scale fusion algorithm based on pyramid decomposition, which constructs a Gaussian pyramid (downsampling layer by layer) and a Laplacian pyramid (preserving high-frequency details) for the registered master model and the matching model images respectively; and fuses them according to weights at different pyramid levels (e.g., high-frequency layers preserve edge information, and low-frequency layers have a smooth transition).

[0156] In another alternative embodiment, the first decomposition algorithm may also be an end-to-end fusion network based on deep learning, which uses a convolutional neural network or a generative adversarial network to directly learn the optimal fusion strategy for multi-source images; the network automatically extracts and fuses salient features.

[0157] S500: Perform target detection on the panoramic image, and combine it with the first association algorithm to perform target tracking and behavior analysis, generating target trajectory data and alarm information, including the following sub-steps C1 and C2;

[0158] In C1: The YOLOv5 deep learning model is used to perform target detection on the generated panoramic image. The confidence threshold is set to ≥0.5 to quickly identify various targets such as people, vehicles, and objects in the image and mark the location and category of the targets.

[0159] It should be noted that YOLO (You Only Look Once) is a highly influential single-stage object detection algorithm in the field of object detection, and YOLOv5 is a version in its series. Compared with traditional object detection algorithms, YOLOv5 is characterized by its high speed and high accuracy. Its working principle is to divide the input image into multiple grids, and predict the object's category and bounding box location simultaneously through a single forward propagation. Its network structure includes a backbone network, a neck network, and a head. The backbone network is used to extract image features, the neck network is responsible for feature fusion, and the head detects and classifies objects based on the fused features.

[0160] Specifically, in object detection, the model outputs a confidence score for each detected target. This score represents the model's confidence in the detection result. Setting the confidence threshold to ≥0.5 means that only when the model considers the detected target to have a confidence level of 50% or higher will it retain it as a valid detection result. This can filter out some unreliable detections and improve the accuracy of the detection results. For example, if the model detects a human target, but the confidence level is only 0.3, which is lower than the threshold of 0.5, then this detection result will be discarded.

[0161] In C2: Target tracking and behavior analysis are performed using the first association algorithm to generate target trajectory data and alarm information; the first association algorithm includes the Kalman filter algorithm and the Hungarian algorithm; the Kalman filter algorithm is used to predict the target's position and velocity in the next frame, and the Hungarian algorithm is used to calculate the Mahalanobis distance between targets and perform trajectory association.

[0162] In this embodiment, Kalman filtering is an optimal recursive data processing algorithm with wide applications in target tracking. Based on the system's state-space model, it estimates the target's state through two steps: prediction and updating. In target tracking, the target's state typically includes information such as position and velocity. During target tracking, the target's current position information is obtained for each frame. The Kalman filtering algorithm predicts the target's position and velocity in the next frame based on the target's historical state (position and velocity) and the system's dynamic model. For example, assuming a person's position in the current frame is (x1, y1) and velocity is (v1, v2), the Kalman filtering algorithm predicts their position (x2, y2) and velocity (v3, v4) in the next frame based on this information and the person's motion patterns (such as uniform motion or uniformly accelerated motion). When new detection results appear, these predicted information are combined to update the algorithm, resulting in a more accurate target state estimate.

[0163] In this embodiment, the Hungarian algorithm is a classic algorithm for solving the maximum weighted matching problem in bipartite graphs. In target tracking, it can help find the optimal matching relationship between the target detected in the current frame and the target tracked in the previous frame. Mahalanobis distance is a distance metric that considers data covariance information, and it can more accurately measure the similarity between two targets. In target tracking, since the state (position, velocity, etc.) of the target is multidimensional and there are correlations between different dimensions, Mahalanobis distance can better reflect the true distance between targets. For example, when two targets are close in both position and velocity, their Mahalanobis distance will be smaller. This indicates that the two targets are the same target. In multi-target tracking scenarios, multiple targets are detected in each frame of the image. These targets need to be associated with targets in previous frames to form continuous target trajectories. The Hungarian algorithm constructs a cost matrix by calculating the Mahalanobis distance between the targets in the current frame and the previous frame, and then solves for the optimal matching of this cost matrix, thereby associating the targets in the current frame with the targets in the previous frame. For example, if three targets are detected in the current frame and three targets are tracked in the previous frame, the Hungarian algorithm will find a matching method that makes the three targets in the current frame correspond one-to-one with the three targets in the previous frame, forming continuous trajectories.

[0164] In an optional embodiment, the first association algorithm can also be the SORT algorithm based on deep learning, which performs fast data association by combining the confidence of object detection with IOU (Intersection over Union), and is suitable for scenarios with high real-time requirements.

[0165] In another optional embodiment, the first association algorithm can also be a multi-target tracking algorithm based on graph convolutional networks, which realizes long-term trajectory association in complex scenarios by modeling the spatiotemporal relationship between targets, and is particularly suitable for monitoring environments with dense crowds or frequent occlusion.

[0166] In this embodiment of the application, during the target tracking process, the system analyzes the target's movement trajectory and behavior pattern based on a preset behavior rule library (such as rules for people loitering for too long, vehicles speeding, driving in the wrong direction, etc.). Once abnormal behavior is detected, an alarm message is immediately generated to remind monitoring personnel to handle it in a timely manner.

[0167] It should be noted that the above step S500 achieves continuous multi-target tracking based on deep learning target detection and the first association algorithm. Combined with spatiotemporal context analysis of abnormal behavior, it significantly reduces the ID switching error rate in complex scenarios, generates continuous trajectory data and triggers accurate alarms, thereby improving the proactive early warning capability of the monitoring system.

[0168] S600: Stores panoramic images, target trajectory data, and alarm information in a distributed storage system, and transmits them to the monitoring center after compression and encryption.

[0169] In this embodiment, the generated panoramic images, target trajectory data, and alarm information are classified and organized. The multi-node storage characteristics of the distributed storage system are used to distribute the data across different storage nodes, ensuring data reliability and redundancy and preventing data loss due to single point of failure.

[0170] In this embodiment, the panoramic image is compressed using the H.265 encoding algorithm, which significantly reduces the amount of image data and improves storage efficiency and transmission speed while ensuring image quality. The target trajectory data and alarm information are compressed using an efficient data compression algorithm (such as the ZIP algorithm). Before data transmission, all data is encrypted using the AES-256 encryption algorithm to convert plaintext data into ciphertext, preventing data from being stolen or tampered with during transmission.

[0171] In this embodiment, the encrypted and compressed data is transmitted to the monitoring center through a secure network channel (such as an SSL / TLS encrypted channel). After receiving the data, the monitoring center performs decryption and decompression operations to restore the panoramic image, target trajectory data and alarm information, so that monitoring personnel can view and analyze them in real time.

[0172] The panoramic surveillance method based on a master-slave model provided in this embodiment achieves wide-area coverage and overlap without blind spots by setting a high-resolution, narrow-field-of-view master model at the center of the surveillance area to focus on key areas, and arranging a wide-field-of-view slave model around the periphery to ensure comprehensive and accurate surveillance. The master and slave models acquire images at different sampling frequencies and adaptive exposure parameters, and combine targeted preprocessing to improve image quality. High-precision image registration and fusion are completed using advanced technologies such as the SIFT algorithm and wavelet decomposition to generate a panoramic image rich in detail. Efficient target detection, accurate tracking, and intelligent behavior analysis are achieved by using the YOLOv5 model, Kalman filtering, and Hungarian algorithm. Finally, distributed storage and compression encryption ensure reliable data transmission. This method effectively solves the problems of insufficient coverage, poor image stitching quality, and low detection and tracking accuracy in traditional panoramic surveillance. It has significant advantages such as wide surveillance range, high image quality, accurate target recognition, fast data processing, and security and reliability, and can be widely used in security, transportation, industry, and other fields, greatly improving the intelligence level and application value of panoramic surveillance.

[0173] Example 3: This example provides a panoramic monitoring system based on a master-slave model, including:

[0174] The model setting module is used to set up the main model at the monitoring center and distribute multiple auxiliary models in a ring or array around the main model to form a monitoring coverage without blind spots.

[0175] The image acquisition module is used to acquire images of the main model and the auxiliary model in real time according to the first preset parameters, and to monitor the working status of the sensor in real time. In case of failure, it switches to the backup sensor.

[0176] The image processing module is used to preprocess the acquired main model image and the auxiliary model image respectively;

[0177] The registration and fusion module is used to register the preprocessed image based on the first matching algorithm, and to fuse the registered master model image and the matching model image using the first decomposition algorithm to generate a panoramic image.

[0178] The target analysis module is used to detect targets in panoramic images and perform target tracking and behavior analysis in conjunction with the first association algorithm to generate target trajectory data and alarm information.

[0179] The data transmission module is used to store panoramic images, target trajectory data, and alarm information into a distributed storage system, and then transmit them to the monitoring center after compression and encryption.

[0180] Specifically, the image acquisition module includes a sensor redundancy switching unit, which is used to monitor the working status of the main model and the backup model sensors in real time. When a sensor failure is detected, it automatically switches to the backup sensor and generates a fault log, recording the fault sensor number, fault time, and switching operation information.

[0181] Specifically, the image registration module uses a GPU parallel computing architecture to accelerate SIFT feature point extraction and homography matrix calculation.

[0182] It should be noted that the panoramic monitoring system based on the master-slave model provided in this embodiment effectively solves the problems of coverage blind spots, low image stitching accuracy, and poor data processing efficiency in traditional panoramic monitoring through the collaborative work of various modules in the system. It has the advantages of comprehensive coverage, stable operation, high processing efficiency, and security and reliability, and can be widely used in fields such as security, traffic management, and industrial production, significantly improving the intelligence and practicality of panoramic monitoring.

[0183] It should be noted that the technical solution of the panoramic monitoring system based on the master-slave model is based on the same concept as the above-mentioned panoramic monitoring method based on the master-slave model. For details not described in detail in the technical solution of the panoramic monitoring system based on the master-slave model in this embodiment, please refer to the description of the above-mentioned panoramic monitoring method based on the master-slave model.

[0184] The above-mentioned unit modules can be embedded in the processor of the electronic device in hardware form or independent of it, or they can be stored in the memory of the electronic device in software form, so that the processor can call and execute the corresponding operations of the above modules.

[0185] This embodiment also provides an electronic device, which includes a processor, a memory, a communication interface, a display screen, and an input device connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a panoramic monitoring method based on a master-slave model. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the device's casing, or an external keyboard, touchpad, or mouse.

[0186] This embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method proposed in the above embodiments.

[0187] The storage medium proposed in this embodiment belongs to the same inventive concept as the method proposed in the above embodiments. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0188] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute the method of the embodiments of the present invention.

[0189] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A panoramic monitoring method based on a master recipe model, characterized by, The application relates to a monitoring system and a monitoring method. A main model is arranged in a monitoring center, and a plurality of auxiliary models are arranged in a ring or array around the main model to form a dead-angle-free monitoring coverage; The main model and the auxiliary models respectively collect main model images and auxiliary model images in real time according to first preset parameters, and monitor the working states of sensors in real time, and switch to standby sensors when faults occur; The collected main model images and auxiliary model images are respectively preprocessed; The preprocessed images are registered based on a first matching algorithm, and the registered main model images and auxiliary model images are fused based on a first decomposition algorithm to generate panoramic images; Target detection is performed on the panoramic images, and target tracking and behavior analysis are performed in combination with a first correlation algorithm to generate target trajectory data and alarm information; The panoramic images, the target trajectory data and the alarm information are stored in a distributed storage system and are transmitted to the monitoring center after being compressed and encrypted.

2. The method of claim 1, wherein the master model is a panoramic model. The registration of the preprocessed images based on the first matching algorithm comprises the following steps: Feature points of the main model images and the auxiliary model images are respectively extracted; Euclidean distances between the feature points in the main model images and the auxiliary model images are calculated, a first distance threshold is set, and feature point pairs with distances less than the first distance threshold are reserved as initial matching pairs; The initial matching pairs are verified through bidirectional matching to obtain final matching pairs; A homography matrix is calculated according to the final matching pairs, and a geometric transformation relationship between the main model images and the auxiliary model images is established; The auxiliary model images are subjected to perspective transformation by using the homography matrix, so that the auxiliary model images are aligned with the main model images in a spatial coordinate system, and image registration is completed.

3. The method of claim 2, wherein the master model is a panoramic model. The generation of the panoramic images comprises the following steps: The registered main model images and auxiliary model images are respectively decomposed to obtain low-frequency subband images and high-frequency subband images; The high-frequency subband images of the main model images and the auxiliary model images are fused by using a first fusion rule; The low-frequency subband images of the main model images and the auxiliary model images are fused by using a second fusion rule; The fused high-frequency subband images and low-frequency subband images are subjected to inverse wavelet transformation to obtain the panoramic images.

4. The method of claim 3, wherein the master model is a panoramic model. The main model adopts a narrow-field sensor with a first resolution to monitor a key region; the auxiliary model adopts a wide-field sensor with a second resolution to monitor a region not covered by the main model, and the monitoring ranges of adjacent auxiliary models overlap.

5. The method of claim 4, wherein the master model is a panoramic model. The main model and the auxiliary models collect main model images and auxiliary model images in real time according to first preset parameters, and the exposure parameters of the main model and the auxiliary models are automatically adjusted according to the intensity of environmental light, and the adjustment range is ISO 100-3200.

6. A method of panoramic surveillance based on a master profile model as claimed in claim 5, wherein, The preprocessing of the collected main model images and auxiliary model images comprises the following steps: The preprocessing of the main model images comprises noise reduction processing and geometric correction processing; The preprocessing of the auxiliary model images comprises brightness equalization processing and motion detection processing.

7. A method of panoramic surveillance based on a master profile model according to claim 6, characterized in that, The first correlation algorithm comprises a Kalman filtering algorithm and a Hungarian algorithm. The Kalman filtering algorithm is used to predict the position and speed of the target in the next frame, and the Hungarian algorithm is used to calculate the Mahalanobis distance between targets and perform track association.

8. A panoramic monitoring system based on a master-slave model, applying a panoramic monitoring method based on a master-slave model as claimed in any one of claims 1 to 7, characterized in that, The method comprises the following steps: The model setting module is used for setting a main model in the monitoring center and arranging a plurality of auxiliary models in a ring or array around the main model to form a dead angle-free monitoring coverage. The image acquisition module is used for acquiring main model images and auxiliary model images by the main model and the auxiliary models respectively according to first preset parameters, and monitoring the working state of the sensors in real time, and switching to backup sensors when a fault occurs. The image processing module is used for pre-processing the acquired main model images and auxiliary model images. The registration and fusion module is used for registering the pre-processed images based on a first matching algorithm, and fusing the registered main model images and auxiliary model images based on a first decomposition algorithm to generate panoramic images. The target analysis module is used for target detection on the panoramic images, and target tracking and behavior analysis are performed in combination with a first correlation algorithm to generate target track data and alarm information. The data transmission module is used for storing the panoramic images, target track data and alarm information in a distributed storage system, and transmitting the compressed and encrypted data to the monitoring center. 9.An electronic device comprising a memory and a processor, the electronic device characterized by: The memory is used for storing computer executable instructions, and the processor executes the computer executable instructions to realize the steps of the panoramic monitoring method based on the main and auxiliary models according to any one of claims 1-7.

10. A computer-readable storage medium having computer-executable instructions stored thereon, characterized in that: The computer executable instructions are executed by the processor to realize the steps of the panoramic monitoring method based on the main and auxiliary models according to any one of claims 1-7.

Images