A community management method based on pure visual solution

By using multispectral cameras and prior Bayesian networks in community monitoring systems, combining multi-view stereoscopic vision and structured light technology, the existing system's insufficient monitoring in complex scenes and extreme lighting conditions is solved, and efficient Anomaly monitoring and management improves the reliability and efficiency of the system.

CN119068423BActive Publication Date: 2025-02-07CHENGDU ZHONGGUI RAILEQUIPMENT CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411555147.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-11-04
Publication Date
2025-02-07
Estimated Expiration
2044-11-04

AI Technical Summary

Technical Problem

The existing community monitoring system based on monocular cameras is difficult to achieve high-precision monitoring and abnormal behavior detection in complex community scenarios. Especially under extreme lighting conditions, the system has a high false alarm rate and cannot automatically identify complex abnormal behaviors. , resulting in the need of a lot of human resources for video surveillance.

Method used

Multispectral cameras combined with non-downsampled contour wave transform (NSCT) and weighted fusion technology to generate fusion images; multi-view stereo vision and structured light technology are used for deep fusion to obtain three-dimensional depth information; then through a priori Bayesian network Anomaly detection, generates anomaly detection probability.

Benefits of technology

It realizes efficient abnormal monitoring and management of community scenarios. The system has excellent three-dimensional spatial perception capabilities and can maintain high-quality monitoring effects under complex lighting conditions, improves monitoring reliability and efficiency, and reduces the demand for human resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068423B_ABST
    Figure CN119068423B_ABST
Patent Text Reader

Abstract

The present invention belongs to the field of image processing technology, and specifically relates to a community management method based on a pure visual solution, the method comprising: step 1: setting multiple multispectral cameras in the community to obtain multispectral image information in real time in the community; step 2: based on the fused image, using multi-view stereo vision and structured light technology to perform deep fusion to obtain fusion depth; step 3: based on the fusion depth, using a priori Bayesian network to perform anomaly detection to obtain anomaly detection probability. The present invention realizes more efficient anomaly monitoring and management of community scenes.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a community management method based on a pure visual solution. Background Art

[0002] With the rapid development of modern society, the demand for intelligent and automated community management is increasing, especially in large-scale communities, residential areas, commercial complexes and other scenarios. How to effectively improve the safety, convenience and management efficiency of the community has become a technical problem that needs to be solved urgently. In this context, vision-based community management methods have gradually become a research hotspot and have been applied and promoted to a certain extent. However, the existing visual management system still has some shortcomings in technical implementation, and it is difficult to meet the needs of high-precision monitoring and abnormal behavior detection in complex community scenarios.

[0003] Among the existing community monitoring systems, the most common technical solutions are based on monocular cameras or traditional video monitoring. These systems capture the images in the community in real time through cameras installed at various locations in the community, and monitor and analyze the images through background personnel or simple image processing algorithms. However, this monitoring solution based on monocular cameras has many problems, which are mainly manifested in the following aspects: the existing traditional monitoring system mainly relies on monocular cameras for video acquisition, and the acquired images are two-dimensional image information, lacking the ability to perceive scene depth and three-dimensional space. In a complex three-dimensional scene such as a community, it is difficult to accurately determine the position, distance and movement trajectory of an object in space based on a two-dimensional image alone. This limitation causes the existing system to perform poorly in some key scenarios, especially in monitoring human behavior and judging whether an object has crossed the boundary, and it cannot provide sufficient spatial information. For example, when someone approaches the boundary of the community or climbs over the fence, the two-dimensional image may not be able to clearly determine the distance and height between the person and the boundary, which is prone to misjudgment or omission. The existing community monitoring system based on monocular cameras can usually only work normally under good lighting conditions. When the ambient light is poor, especially at night or in scenes with drastic light changes, the quality of the monitoring image will be significantly reduced, resulting in unclear outlines of objects and people and loss of details. Although some monitoring systems have introduced infrared cameras or night vision technology, these technologies still cannot completely solve the monitoring problem under extreme lighting conditions. Especially in complex lighting environments, such as shadows of buildings in the community, tree occlusions, vehicle lights, etc., traditional monitoring systems cannot distinguish between the interference caused by ambient light and the real movement of objects, resulting in a high false alarm rate. Traditional community monitoring systems often rely on real-time observation of videos by backstage personnel or simple motion detection algorithms. Since the monitoring system cannot automatically identify complex abnormal behaviors, such as illegal intrusions, abnormal movement of objects, or other emergencies, a large amount of human resources is required for video monitoring. At the same time, the existing motion detection algorithms based on image differences usually only focus on whether the object is moving, but cannot deeply analyze the behavior pattern of the object. For some potential abnormal behaviors, such as slowly approaching the community boundary and quickly crossing certain sensitive areas, the recognition ability of the existing system is seriously insufficient. This not only increases the workload of monitoring, but also fails to provide real-time and intelligent security warning functions. Summary of the invention

[0004] In view of this, the main purpose of the present invention is to provide a community management method based on a pure visual solution, which realizes more efficient abnormal monitoring and management of community scenes.

[0005] The technical solution adopted by the present invention is as follows:

[0006] A community management method based on a purely visual solution, the method comprising:

[0007] Step 1: Set up multiple multispectral cameras in the community to obtain multispectral image information in the community in real time; process the obtained multispectral image information through non-subsampled contourlet transform and weighted fusion to obtain a fused image;

[0008] Step 2: Based on the fused image, use multi-view stereo vision and structured light technology to perform depth fusion to obtain the fusion depth;

[0009] Step 3: Based on the fusion depth, use the prior Bayesian network to perform anomaly detection and obtain the anomaly detection probability.

[0010] Furthermore, in step 1, it is assumed that the multispectral camera acquires Multispectral image of bands; The wavelength is The weight of the multispectral image corresponding to the band is ; The fused image is calculated by the following formula :

[0011] ;

[0012] in, Indicates Multispectral image corresponding to bands The coefficients obtained after performing non-subsampled contourlet transform; is the central wavelength, which is equal to the mean wavelength of all bands; is the standard deviation, which is equal to the standard deviation of wavelengths in all bands; and All are integer subscript indices.

[0013] Furthermore, step 2 specifically includes:

[0014] Step 2.1: Obtain any image from any two multispectral cameras at different positions, respectively as the first image and the second image; calculate the disparity map through the first image and the second image according to the position information of the two multispectral cameras;

[0015] Step 2.2: Calculate the structured light depth; calculate the disparity depth through the disparity map; perform depth fusion based on the disparity depth and the structured light depth to obtain the fused depth.

[0016] Furthermore, the structured light depth is calculated using the following formula:

[0017] ;

[0018] in, Represents pixels The structured light depth at is the focal length of structured light; To project light on a pixel The initial position of It is the displacement difference of the projected light after deformation in the scene; is the expected change position of the projected light in the scene; is the standard deviation of the displacement difference of the projected light after deformation in the scene; is the cosine value of the incident angle of the projected light beam; is the incident angle of the projected light beam on the surface; is the reflection loss function; is the reflectance, which indicates the intensity of reflected light The intensity of the incident light ratio.

[0019] Furthermore, the parallax depth is calculated by the following formula:

[0020] ;

[0021] in, Pixel The depth value at ; is the focal length of the multispectral camera; is the physical distance between the two multispectral cameras at different locations in step 2.1; Pixels in the disparity map The disparity value at ; Pixel The depth gradient at is the expected value of the depth gradient; is the standard deviation of the depth gradient.

[0022] Furthermore, the following formula is used to perform deep fusion and obtain the fusion depth: :

[0023] ;

[0024] in, is the set fusion weight.

[0025] Furthermore, in step 3, the anomaly detection probability is obtained by the following formula:

[0026] ;

[0027] in, Indicates at location The probability of anomaly occurring under given fusion depth and historical prior data is the anomaly detection probability; Indicates that in the event of abnormal behavior, the location The probability that the difference between the fusion depth at and the standard value exceeds the judgment threshold; Based on historical prior data, the location The prior probability of abnormal behavior occurring at It is the edge probability that the difference between the fusion depth and the standard value exceeds the judgment threshold.

[0028] Furthermore, Calculated using the following formula:

[0029] ;

[0030] in, Indicates that based on historical prior data, the position The probability of no abnormal behavior; Indicates that when no abnormal behavior occurs, the position The probability that the difference between the fusion depth at and the standard value exceeds the judgment threshold.

[0031] Furthermore, step 3 also includes: when the abnormal detection probability exceeds a set warning value, issuing a warning.

[0032] By adopting the above technical scheme, the present invention produces the following beneficial effects: the present invention introduces a multispectral camera and combines stereoscopic vision and structured light technology, so that the system has excellent three-dimensional space perception ability. Through the image information obtained by multispectral cameras at multiple different positions, the system can generate a three-dimensional fusion image with depth information through parallax calculation and structured light depth extraction technology. Compared with the traditional monitoring system based on two-dimensional images, the present invention can accurately identify the three-dimensional position of objects and personnel in community scenes, especially the accurate perception of information such as personnel height and object distance, which greatly improves the reliability of monitoring. For example, in traditional two-dimensional monitoring, it is difficult for the camera to determine whether a person has climbed over the fence, but through depth information, the present invention can quickly capture the height change of personnel and detect abnormal behavior in time. This three-dimensional space perception ability makes the system perform better in complex environments and can effectively identify potential threats such as personnel intrusion and object displacement. Through the application of multispectral cameras, the present invention can maintain high monitoring accuracy under various complex lighting conditions. Traditional cameras rely on visible light, and monitoring blind spots often appear at night or in environments with drastic light changes. Multispectral cameras can capture light information in multiple bands, including infrared, ultraviolet and other light sources of different wavelengths, so that the system can still obtain clear image information in poor light or even completely dark environments. Combined with the weighted fusion algorithm of the present invention, image information in different bands is weighted to generate the final fused image, ensuring that the system can maintain high-quality monitoring effects regardless of daytime, night, shadow areas or scenes with complex lighting conditions. This strong adaptability under different lighting conditions significantly expands the application scenarios of the system, especially in nighttime security monitoring, behavior detection in low-light environments, etc., and has important practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A method flow chart of a community management method based on a pure visual solution provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0034] All features disclosed in this specification, or steps in all methods or processes disclosed, except mutually exclusive features and / or steps, can be combined in any manner.

[0035] Any feature disclosed in this specification (including any additional claims and abstract), unless otherwise stated, may be replaced by other alternative features that are equivalent or have similar purposes. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.

[0036] Example 1: Reference Figure 1 , a community management method based on a purely visual solution, the method comprising:

[0037] Step 1: Set up multiple multispectral cameras in the community to obtain multispectral image information in the community in real time; process the obtained multispectral image information through non-subsampled contourlet transform and weighted fusion to obtain a fused image;

[0038] First of all, the core advantage of using a multispectral camera is that it can capture light information in different bands, not only limited to the visible light band, but also other bands such as infrared and ultraviolet. Traditional RGB cameras can only capture light information in the three colors of red, green, and blue that the human eye can see, which may lead to the loss of image information under certain environmental conditions, especially in low light or with specific interference. Multispectral cameras can obtain more dimensional environmental information by expanding to more bands. For example, at night or in dark environments, the infrared band can provide more details than the visible light band, which greatly enhances the system's all-weather monitoring capabilities. At the same time, information in different bands can also help the system better identify and classify objects, such as detecting the activity status of the human body through thermal imaging, further improving the accuracy of monitoring.

[0039] After obtaining multispectral image information, it is crucial to efficiently process this image information to maximize its role. Here, the present invention adopts the non-subsampled contourlet transform (NSCT) technology, which is an advanced image processing technology that can decompose the image in multiple scales and directions, thereby capturing detailed information such as edges and textures in the image. Unlike the traditional discrete wavelet transform (DWT), the biggest feature of NSCT is that it has no downsampling process, which means that the resolution of the image will not be reduced after the transformation, and the spatial information of all images can be fully preserved. This is particularly critical for monitoring applications, because in monitoring scenarios, detailed information, such as facial features, object contours, moving boundaries, etc., is often the core basis for judging abnormal behavior and making intelligent decisions. Through NSCT, the system can extract these details from multiple angles and levels, thereby ensuring that the image quality is not degraded due to processing.

[0040] The processed multispectral image is not used in a single form, but the information from different bands is processed comprehensively through weighted fusion technology to generate a fused image. Each band of the multispectral image carries different environmental information, but not all bands have the same importance in all scenarios. Weighted fusion sets the weights of different bands and integrates the information with higher value into the final image first according to the information quality of specific bands, environmental conditions and other factors. This weighting method can be dynamically adjusted according to real-time monitoring needs. For example, in low-light environments, the weight of the infrared band can be increased, while in daytime, the information of the visible light band can be used first. This flexible processing method ensures that the system can generate high-quality fused images in any environment to support subsequent deep calculation and behavior analysis. Traditional monitoring systems usually use only single-band RGB cameras, which makes it difficult to ensure image quality when the light changes and the environment is complex, especially at night or in scenes with uneven light, where the monitoring system is prone to blind spots. The present invention overcomes these limitations through weighted fusion of multispectral images. It can not only obtain more details under complex environmental conditions, but also ensure the best observation effect through intelligent weighted processing. This technical solution not only expands the application scenarios of the visual system, but also provides a more reliable image foundation for subsequent deep information extraction and abnormal behavior detection.

[0041] Step 2: Based on the fused image, use multi-view stereo vision and structured light technology to perform depth fusion to obtain the fusion depth;

[0042] First, multi-view stereo vision technology based on fused images uses multiple cameras to obtain images of the same scene from different angles, thereby using the principle of parallax to infer depth information. The physical principle of this parallax effect is similar to the human binocular visual system: when humans observe objects, due to the different positions of the two eyes, two slightly different images are formed on the retina, and the visual system judges the distance of the object by analyzing these differences. Similarly, in multi-view stereo vision technology, the system can calculate the specific depth information of each pixel in three-dimensional space through the difference between the images taken by the camera from different angles. Compared with traditional single-view images, multi-view images contain more spatial information, so that the three-dimensional scene in the community can be restored more accurately. This has great application value for monitoring and management. For example, in community management, stereo vision can effectively capture the relative position, size and movement trajectory of objects in space, helping the system to more accurately monitor the behavior trajectory of people and the dynamic changes of objects. However, relying solely on multi-view stereo vision is not enough to fully cope with all complex community environments. Especially in the community, the camera may be affected by factors such as lighting conditions and the complexity of the texture of the object surface, resulting in a decrease in the accuracy of depth information acquisition. In order to make up for these shortcomings, the present invention introduces structured light technology to further improve the accuracy of depth information. Structured light technology calculates the depth information of the object by projecting a known light pattern (such as stripes, grids, etc.) into the scene and capturing the deformation of the light pattern on the surface of the object through the camera. When the light pattern is projected onto the surface of the object, if the surface of the object is not completely flat, the shape of the light pattern will be deformed, and the degree of deformation is related to the depth of the object. By accurately analyzing the laws of these deformations, the system can very accurately infer the depth information of the object. Compared with pure visual stereo calculation, structured light technology provides a more active way to obtain depth, especially for those surfaces with less texture and smoother surfaces.

[0043] Step 3: Based on the fusion depth, use the prior Bayesian network to perform anomaly detection and obtain the anomaly detection probability.

[0044] First, the fused depth map provides three-dimensional information of the scene, which includes not only the plane position information of the object, but also the height, volume and relative position relationship of the object in space. The introduction of the depth map enables the system to not only understand the surface features of each object in the community, but also to judge certain abnormal behaviors by analyzing the motion patterns in the depth map. For example, when someone is walking normally in the community, the system can understand the person's regular behavior pattern through historical data and match it with the normal behavior model. However, when someone suddenly crosses a fence or climbs a building, the depth map will clearly show the abnormal changes in the height and position of the person. This change cannot be effectively detected only by two-dimensional images, and the depth information can provide effective data support for this. Based on the depth map, the present invention introduces a priori Bayesian network for anomaly detection. The Bayesian network is a machine learning model based on probabilistic reasoning, which represents the conditional dependency between variables through a graph model and makes inferences and decisions based on known information. In the community management method of the present invention, the priori Bayesian network establishes a probabilistic model of regular behavior in the community by learning a large amount of historical data of community activities. The advantage of the prior Bayesian network is that it can compare new observation data (such as real-time depth map information) with the existing probability model and calculate the probability of whether a certain behavior is an abnormal behavior based on the Bayesian theorem. Specifically, the role of the prior Bayesian network in anomaly detection is reflected in two aspects: First, it can model these regular behaviors as a dynamic probability distribution by learning the distribution of regular activities in the community, such as people's daily paths, stop locations, interactive behaviors, etc. When new behavior data is input, the system can determine whether the behavior is an abnormal phenomenon by calculating the deviation from the regular behavior model. Secondly, the Bayesian network has strong robustness and adaptability, and it can perform online learning and updates based on real-time data. The behavior pattern in the community environment is dynamically changing. For example, a certain area may show different behavior characteristics due to changes in the flow of people in different time periods. The prior Bayesian network can adapt to these changes by continuously updating its own probability model, thereby maintaining efficient anomaly detection capabilities in a dynamic environment.

[0045] Another important feature of Bayesian networks in anomaly detection is that they can handle uncertainty. In actual community monitoring, data is often incomplete or ambiguous. For example, in poor lighting conditions, the camera may not be able to capture a clear image, or some key visual information may be missing due to factors such as occlusion. The probabilistic reasoning characteristics of Bayesian networks enable it to still output a reasonable probability of anomaly detection under these uncertain conditions. This ability to handle uncertainty and fuzzy data makes Bayesian networks particularly outstanding in the actual application of community management. Traditional anomaly detection algorithms often rely on deterministic data input, and once the input data is noisy or incomplete, the detection effect will be greatly reduced. However, by calculating the conditional probability distribution of each observed variable, Bayesian networks can still give relatively accurate anomaly detection results in the case of incomplete data, thereby ensuring that the community management system can operate stably in complex real-world scenarios. In addition, when detecting abnormal behavior, the a priori Bayesian network does not rely solely on a single behavior or scenario, but makes decisions through a comprehensive analysis of multiple information. For example, when someone moves quickly in a community, it may not be possible to directly determine whether it is an abnormal behavior based on this behavior alone. However, if the person's current area, time, and distance changes from surrounding objects are combined, the Bayesian network can make more accurate abnormal judgments based on this comprehensive information. For example, if someone quickly approaches a monitored area, and there should usually be no human activity in the area during this time period, the system will identify the abnormality of the behavior and issue an alarm. By combining the prior Bayesian network, the community management method of the present invention realizes intelligent detection and dynamic adaptability of abnormal behavior. Compared with traditional community monitoring technology, this method can not only perform in-depth analysis of multi-dimensional visual information, but also maintain efficient monitoring and anomaly detection performance when the data is incomplete or uncertain. The innovation of this method is reflected in its ability to use the combination of depth maps and probabilistic models to provide a more accurate and reliable solution for community security management. The prior Bayesian network provides adaptability and robustness for the entire monitoring system, enabling the system to always maintain a high degree of monitoring accuracy and flexibility in a complex community environment, thereby effectively improving the security and management efficiency of the community.

[0046] Example 2: In step 1, the multispectral camera acquires a total of multispectral image of bands; The wavelength is The weight of the multispectral image corresponding to the band is ; The fused image is calculated by the following formula :

[0047] ;

[0048] in, Indicates Multispectral image corresponding to bands The coefficients obtained after performing non-subsampled contourlet transform; is the central wavelength, which is equal to the mean wavelength of all bands; is the standard deviation, which is equal to the standard deviation of wavelengths in all bands; and All are integer subscript indices.

[0049] Specifically, by fusing multispectral image information from multiple bands, the details and features in the images of each band are retained to the maximum extent, thereby generating a fused image containing rich information. This process first involves the acquisition of multispectral images. Multispectral cameras can collect information about community scenes in multiple bands, which can extend from visible light to infrared bands. The details and texture information captured by each band are different. Therefore, how to effectively integrate the information of these different bands to maintain the details of the image while avoiding excessive information redundancy or loss has become a key technical challenge in image fusion. In image fusion, non-subsampled contourlet transform (NSCT) is an important tool. NSCT can decompose images at multiple scales and in multiple directions. Unlike traditional discrete wavelet transform (DWT), NSCT does not involve a downsampling process, so the spatial resolution of the image will not be lost during the transformation process. Through NSCT, the image is decomposed into several coefficients at different scales, which contain important feature information such as edges and textures in the image. Therefore, when the multispectral image of each band is After NSCT transformation, the coefficients obtained can provide the necessary image details for subsequent weighted fusion. Through NSCT, the system extracts the key feature information in each band image, ensuring that these features are retained in the fused image, making it high-fidelity and rich in details.

[0050] At the same time, the formula uses normalization to ensure that the characteristic information of different bands is balanced during the fusion process. The normalization process is actually to standardize the NSCT coefficient of each band to prevent the information of a certain band from being too prominent or ignored. The information of each band is compared with the sum of the coefficients of all bands through its NSCT coefficient. This normalization not only balances the information weights between different bands, but also effectively prevents some band information from being distorted or over-amplified during the fusion process. In addition, a key Gaussian attenuation function is introduced in the formula, which further optimizes the fusion process by weighting the difference in wavelength. Specifically, the image information of different bands is not equivalent. The images of some bands may contain more important details, while other bands may be affected by environmental noise or light conditions. The Gaussian attenuation function dynamically adjusts the weight of each band according to the difference between the wavelength of each band and the central wavelength. The central wavelength here is the mean of the wavelengths of all bands, and the standard deviation It reflects the degree of discreteness of the wavelength distribution between bands. In this way, the closer the wavelength is to the central wavelength, the higher its weight is, which means that its information will occupy a more important position in the fused image. Conversely, the further the wavelength deviates from the central wavelength, the lower its weight is, and the smaller the impact of the information on the final fused image. The design of this weighting strategy is based on the actual application scenarios of images of different bands in community monitoring, because in complex environments, some bands (such as infrared bands) may show superior capture capabilities at night or in poor lighting conditions, while other bands (such as visible light) may be more important in daytime scenes. Gaussian weighting ensures the dynamics and adaptability of image fusion, enabling the system to generate fused images with the highest utility under different conditions. Finally, the weight coefficient The introduction of gives the formula more flexibility. Weight Adjustments can be made according to actual needs to further optimize the image fusion effect. For example, in certain specific scenarios, the information weight of certain specific bands can be prioritized to highlight specific monitoring targets. This design makes the entire community management system highly adaptable and can flexibly adjust monitoring strategies according to different environmental requirements, thereby improving monitoring effects.

[0051] Embodiment 3: Step 2 specifically includes:

[0052] Step 2.1: Obtain any image from any two multispectral cameras at different positions, respectively as the first image and the second image; calculate the disparity map through the first image and the second image according to the position information of the two multispectral cameras;

[0053] The system obtains two images from two multispectral cameras at different positions, which are called the first image and the second image. These images are part of the multi-band images taken by the multispectral camera. Each camera takes pictures at different angles, so the first image and the second image represent different perspectives of the same scene. Through these images at different positions, the depth information of the scene can be derived by calculating the disparity map. The disparity map is generated based on the position difference of the same object in the two images under different perspectives. The distance of the object from the camera will be reflected as different disparity values ​​in the disparity map. The calculation of the disparity map is based on the principle of geometry. The distance information of each point in the scene can be calculated by using the relative position of the two cameras and the corresponding points in the image. Specifically, the calculation process of the disparity map is achieved by comparing the position offset of the same object in the first image and the second image. The larger the displacement, the closer the object is to the camera. Conversely, the smaller the displacement, the farther the object is from the camera. Each pixel value in the disparity map represents the relative depth of the corresponding object in three-dimensional space. Disparity calculation is one of the core technologies of stereo vision. Through this calculation, the system can perform three-dimensional positioning of objects in community scenes. This process simulates the binocular parallax principle in the human visual system and captures the depth information of the scene through the position difference of the binocular cameras. This method has a high spatial resolution and can provide more accurate depth information, especially when dealing with objects at a long distance.

[0054] Step 2.2: Calculate the structured light depth; calculate the disparity depth through the disparity map; perform depth fusion based on the disparity depth and the structured light depth to obtain the fused depth.

[0055] Structured light is an active depth measurement method that projects a known light pattern (such as stripes or grids) into the scene, and then uses a camera to capture the deformation of the light pattern on the surface of the object to calculate the depth information of the object. Compared with disparity maps, structured light technology does not rely on passive light conditions when measuring the depth of an object, but actively generates light sources for measurement. Therefore, structured light is particularly suitable for detecting the depth information of objects with less texture and smooth surfaces. For example, in some scenes, objects with flat or reflective surfaces may not be able to accurately capture the depth through disparity calculation, while structured light can obtain the precise depth of these objects through the deformation of the light pattern. Parallax depth and structured light depth use different principles to extract depth information. The former relies on the geometric difference between the two perspectives, while the latter is based on the deformation of the actively projected light pattern. However, in practical applications, a single method of obtaining depth information may be limited by environmental conditions. For example, when disparity maps are processing scenes with less texture, it may be difficult to capture accurate corresponding points, resulting in inaccurate depth information; while structured light technology may be limited by the attenuation or scattering of the projected light when processing distant objects. Therefore, step 2.2 combines the advantages of the two methods by fusing the disparity depth and structured light depth to obtain more accurate and complete depth information. The depth fusion process can be understood as a weighted average operation. The system assigns different weights to the disparity depth and structured light depth according to the accuracy and reliability of different depth acquisition methods. For certain specific scenes, such as objects at a distance, the disparity map may be more reliable than structured light technology, so the system tends to increase the weight of the disparity depth; on the contrary, at closer distances or with less surface texture, the structured light depth information may dominate. By dynamically adjusting the weights in this way, the depth fusion process can adapt to different environmental conditions and scene changes, thereby ensuring that the generated fused depth information has the highest accuracy.

[0056] Embodiment 4: The structured light depth is calculated by the following formula:

[0057] ;

[0058] in, Represents pixels The structured light depth at is the focal length of structured light; To project light on a pixel The initial position of It is the displacement difference of the projected light after deformation in the scene; is the expected change position of the projected light in the scene; is the standard deviation of the displacement difference of the projected light after deformation in the scene; is the cosine value of the incident angle of the projected light beam; is the incident angle of the projected light beam on the surface; is the reflection loss function; is the reflectance, which indicates the intensity of reflected light The intensity of the incident light ratio.

[0059] Specifically, the formula is the focal length of the structured light system, which indicates the focal position of the light beam emitted by the light source after being focused by the optical system. The focal length directly affects the propagation characteristics of the light beam. Especially in depth calculation, the focal length determines the divergence and projection distance of the light beam. A longer focal length allows the system to maintain higher accuracy at a longer distance, while a shorter focal length is suitable for fine measurement at close range. The choice of focal length depends on the specific scenario of community monitoring. In a large-scale community scenario, a long focal length can ensure the accuracy of depth information acquisition of distant objects. One of the key parameters in the formula is , which indicates the projection light at the pixel The initial position at which the beam should have reached if it had not encountered an object. It indicates the displacement difference caused by the deformation of the projected light after encountering the surface of the object. This displacement difference is the core of structured light depth calculation, because the surface of the object will cause the light beam to deviate from its initial trajectory, and this deviation reflects the depth information of the object surface. By measuring the displacement difference of the light beam from the initial position to the actual position, the system can infer the spatial position of the object surface.

[0060] In order to accurately describe this displacement difference, a Gaussian attenuation function is used in the formula , which describes the deformation of the projected light relative to the desired position The degree of deviation. Expected position It is the average position that the projected light should reach after deformation on the surface of the object in theory. is the standard deviation of this displacement difference. Through the Gaussian attenuation function, this deviation can be weighted. The closer the deviation is to the expected position, the higher the accuracy of the depth calculation. On the contrary, if the deviation is large, it means that the surface of the object is more complex, which may affect the accuracy of the depth. Therefore, the system will adjust the weight of the depth information accordingly. This processing method enables structured light depth calculation to have a certain fault tolerance when dealing with complex object surfaces, avoiding errors caused by surface irregularities or noise affecting the overall depth calculation. The formula also takes into account the incident angle of the light beam. Impact on depth calculation. The incident angle of the light beam determines how the light intersects the surface of the object, so the cosine value of the incident angle needs to be introduced when calculating the depth. To correct the degree of beam deviation. A smaller incident angle will cause the beam to project a longer length on the surface, which in turn affects the degree of beam deformation and displacement difference, while a larger incident angle will cause the beam to project a smaller length on the surface, and the deformation will be less obvious. By introducing the cosine correction term, the formula can more accurately reflect the actual deviation of the beam, thereby improving the accuracy of depth calculation. Reflection loss It is another important factor affecting depth calculation. In actual scenes, when light encounters different surfaces, its reflection characteristics will be different. Some light may be absorbed and some light will be reflected back to the camera. Reflection loss function It represents the ratio of the reflected light intensity to the incident light intensity. Through this ratio, the system can quantify the light reflection ability of the object surface. If the reflection loss is large, that is, A higher value means that most of the incident light is reflected by the surface of the object. In this case, the depth calculation will rely more on the characteristics of the reflected light; If the value is small, it means that the surface of the object absorbs most of the light, and the system needs to make corresponding adjustments to the depth calculation to avoid errors caused by too little reflected light. The effect of reflected light on depth information has been further corrected to ensure stable depth calculation results on both high-reflection and low-reflection surfaces.

[0061] Embodiment 5: Calculate the parallax depth by the following formula:

[0062] ;

[0063] in, Pixel The depth value at ; is the focal length of the multispectral camera; is the physical distance between the two multispectral cameras at different locations in step 2.1; Pixels in the disparity map The disparity value at ; Pixel The depth gradient at is the expected value of the depth gradient; is the standard deviation of the depth gradient.

[0064] Specifically, the core part of the formula is To calculate the pixel The initial depth value at . Here, Represents the focal length of the multispectral camera. The focal length determines the imaging capability of the camera, especially when processing distant objects. A longer focal length can improve the depth calculation accuracy of distant targets. represents the baseline distance between the two cameras, that is, the physical distance between the two cameras mentioned in step 2.1. The longer the baseline, the greater the parallax that the system can produce, thereby improving the resolution of the depth calculation. By multiplying the focal length of the camera and the baseline distance, the displacement difference in the disparity map can be effectively quantified as the actual depth of the object. Next, is the pixel in the disparity map The disparity value at the position. The disparity value reflects the difference in pixel displacement of the same object in two views in the images taken by the camera from different angles. According to the principle of geometry, the closer the object is to the camera, the greater the disparity, and the farther the object is, the smaller the disparity. Therefore, the depth of the object can be inferred from the disparity value. The calculation of the disparity value is the key to the entire stereoscopic vision depth estimation, which reflects the relative position of the object in three-dimensional space. Based on the relationship in the formula, the larger the disparity value, the deeper the depth. The smaller the parallax value, the closer the object is to the camera; conversely, the smaller the parallax value, the deeper the depth. The larger it is, the farther the object is from the camera.

[0065] In addition to the disparity value and camera parameters, a Gaussian correction term is also introduced in the formula , used to adjust and optimize the results of depth calculation. Represents pixels The depth gradient at the pixel point is the rate at which the depth value changes around the pixel point. In real scenes, the depth gradient reflects the degree of change on the surface of an object. For example, a flat surface has a smaller depth gradient, while a steep surface has a larger depth gradient. By calculating the depth gradient, the system can identify which areas in the scene have significant depth changes, and thus pay extra attention to and optimize these areas. Expected depth gradient and the standard deviation of the depth gradient Respectively represent the expected value and allowed range of variation of the normal depth gradient of the system. These parameters are usually determined by prior scene knowledge or a large amount of training data. The Gaussian correction term of the depth gradient adjusts the depth calculation weight at the pixel point so that the depth estimation can better adapt to local changes in the scene. For example, if the depth gradient of a pixel point is far from the expected value, it means that there may be a large depth change or error in the area, and the system will reduce the trust in the depth value through the correction term. On the contrary, if the depth gradient is close to the expected value, it means that the depth estimation at the pixel is more reliable, and the system will retain this estimation result. The importance of this correction process lies in that it can improve the robustness of depth calculation, especially in complex scenes, such as trees, buildings and pedestrians in the community. The surfaces of objects have different depth characteristics. By correcting the depth gradient, the system can more accurately capture the depth information of these complex objects. For example, when monitoring pedestrians in the community, the movement and posture of pedestrians will cause dynamic changes in the depth gradient. By correcting these changes, the system can more accurately track the position and behavior of pedestrians. Near static objects (such as buildings), the depth gradient is small, and the system can use this information to ensure that the depth estimation of static areas is stable and accurate. In addition, the introduction of this correction term can also help to deal with some potential depth estimation errors. Disparity calculation depends on the matching accuracy of the camera. In actual applications, due to reasons such as light and occlusion, the disparity calculation of certain areas may be biased. Through the depth gradient correction term, the system can effectively reduce the impact of these errors and ensure that the final generated depth map is more reliable.

[0066] Example 6: Perform deep fusion using the following formula to obtain the fusion depth: :

[0067] ;

[0068] in, is the set fusion weight.

[0069] Specifically, the formula is a key fusion weight parameter that determines the parallax depth and structured light depth The relative contribution to the final fusion depth. The value of can control the priority of these two depth acquisition methods in different scenarios. In practical applications, The value of can be dynamically adjusted according to the characteristics of the scene, changes in the environment, and monitoring requirements to maximize the accuracy and robustness of the fusion depth. It is calculated by the stereo vision system of the multispectral camera and has strong long-distance measurement capabilities. When the distance is far or the scene is relatively open, parallax depth can usually provide a more accurate depth estimate because the camera parallax can effectively capture the geometric differences between distant objects. Moreover, under complex lighting conditions (such as strong light or weak light environments), parallax depth relies on the difference in geometric position and is not easily affected by changes in light intensity. Therefore, under such conditions, The value of may be set relatively large, making the disparity depth dominate the fusion depth.

[0070] On the other hand, structured light depth Depth is measured by projecting a known light pattern into the scene and analyzing the deformation of the light. Compared to parallax depth, structured light depth has advantages in close-range measurement and complex surface processing. Especially when the surface of the object has less texture or the surface is smooth, structured light can provide very detailed depth information through precise light pattern deformation. In these scenes, parallax calculations may fail to match due to lack of sufficient texture information, while structured light depth can reliably obtain the depth of close objects. Therefore, when the system detects that the target object is at a closer distance or the surface is more complex, The value of may be smaller, and the weight of structured light depth is greater to ensure the accuracy of depth estimation. In actual community management scenarios, distant and near scenes often exist at the same time. For example, a surveillance camera may need to capture distant buildings and nearby pedestrian activities at the same time. Through this depth fusion formula, the system can intelligently fuse parallax depth and structured light depth according to the needs of specific scenes, thereby providing the most suitable three-dimensional scene information. The fused depth It combines the long-distance measurement advantages of parallax depth and the close-range fine measurement capabilities of structured light depth, providing the monitoring system with a full range of depth perception capabilities. In addition to processing long and short distance scenes, the depth fusion formula also has the ability to handle environmental changes. In scenes with large changes in lighting conditions, such as from day to night, or in complex lighting environments (such as tree shadows, vehicle lights, etc.), the reliability of parallax depth and structured light depth may be affected to varying degrees. By adjusting the fusion weights , the system can dynamically choose a more reliable depth estimation method under different lighting conditions. For example, at night, structured light depth may be more reliable than parallax depth due to the stability of light projection, so The value of may be reduced, making the structured light depth play a larger role in the fusion process.

[0071] Example 7: In step 3, the anomaly detection probability is obtained by the following formula:

[0072] ;

[0073] in, Indicates at location The probability of anomaly occurring under given fusion depth and historical prior data is the anomaly detection probability; Indicates that in the event of abnormal behavior, the location The probability that the difference between the fusion depth at and the standard value exceeds the judgment threshold; Based on historical prior data, the location The prior probability of abnormal behavior occurring at It is the edge probability that the difference between the fusion depth and the standard value exceeds the judgment threshold.

[0074] Specifically, the core part of the formula is , which means that at a given fusion depth and historical prior data In case of location The probability of an anomaly occurring at a location. This probability quantifies the system's judgment of abnormal behavior and is the final output of the entire anomaly detection process. This probability depends not only on the depth information of the location at the current moment, but also combines the behavior patterns and event records in the historical data. By introducing historical prior data, the system can evaluate the current detected situation based on similar events or behaviors that have occurred before. If a location has frequently experienced abnormal behavior in history, the system will increase the sensitivity of abnormalities at that location in the current analysis. In the formula, the depth information It plays a vital role in anomaly detection. Fusion Depth It is obtained through the previous steps, combining the advantages of parallax depth and structured light depth. Depth information can reflect the relative position, shape, and spatial layout of objects in the scene. Therefore, in behavior detection, abnormal changes in depth often indicate abnormal behavior of objects or people. For example, in a community scene, if the depth value of a certain area suddenly changes dramatically, it may mean that there are people moving quickly or objects in the area have shifted, which may constitute potential abnormal behavior.

[0075] In the formula Reflects the depth of structured light Fusion Depth This difference is used to determine the degree of deviation between the current measured depth and the standard depth. The larger the depth difference, the more obvious the difference between the current scene and the normal state, and the greater the possibility of abnormality. Therefore, by comparing this difference with a certain standard threshold By comparing, the system can effectively determine whether the depth of the location is beyond the normal range, thereby identifying possible abnormal behavior. The next part of the formula further refines the probability calculation of anomaly detection through Bayesian reasoning. Indicates at location The probability that the difference between the fused depth and the standard depth exceeds a certain threshold when abnormal behavior occurs. This probability reflects the changing characteristics of the depth information under the premise that the abnormality is known to occur. If the change of depth information is regular and predictable when abnormal behavior occurs, the system can more accurately identify the characteristics of abnormal behavior through this part.

[0076] at the same time, Represents the historical prior data, at the location The prior probability of abnormal behavior occurring at a location. This prior probability is based on the system's analysis and learning of past data. If there have been many abnormal events in the history of a specific location, such as multiple illegal entries into an area or frequent abnormal movement of objects in the area, then the prior probability of that location will be higher. When the system detects new data at the current moment, the prior probability will affect the entire anomaly detection result, making the system more sensitive to abnormal behavior at that location. The introduction of this historical data gives anomaly detection a "memory" function, which can make more accurate judgments about current behavior based on past experience. Finally, in the formula is the marginal probability that the fused depth is outside the normal range. This marginal probability is an overall probability estimate of the depth change in the entire scene, independent of whether abnormal behavior occurs. By comparing the actual depth change with the expected depth change, the system can adjust the confidence of the anomaly detection. If the depth information occasionally fluctuates greatly when there is no abnormal behavior, this part of the probability can help the system reduce the possibility of false alarms. Through this formula, the system can estimate the confidence level of each pixel. Conduct probabilistic analysis of abnormal behavior. The design of this formula combines multiple factors, including changes in depth information, prior probabilities of historical data, and the overall characteristics of depth changes, to ensure the accuracy and reliability of anomaly detection results. Especially in complex community scenarios, the formula can dynamically adjust the judgment of abnormal behavior, so that the system still has high sensitivity and robustness in dealing with different environmental conditions. For example, in a community management scenario, the system can use this formula to monitor the depth information of multiple areas in real time. If the depth of a certain location suddenly changes significantly, and abnormal events have occurred frequently in the area in history, the system will combine historical data and current depth changes to calculate the probability of anomalies at that location. This design enables the system to issue timely warnings when people illegally intrude, objects move abnormally, or potential security threats appear, thereby improving the efficiency and intelligence of community security management.

[0077] Embodiment 8: Calculated using the following formula:

[0078] ;

[0079] in, Indicates that based on historical prior data, the position The probability of no abnormal behavior; Indicates that when no abnormal behavior occurs, the position The probability that the difference between the fusion depth at and the standard value exceeds the judgment threshold.

[0080] Specifically, by evaluating the probabilities in the two cases respectively, the formula derives the overall probability of whether the difference between the fusion depth and the standard value exceeds the judgment threshold in the entire scene. This method ensures that the system can not only make reasonable judgments when an anomaly is detected, but also minimize the possibility of false alarms when no anomaly occurs. First, the formula Indicates at location The probability that the difference between the fused depth and the standard depth exceeds the judgment threshold when abnormal behavior occurs. This probability reflects how the depth information deviates from the normal state when abnormal behavior occurs. For example, in a community, if someone suddenly approaches the camera quickly, or an object is moved, these abnormal behaviors may cause significant changes in the depth value. In this case, the change in depth information is recognized by the system as beyond the normal range, so This probability is usually based on the deep change patterns of similar abnormal behaviors in history. By analyzing historical abnormal events, the system can establish a relationship model between abnormal behaviors and deep changes. is in position The prior probability of abnormal behavior occurring at a location. This prior probability is based on historical data and specific conditions of the scenario. For example, if a location has experienced multiple abnormal events in past monitoring records, or if the location is a high-risk area (such as an entrance or exit, a monitoring blind spot, etc.), then The value of will be higher. This enables the system to more accurately identify potential abnormal behaviors when evaluating current data. The introduction of prior probability enhances the system's predictive ability, especially when there are no clear signs of abnormal behavior, historical data can provide an effective reference for the system.

[0081] Next, the other part of the formula It reflects the probability that the difference between the fused depth and the standard depth exceeds the judgment threshold when no abnormal behavior occurs. This part is very important because under normal conditions, changes in depth information may also fluctuate due to certain environmental factors (such as light, shadow, reflection, etc.). By calculating this probability, the system can avoid mistaking these normal fluctuations for abnormal behavior. This probability is usually low unless environmental changes significantly affect the depth information. In practical applications, the calculation of this part of the probability can also be based on historical data. By analyzing the depth fluctuations under normal conditions in the past, the system can build a depth change model under normal conditions. At the same time, Indicates at location The prior probability that no abnormal behavior occurs at . This probability is usually related to They are complementary sets and reflect the system's judgment of the state of the location when no abnormal behavior is detected. For example, if a location has historically been in a normal state or is a low-risk area, then The value of will be higher, which means that the system is more inclined to think that the position is normal based on the current data. By combining these two parts, the formula finally calculates the position The overall probability that the fusion depth exceeds the judgment threshold . This result plays an important role in anomaly detection because it helps the system determine whether the current depth information change exceeds the normal range and whether this change means potential abnormal behavior. The higher the probability value, the more the system tends to believe that there is abnormal behavior in the current scene, otherwise, the scene state is considered normal. The design of the formula fully considers the difference in the performance of abnormal behavior and normal behavior in depth information, and combines historical data with current detection results through Bayesian reasoning. This method greatly improves the judgment accuracy of the system, making the community management system more intelligent and adaptive in anomaly detection. For example, under poor lighting conditions at night, depth information may fluctuate due to environmental changes, but if the prior probability of the location shows that abnormal behavior is unlikely to occur here, the system can reduce the probability of false alarms through this formula. Similarly, in some high-risk areas, even if the environmental changes are small, the system can still improve the sensitivity of anomaly detection due to the repeated abnormal events in the area in history. In summary, Example 8 combines historical prior data with current depth changes through the fusion of Bayesian reasoning and depth information to accurately calculate the location. The overall probability of depth change This formula provides a reliable mathematical basis for anomaly detection in community management systems, allowing the system to flexibly adjust its judgment of abnormal behavior based on changes in deep information and historical data. Through this intelligent deep information processing method, the system can better respond to community monitoring needs in complex environments and improve community security and management efficiency.

[0082] Abnormal behavior is usually accompanied by significant changes in the position of objects or people in the scene, and depth information (such as parallax depth and structured light depth) can accurately capture these changes. Depth information refers to the distance of an object or person relative to the camera in three-dimensional space. When someone suddenly approaches the camera, an object is displaced, or an abnormal movement trajectory occurs, the depth information will change significantly. Compared with traditional two-dimensional image information, depth information directly reflects the geometric changes in space, so it is very sensitive to capturing the occurrence of abnormal behavior. For example, when a person moves in a normal state, the depth change will show a smooth trend; if someone suddenly climbs over a fence or moves quickly, the depth information will fluctuate violently. This spatial change is an important feature of abnormal behavior, and the depth fusion method is to detect abnormalities by capturing these changes. The fusion of parallax depth and structured light depth improves the accuracy of the system in judging abnormal behavior. A single depth acquisition method may have limitations in certain specific scenarios. For example, parallax depth works better for distant objects, while structured light depth has more advantages in processing close objects and surface details. By fusing these two types of depth information, the system can obtain more accurate three-dimensional data in various scenarios. Fusion depth can better reflect the real spatial situation, especially in complex community environments, where the texture of the object surface, lighting conditions, and dynamic changes in the scene may affect depth perception. Through weighted fusion, the system can dynamically adjust the weights of different depth information to ensure that both the movement of pedestrians at close range and the change of objects at a distance can be accurately perceived. This fusion of multi-source information provides the system with a more comprehensive spatial perception capability, thereby improving the accuracy of anomaly detection. The judgment of abnormal behavior not only depends on the change of depth information at the current moment, but is also closely related to historical data and prior knowledge. Bayesian reasoning provides a reasonable decision-making mechanism by combining current observation data (fusion depth) with historical prior information. The core of the Bayesian formula is that it can calculate the probability of an event through probability updates. For abnormal behavior detection, Bayesian reasoning can adjust the current abnormal judgment based on the frequency of abnormal occurrence at a certain location in history.

[0083] For example, the formula and Represent the prior probabilities of an abnormality occurring and not occurring at that location, respectively. If a location (such as a community entrance or exit, a monitoring blind spot) has experienced many abnormal events in history, the system will use historical prior data to increase the sensitivity of anomaly detection at that location. This means that even if the depth information changes slightly, the system will consider that the location may have potential abnormal behavior, thereby increasing the probability of anomaly detection. Similarly, if historical data shows that there has been almost no abnormal behavior at a certain location, the system will reduce its sensitivity to abnormalities at that location to avoid false alarms. The advantage of this decision-making mechanism is that it not only relies on current observation data, but also makes adjustments through historical experience, making the system more adaptive when facing complex environments. The probabilistic update mechanism of Bayesian reasoning ensures that the system can reasonably combine current data with historical experience, which can not only effectively detect abnormal behavior, but also reduce the false alarm rate, thereby improving the reliability of the overall system. In practical applications, abnormal behavior is often accompanied by significant depth changes. The formula Indicates the probability that the fused depth value exceeds a certain threshold when abnormal behavior occurs. Usually, abnormal behavior (such as climbing over a fence, sudden movement of an object) causes the position of objects or people in the scene to change rapidly, and the depth information can capture these changes. Therefore, sudden changes in depth can be used as an important indicator for judging abnormal behavior. For example, if the depth information of a certain area fluctuates dramatically in a short period of time, the system can infer that there may be abnormal behavior in that area. At the same time, It reflects the possibility of depth change when there is no abnormal behavior. When there is no abnormal behavior, the depth change is usually relatively gentle. Even if there are some natural fluctuations (such as changes in ambient light and movement of small objects), the fluctuation of depth information will not exceed the standard threshold. Therefore, by calculating the possibility of depth change and combining the conditions of whether abnormal behavior occurs, the formula can accurately determine whether the current depth information change is worthy of attention and make a judgment.

[0084] Embodiment 9: Step 3 also includes: when the abnormal detection probability exceeds the set warning value, issuing a warning.

[0085] Although the specific embodiments of the present invention are described above, it should be understood by those skilled in the art that these specific embodiments are only illustrative, and those skilled in the art may omit, replace, and change the details of the above methods and systems in various ways without departing from the principles and essence of the present invention. For example, merging the above method steps so as to perform substantially the same functions in substantially the same manner to achieve substantially the same results is within the scope of the present invention. Therefore, the scope of the present invention is limited only by the appended claims.

Claims

1. A community management method based on a purely visual solution, characterized in that: The method comprises: Step 1: Set up multiple multispectral cameras in the community to obtain multispectral image information in the community in real time; process the obtained multispectral image information through non-subsampled contourlet transform and weighted fusion to obtain a fused image; Step 2: Based on the fused image, use multi-view stereo vision and structured light technology to perform depth fusion to obtain the fusion depth; Step 3: Based on the fusion depth, use the prior Bayesian network to perform anomaly detection and obtain the anomaly detection probability; Step 2 specifically includes: Step 2.1: Obtain any image from any two multispectral cameras at different positions, respectively as the first image and the second image; calculate the disparity map through the first image and the second image according to the position information of the two multispectral cameras; Step 2.2: Calculate the structured light depth; calculate the disparity depth through the disparity map; perform depth fusion based on the disparity depth and the structured light depth to obtain the fusion depth; The structured light depth is calculated using the following formula: ; in, Represents pixels The structured light depth at is the focal length of structured light; To project light on a pixel The initial position of It is the displacement difference of the projected light after deformation in the scene; is the expected change position of the projected light in the scene; is the standard deviation of the displacement difference of the projected light after deformation in the scene; is the cosine value of the incident angle of the projected light beam; is the incident angle of the projected light beam on the surface; is the reflection loss function; is the reflectance, which indicates the intensity of reflected light The intensity of the incident light The ratio of The parallax depth is calculated using the following formula: ; in, Pixel The depth value at ; is the focal length of the multispectral camera; is the physical distance between the two multispectral cameras at different locations in step 2.1; Pixels in the disparity map The disparity value at ; Pixel The depth gradient at is the expected value of the depth gradient; is the standard deviation of the depth gradient; Through the following formula, deep fusion is performed to obtain the fusion depth : ; in, is the set fusion weight.

2. The community management method based on pure visual solution as claimed in claim 1, characterized in that: In step 1, it is assumed that the multispectral camera acquires a total of multispectral image of bands; The wavelength is The weight of the multispectral image corresponding to the band is ; The fused image is calculated by the following formula : ; in, Indicates Multispectral image corresponding to bands The coefficients obtained after performing non-subsampled contourlet transform; is the central wavelength, which is equal to the mean wavelength of all bands; is the standard deviation, which is equal to the standard deviation of wavelengths in all bands; and All are integer subscript indices.

3. The community management method based on pure visual solution as claimed in claim 2, characterized in that: In step 3, the anomaly detection probability is obtained by the following formula: ; in, Indicates at location The probability of anomaly occurring under given fusion depth and historical prior data is the anomaly detection probability; Indicates that in the event of abnormal behavior, the location The probability that the difference between the fusion depth at and the standard value exceeds the judgment threshold; Based on historical prior data, the location The prior probability of abnormal behavior occurring at It is the edge probability that the difference between the fusion depth and the standard value exceeds the judgment threshold.

4. The community management method based on pure visual solution as claimed in claim 3, characterized in that: Calculated using the following formula: ; in, Indicates that based on historical prior data, the position The probability of no abnormal behavior; Indicates that when no abnormal behavior occurs, the position The probability that the difference between the fusion depth at and the standard value exceeds the judgment threshold.

5. The community management method based on pure visual solution as claimed in claim 4, characterized in that: Step 3 also includes: when the abnormal detection probability exceeds the set warning value, issuing a warning.

Citation Information

Patent Citations

  • A hybrid depth estimation method based on light field data

    CN109360235A

  • Remote sensing image fusion method based on HIS, NSCT and PCNN

    CN115272845A