Suspicious Target Detection Method and Apparatus Based on Visible Light Images and Thermal Imaging

Abstract

This application discloses a method and apparatus for detecting suspicious targets based on visible light images and thermal imaging, relating to the fields of image processing and computer vision technology. The method includes: acquiring N images of a target area using a visible light camera to obtain N visible light images; outlining the contours of fixed objects in the visible light images to obtain a base image of the target area; acquiring thermal imaging images of the suspicious target using a thermal imaging camera; marking judgment points in the thermal imaging image based on a preset human body proportion to obtain a frame of the suspicious target; performing body shape matching and contour application on the frame of the suspicious target based on a human body shape database to obtain the human body contour corresponding to the suspicious target; projecting the human body contour corresponding to the suspicious target onto the base image to obtain the target image; and detecting the behavioral characteristics of the suspicious target in the target area based on the target image. This application solves the technical problem of insufficient real-time performance in identifying the behavioral characteristics of suspicious targets in existing technologies.

Description

Technical Field

[0001] This application relates to the fields of image processing and computer vision technology, and more specifically, to a method and apparatus for detecting suspicious targets based on visible light images and thermal imaging. Background Technology

[0002] In the process of monitoring dangerous areas, the accuracy, timeliness and environmental adaptability of target recognition directly determine the effectiveness of the task. Traditional target recognition technologies mostly use a single modal image to perform comprehensive edge detection on the target in order to identify the target outline.

[0003] Traditional recognition processes require full-pixel-level target segmentation and feature extraction, and complex operations such as edge detection and contour tracking must be performed pixel by pixel. This results in high computational load and response latency, making it difficult to meet the rapid response requirements of real-time monitoring scenarios. At the same time, there is a lack of efficient methods for constructing the contours of suspicious targets, relying heavily on complex detection algorithms. The framework construction process is not simplified by incorporating the regularity of human skeletal structure, resulting in low practicality and functionality.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides a method and apparatus for detecting suspicious targets based on visible light images and thermal imaging, so as to at least solve the technical problem of insufficient real-time performance in the recognition of behavioral features of suspicious targets in the prior art.

[0006] According to one aspect of the embodiments of this application, a method for detecting suspicious targets based on visible light images and thermal imaging is provided, comprising: acquiring N images of a target area using a visible light camera to obtain N visible light images; then outlining the contours of fixed objects in the visible light images to obtain a base image of the target area, wherein N is an integer greater than or equal to 1; acquiring thermal imaging images of the suspicious target using a thermal imaging camera; then marking judgment points in the thermal imaging image based on a preset human body proportion to obtain a frame of the suspicious target; performing body shape matching and contour application on the frame of the suspicious target based on a human body shape library to obtain a human body contour corresponding to the suspicious target; projecting the human body contour corresponding to the suspicious target onto the base image to obtain a target image; and detecting the behavioral characteristics of the suspicious target in the target area based on the target image.

[0007] Optionally, the target area is captured N times by a visible light camera to obtain N visible light images, including: acquiring parameter information of the target area, wherein the parameter information of the target area includes at least: the area of ​​the target area and the monitoring distance; and capturing N visible light images of the target area N times by a visible light camera based on the parameter information of the target area.

[0008] Optionally, the outline of a fixed object in the visible light image is delineated to obtain a base image of the target region. This includes: converting each acquired visible light image from color space to grayscale space to obtain N grayscale images of the target region and assembling them into an image sequence; initializing K Gaussian distributions for each pixel in the grayscale images of the image sequence, and setting the initial mean, variance, and weight of each Gaussian distribution, where K is an integer greater than 1; obtaining the grayscale value of each pixel in the grayscale image; determining the matching degree between each pixel and each Gaussian distribution based on the grayscale value of each pixel and the initial mean of each Gaussian distribution of that pixel; and delineating the outline of the fixed object in the visible light image based on the matching degree between each pixel and each Gaussian distribution of that pixel to obtain a base image of the target region.

[0009] Optionally, based on the grayscale value of each pixel and the initial mean of each Gaussian distribution of that pixel, the matching degree between each pixel and each Gaussian distribution of that pixel is determined, including: calculating the difference between each pixel and the initial mean of the i-th Gaussian distribution of that pixel, and calculating the absolute value of the difference to obtain the matching degree between the pixel and the i-th Gaussian distribution of that pixel, where i is a positive integer less than or equal to K.

[0010] Optionally, based on the matching degree of each pixel with each Gaussian distribution of that pixel, the contour of a fixed object in the visible light image is delineated to obtain a base image of the target region. This includes: detecting whether the matching degree of each pixel with the i-th Gaussian distribution of that pixel is greater than a target threshold, and obtaining a detection result, wherein the target threshold is determined based on the variance of the i-th Gaussian distribution of that pixel; and delineating the contour of the fixed object in the visible light image based on the detection result to obtain a base image of the target region.

[0011] Optionally, the matching degree between each pixel and the i-th Gaussian distribution of that pixel is greater than a target threshold is detected to obtain a detection result, including: if the matching degree between any pixel and the i-th Gaussian distribution of that pixel is greater than the target threshold, then it is determined that the matching degree between the pixel and the i-th Gaussian distribution of that pixel has failed; if the matching degree between any pixel and the i-th Gaussian distribution of that pixel is less than or equal to the target threshold, then it is determined that the matching degree between the pixel and the i-th Gaussian distribution of that pixel has succeeded.

[0012] Optionally, based on the detection results, the contours of fixed objects in the visible light image are delineated to obtain the base image of the target region. This includes: calculating the ratio of the weight to the variance of each Gaussian distribution to obtain the target value of each Gaussian distribution; based on the target value of each Gaussian distribution, the K Gaussian distributions for each pixel are sorted in descending order, and then the top B Gaussian distributions are selected as the background model for that pixel, where B needs to satisfy... T is the background threshold; based on the detection results, if any Gaussian distribution in the background model of each pixel successfully matches the pixel, then the pixel is determined to be a background pixel; if every Gaussian distribution in the background model of each pixel fails to match the pixel, then the pixel is determined to be a foreground pixel; based on the background pixels and foreground pixels, the outline of the fixed object in the visible light image is drawn to obtain the basic image of the target area.

[0013] Optionally, the contours of fixed objects in the visible light image are delineated based on background and foreground pixels to obtain a base image of the target region. This includes: assigning a value of 255 to the detected foreground pixels and a value of 0 to the background pixels to generate a binary image; using an edge detection algorithm to extract the contours of fixed objects within the target region based on the binary image; swapping the background and foreground pixel values ​​in the binary image, and then using the edge detection algorithm again to extract the background edge contours to obtain the complete contours of the target region; approximating the extracted contours of fixed objects and the contours of the target region using the Douglas-Puk algorithm, then performing polygon fitting using the least squares method, and drawing the fitted polygons on a blank image to obtain the base image of the target region.

[0014] Optionally, based on a preset human body proportion, judgment points are marked in the thermal imaging image to obtain a suspicious target frame, including: determining a scale based on the camera parameters of the thermal imaging camera and the size of the reference object corresponding to the target area; performing a scale conversion on the suspicious target according to the scale to obtain the human height result of the suspicious target; marking the judgment points of the suspicious target according to the human height result of the suspicious target and in combination with the preset human body proportion, wherein the judgment points include the start, end, and midpoint of the forearm, upper arm, lower leg, thigh, chest cavity, waist and abdomen, and the center of the head; identifying points on both sides of the contour of the same judgment point, and taking the midpoint connecting the identified points on both sides as the frame point; connecting all the frame points to obtain the suspicious target frame.

[0015] Optionally, the scale is determined based on the camera parameters of the thermal imaging camera and the size of the reference object corresponding to the target area, including: determining the proportional relationship between pixels at different positions in the thermal imaging image and the actual physical size based on the camera parameters of the thermal imaging camera and the actual size of the reference object in the thermal imaging image; and determining the scale based on the proportional relationship, according to the actual length of the reference object and the pixel length corresponding to the reference object in the thermal imaging image.

[0016] Optionally, the height of the suspected target is calculated based on the scale, including: converting the pixel size into the actual physical size according to the number of pixels corresponding to the height of the suspected target in the thermal imaging image; obtaining the pixel distance between the top of the head and the bottom of the feet of the suspected target in the thermal imaging image; and calculating the product of the pixel distance and the scale to obtain the height of the suspected target.

[0017] Optionally, point identification is performed on both sides of the contour of the same judgment point, and the midpoint connecting the identified points on both sides is used as the frame point. This includes: for each obtained judgment point, setting a search range centered on each judgment point; detecting an edge image based on the search range using an edge detection algorithm; tracking the contour in the edge image, obtaining the coordinates of each point on the contour, and calculating the curvature of each point on the contour, wherein the curvature is obtained by calculating the angle formed by three adjacent points; when the curvature of any point on the contour in the edge image is greater than the curvature threshold, the point is marked as a feature point; the coordinates of the feature points on both sides of each judgment point are recorded respectively; the coordinate values ​​of the feature points on both sides are averaged to obtain the coordinate value of the midpoint connecting the points on both sides, and then the midpoint connecting the points is used as the frame point.

[0018] Optionally, all frame points are connected to obtain a suspected target frame, including: using the coordinates of the frame points, first connecting the center point of the head to the midpoint of the chest cavity based on the human body sequence, then connecting the start and end points of the chest cavity sequentially from the midpoint of the chest cavity, then connecting the start, mid, and end points of the waist and abdomen, then connecting the start, mid, and end points of the upper arm and the forearm, and finally connecting the start, mid, and end points of the thigh and the lower leg to obtain a suspected target frame.

[0019] Optionally, based on a human body shape database, body shape matching and contour application are performed on the suspicious target frame to obtain the human body contour corresponding to the suspicious target. This includes: collecting contour images of people with different body shapes and establishing a human body shape database. For each collected human body contour image, the midpoint contour distances of the thigh, upper arm, chest, and waist are calculated respectively. The midpoint contour distances of the thigh, upper arm, chest, and waist corresponding to each human body contour image are used as a set of feature vectors and stored in the human body shape database. A judgment vector is established based on the midpoint contour distances of the thigh, upper arm, chest, and waist of the suspicious target. The cosine similarity between the judgment vector and the feature vectors in the human body shape database is calculated. The human body contour with the highest cosine similarity is selected from the human body shape database as the contour corresponding to the suspicious target.

[0020] Optionally, detecting the behavioral characteristics of a suspicious target in the target region based on the target image includes: obtaining a first set of points on the human contour corresponding to the suspicious target and a second set of points on the contour of a dangerous fixed object in the target image; for each point in the first set of points, calculating the distance from that point to each point in the second set of points, and then sorting all the calculated distances in ascending order to obtain the shortest distance from each point in the first set of points to the contour of the dangerous fixed object; determining a target point from the first set of points based on the shortest distance from each point in the first set of points to the dangerous fixed object, wherein the target point is the point in the first set of points that is closest to the contour of the dangerous fixed object; and determining that the suspicious target has dangerous behavior in the target region when the target distance between the target point and the contour of the dangerous fixed object is greater than a dangerous distance threshold.

[0021] Optionally, the suspicious target detection method based on visible light images and thermal imaging further includes: after determining that the suspicious target has dangerous behavior in the target area, taking each point in the first point set as the point to be judged, sequentially using the ray method to determine the overlap of the contours of the dangerous fixed objects, and marking the suspicious targets with overlap as dangerous.

[0022] According to another aspect of the embodiments of this application, a suspicious target detection device based on visible light images and thermal imaging is also provided, comprising: a first acquisition unit, configured to acquire N images of a target area using a visible light camera to obtain N visible light images, and then delineate the outline of a fixed object in the visible light images to obtain a base image of the target area, wherein N is an integer greater than or equal to 1; a second acquisition unit, configured to acquire thermal imaging images of the suspicious target using a thermal imaging camera, and then mark judgment points in the thermal imaging image based on a preset human body proportion to obtain a frame of the suspicious target; a first processing unit, configured to perform body shape matching and outline application on the frame of the suspicious target based on a human body shape library to obtain a human body outline corresponding to the suspicious target; a second processing unit, configured to project the human body outline corresponding to the suspicious target onto the base image to obtain a target image; and a behavior feature detection unit, configured to detect the behavior features of the suspicious target in the target area based on the target image.

[0023] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, which stores a computer program, wherein when the computer program is executed, the device on which the computer-readable storage medium is located performs the above-described suspicious target detection method based on visible light images and thermal imaging.

[0024] According to another aspect of the embodiments of this application, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to perform the above-described suspicious target detection method based on visible light images and thermal imaging.

[0025] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program or instructions that, when executed by a processor, implement the above-described method for detecting suspicious targets based on visible light images and thermal imaging.

[0026] In this application, the target recognition system first acquires N images of the target area using a visible light camera, obtaining N visible light images. Then, it delineates the contours of fixed objects in the visible light images to obtain a base image of the target area. Next, it acquires thermal images of suspicious targets using a thermal imaging camera. Based on preset human proportions, it marks judgment points in the thermal images to obtain a frame of the suspicious target. Then, based on a human body shape database, it performs body shape matching and contour application on the frame of the suspicious target to obtain the human body contour corresponding to the suspicious target. This human body contour is projected onto the base image to obtain the target image. Finally, it detects the behavioral characteristics of the suspicious target in the target area based on the target image.

[0027] As described above, this application pre-captures and extracts fixed object contours using a visible light camera to construct a simplified base image of the target area, reducing the need for real-time full-pixel analysis of visible light images. It also utilizes thermal imaging images combined with preset human body proportions to quickly calibrate key human body points and generate a framework, replacing complex real-time human body key point detection algorithms. Furthermore, contour matching and application are performed based on a pre-built human body shape library to directly obtain the real human body contour, eliminating the cumbersome calculation of extracting contours pixel-by-pixel from thermal imaging. Finally, the human body contour is projected onto the base image, and dangerous behavior is quickly determined through the geometric relationships between the contours. This application, by differentially processing the core information of both visible light and thermal imaging modalities, simplifying the target framework construction process, and focusing on key contour analysis, significantly reduces real-time computation while ensuring environmental adaptability and improving the speed and real-time response capability of suspicious target behavior identification in monitoring scenarios.

[0028] Furthermore, this application combines a visible light camera with a thermal imaging camera. The visible light camera acquires a basic image of the target area and simplifies the lines of dangerous, fixed objects in the image. For suspicious targets obtained by the thermal imaging camera, frame extraction and human contour overlay are performed to quickly determine the approximate range and shape of the target. This avoids complex target segmentation and feature extraction processes, simplifies the actual target recognition process, and improves the overall response speed. By combining a visible light camera and a thermal imaging camera, this application enables the system to operate normally under different lighting, weather, and environmental conditions, improving overall stability. Simultaneously, it performs anomaly identification and judgment on the contours of suspicious targets and dangerous objects, simplifying the calculation process and avoiding complex comprehensive analysis of the entire image. By focusing on key contour information, the computational load and data processing volume are reduced, thus solving the technical problem of insufficient real-time performance in the recognition of behavioral features of suspicious targets in existing technologies. Attached Figure Description

[0029] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0030] Figure 1 This is a schematic diagram of an optional suspicious target detection method based on visible light images and thermal imaging according to an embodiment of this application;

[0031] Figure 2 This is a schematic diagram of an optional suspicious target detection device based on visible light images and thermal imaging, according to an embodiment of this application. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] According to an embodiment of this application, a method embodiment for detecting suspicious targets based on visible light images and thermal imaging is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0035] It should be noted that the information collected in this application (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of this data all comply with relevant laws, regulations, and standards, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding access points are provided for users to choose to authorize or refuse. For example, interfaces are set up between this system and relevant users or organizations, providing users with corresponding access points to choose to agree to or refuse automated decision-making results; if the user chooses to refuse, the process proceeds to the expert decision-making stage.

[0036] According to the embodiments of this application, a target recognition system can be used as the execution subject of the suspicious target detection method based on visible light images and thermal imaging in the embodiments of this application. The system can be a software system or an embedded system combining software and hardware. Of course, the execution subject of the method in the embodiments of this application can also be other forms of execution subject, such as devices, equipment, etc. It should be known by those skilled in the art that this application does not particularly limit the specific form of the execution subject.

[0037] Figure 1 This is a suspicious target detection method based on visible light images and thermal imaging according to embodiments of this application, such as... Figure 1 As shown, the method includes the following steps:

[0038] Step S101: The target area is captured N times by a visible light camera to obtain N visible light images. Then, the outline of the fixed object in the visible light images is drawn to obtain the basic image of the target area, where N is an integer greater than or equal to 1.

[0039] Optionally, a visible light camera refers to an imaging device operating within the visible spectrum range, used to acquire color or grayscale images of a target area. N image acquisitions represent multiple shots of the same target area; for example, 10 shots could be taken in this embodiment to improve the stability of background modeling. Fixed objects refer to objects in a scene whose position remains unchanged, such as walls, railings, and equipment. Contour delineation refers to the process of extracting object edges using image processing techniques, simplifying lines for fixed objects and region contours within the target area. For example, a Gaussian mixture model can be used to model the background from multiple grayscale images, distinguishing foreground and background pixels. Then, edge detection can be used to extract the contours of fixed objects. Finally, the Douglas-Puk algorithm can be used for contour simplification and polygon fitting to generate a basic image containing only key lines.

[0040] By constructing a static environment model of the target area, coordinate references are provided for subsequent behavioral analysis of suspicious targets. By acquiring visible light images multiple times and extracting the contours of fixed objects, dynamic interference can be effectively filtered out, and simplified line drawings containing only dangerous fixed objects such as fences and machine edges can be constructed. This reduces the complexity of subsequent image processing, minimizes real-time full-pixel analysis, and helps improve the system's environmental adaptability and processing efficiency in complex monitoring scenarios.

[0041] Step S102: Use a thermal imaging camera to acquire thermal imaging images of the suspicious target, and then mark the judgment points in the thermal imaging image based on the preset human body proportions to obtain the frame of the suspicious target.

[0042] Optionally, a thermal imaging camera refers to a device that generates a thermal distribution image by detecting infrared radiation, suitable for human detection at night or in low-light conditions. Preset human proportions refer to the proportional relationship between the length of various body parts and height, determined based on human anatomical standards; for example, the length of the upper arm is approximately 0.186 times the height.

[0043] Optionally, the determination points may include key parts of the human body, such as the center of the head, the beginning and ending points of the chest cavity, and the joints of the limbs. A suspected target frame refers to a simplified representation of the human skeleton formed by marking and connecting determination points in a thermal imaging image. For example, the scale can be determined first based on the actual size of a reference object in the image, and then the actual height of the suspected target can be calculated by combining the pixel height of the suspected target in the image, thereby deducing the position of each determination point.

[0044] This application embodiment rapidly extracts human structural information from thermal imaging images, enabling the construction of a basic framework for pose analysis. By combining a preset human body proportion with an image scale, the reliance on complex human keypoint detection algorithms is reduced, allowing the location of major joints and the torso in thermal imaging to be identified, thus improving the efficiency of human structure recognition in thermal imaging.

[0045] Step S103: Based on the human body shape library, perform body shape matching and contour application on the suspicious target frame to obtain the human body contour corresponding to the suspicious target.

[0046] Optionally, the human body shape database refers to a pre-built database containing human contour images and their feature vectors for various body types. The feature vectors can be composed of the midpoint contour distances of parts such as the thigh, upper arm, chest, and abdomen. Body shape matching refers to calculating the similarity between the feature vectors of the suspected target frame and the feature vectors in the human body shape database; for example, cosine similarity can be used. Contour application refers to mapping the most similar human contour from the human body shape database onto the suspected target frame after successful matching, forming a complete human contour. This reduces the computational cost of directly extracting complex contours from thermal imaging, thus improving the efficiency and accuracy of contour generation.

[0047] This application's embodiments can transform a simplified human body frame into an outline with a realistic human body shape, providing a more realistic geometric representation for subsequent spatial relationship analysis with dangerous objects. Through human body shape database matching, the target recognition system can adapt to targets of different body shapes, enhancing the realism of the outline and the reliability of behavior analysis. It reduces the need for pixel-by-pixel outline extraction from thermal imaging images, thus reducing the computational burden and ensuring the rationality and consistency of the outline.

[0048] Step S104: Project the human body outline corresponding to the suspicious target onto the base image to obtain the target image.

[0049] Optionally, projection refers to the process of transforming the human body contour from the image coordinate system in which the human body contour resides to the coordinate system of the base image. For example, the base image has a pre-established coordinate system consistent with the visible light image, and the contours are superimposed through coordinate mapping.

[0050] Optionally, the target image refers to a composite image that fuses the outline of a fixed object with the human outline corresponding to the suspected target, used for subsequent spatial relationship analysis. During projection, it is necessary to ensure that the thermal imaging camera and the visible light camera are positioned identically or calibrated to ensure accurate coordinate alignment.

[0051] The embodiments of this application are beneficial for realizing information fusion of multimodal images, unifying the human body contour extracted from thermal imaging and the environmental structure in visible light images into the same coordinate system, enabling the system to simultaneously analyze the spatial relationship between the human body and fixed objects in the same image, providing a unified visual basis for behavior determination, and reducing alignment errors and computational redundancy caused by processing human body contours and visible light images separately, which is beneficial for improving the overall synergy and analysis efficiency of the system.

[0052] Step S105: Detect the behavioral characteristics of the suspicious target in the target area based on the target image.

[0053] Optionally, behavioral features refer to the spatial relationship between a suspicious target and a dangerous fixed object, such as whether it approaches, contacts, or enters a dangerous area. For example, the shortest distance between the human body contour point set and the dangerous fixed object contour point set can be calculated to determine whether a preset danger distance threshold is exceeded. If it is exceeded, a dangerous behavior is determined to exist; simultaneously, ray casting can be used to determine whether the human body contour coincides with the dangerous object contour to further confirm the intrusion behavior. This application embodiment can focus on the geometric relationship between contours, reducing the need for complex behavioral recognition modeling of the entire image, and can also achieve automated identification and early warning of suspicious target behavior. By analyzing the distance and overlap relationship between the human body contour and the fixed object contour, the target recognition system can quickly determine whether dangerous behaviors such as climbing, approaching, or intruding exist.

[0054] This application pre-captures and extracts fixed object contours using a visible light camera to construct a simplified base image of the target area, reducing the need for real-time full-pixel analysis of visible light images. It utilizes thermal imaging images combined with preset human body proportions to quickly calibrate key human body points and generate a framework, replacing complex real-time human key point detection algorithms. Furthermore, contour matching and application are performed based on a pre-built human body shape library to directly obtain realistic human body contours, eliminating the cumbersome calculation of extracting contours pixel-by-pixel from thermal imaging. Finally, the human body contours are projected onto the base image, and dangerous behaviors are quickly determined through the geometric relationships between the contours. This application's embodiments differentiate the core information of both visible light and thermal imaging modalities, simplify the target framework construction process, and focus on key contour analysis. While ensuring environmental adaptability, it significantly reduces real-time computation and improves the speed and real-time response capability for identifying suspicious target behavior in monitoring scenarios.

[0055] In one optional embodiment, the target area is captured N times by a visible light camera to obtain N visible light images. This includes: the target recognition system can acquire parameter information of the target area, wherein the parameter information of the target area includes at least: the area of ​​the target area and the monitoring distance; and then, based on the parameter information of the target area, the target area is captured N times by a visible light camera to obtain N visible light images.

[0056] Optionally, the area of ​​the target region refers to the physical coverage of the area to be monitored, and the monitoring distance refers to the straight-line distance from the camera installation location to the farthest point of the target region. Based on the parameter information of the target region, a suitable model and specification of visible light camera can be selected. For example, for large-area outdoor scenes, a high-resolution, wide-angle camera can be selected, and the installation location, angle, and number of acquisitions N of the visible light camera can be determined to ensure that the acquired images can completely and clearly cover the target region.

[0057] This application's embodiments, by acquiring and basing key parameters such as the target area's area and monitoring distance, can scientifically guide camera selection and deployment, avoiding image loss, distortion, or insufficient resolution due to improper camera performance or placement. This enables the system to flexibly and efficiently acquire visible light images according to the specific needs of different monitoring scenarios, thereby ensuring the accuracy and completeness of environmental modeling from the source. For example, telephoto lenses can be selected for long-distance monitoring, while multi-camera stitching or wide-angle lenses can be used for large areas.

[0058] In one optional embodiment, contour delineation of a fixed object in a visible light image to obtain a base image of the target region includes: a target recognition system converting each acquired visible light image from a color space to a grayscale space, obtaining N grayscale images of the target region and assembling them into an image sequence. For the grayscale images in the image sequence, K Gaussian distributions for each pixel are initialized, and the initial mean, variance, and weight of each Gaussian distribution are set, where K is an integer greater than 1. For each pixel in the grayscale image, the grayscale value of that pixel is obtained. Based on the grayscale value of each pixel and the initial mean of each Gaussian distribution for that pixel, the matching degree between each pixel and each Gaussian distribution for that pixel is determined. Based on the matching degree between each pixel and each Gaussian distribution for that pixel, the contour delineation of the fixed object in the visible light image is performed to obtain a base image of the target region.

[0059] For example, based on the acquired base image, the lines of fixed objects and the outline of the target area are simplified and transformed, specifically including the following steps:

[0060] First, the acquired base image is converted to grayscale. Then, a Gaussian mixture model is used to identify fixed objects and region contours within the target area of ​​the base image. N target area images are extracted from the grayscale image, and for each pixel within the target area... There are N grayscale values ​​corresponding to N images: For each pixel Initialize K Gaussian distributions, and set an initial mean for each Gaussian distribution. ,variance and weight , .

[0061] It should be noted that K Gaussian distributions are initialized, where K ranges from 3 to 5, and the initial weights are... Initialize the mean It can be randomly initialized to the grayscale value of that pixel in a certain frame of the image. For example, the initial mean of each distribution can be set to the grayscale value of that pixel in the first frame. Variance Setting it to 100 allows you to obtain a grayscale image by calculating the grayscale value of each pixel in the image. The value of N needs to be set according to the actual situation, and can be selected as 10.

[0062] This application's embodiments employ a Gaussian mixture model to probabilistically model the grayscale value changes of each pixel. This allows the target recognition system to effectively distinguish between background fluctuations caused by lighting changes and minor shaking, and truly fixed objects. Converting the image to grayscale space simplifies calculations; initializing multiple Gaussian distributions better simulates complex backgrounds. Through matching degree judgment, fixed object regions can be stably identified, effectively suppressing dynamic noise in the scene and accurately separating the background from the foreground, thus helping to ensure the accuracy and stability of the environmental contours in the base image.

[0063] In one optional embodiment, the matching degree between each pixel and each Gaussian distribution of the pixel is determined based on the gray value of each pixel and the initial mean of each Gaussian distribution of the pixel, including: calculating the difference between each pixel and the initial mean of the i-th Gaussian distribution of the pixel, and calculating the absolute value of the difference to obtain the matching degree between the pixel and the i-th Gaussian distribution of the pixel, where i is a positive integer less than or equal to K.

[0064] Optionally, for a pixel and its i-th Gaussian distribution, the matching degree is determined as follows: calculate the difference between the current grayscale value of the pixel and the initial mean of the Gaussian distribution, and then take the absolute value of the difference. This absolute value is defined as the matching degree between the pixel and the Gaussian distribution. The smaller the absolute value, the closer the current grayscale value is to the background state modeled by the Gaussian distribution, and the higher the matching degree.

[0065] For example, for pixels within the target region in a newly acquired grayscale image, the grayscale value is... Calculate the matching degree between the current pixel and each Gaussian distribution, when If the gray value matches the i-th Gaussian distribution, then the parameters are updated.

[0066] This application uses absolute difference as the matching degree, which reduces complex probability calculations and computational overhead. If the gray value of the current pixel is very close to the mean of a Gaussian distribution, the current pixel is likely to belong to the background pattern represented by that distribution. This direct distance metric provides a clear numerical basis for subsequent judgments of whether a pixel belongs to the background or foreground, making the matching judgment process in the Gaussian mixture model simpler and faster.

[0067] In one optional embodiment, a basic image of the target region is obtained by outlining a fixed object in a visible light image based on the matching degree of each pixel with each Gaussian distribution of that pixel. This includes: the target recognition system detecting whether the matching degree of each pixel with the i-th Gaussian distribution of that pixel is greater than a target threshold, and obtaining a detection result, wherein the target threshold is determined based on the variance of the i-th Gaussian distribution of that pixel. Then, the basic image of the target region is obtained by outlining the fixed object in the visible light image based on the detection result.

[0068] Optionally, the target threshold is not a fixed value, but is dynamically determined based on the variance of the Gaussian distribution. For example, the target threshold can be set to a multiple of the variance, such as 2.5 times the standard deviation. The detection results are used to characterize whether the match is successful or not. Based on the detection results of all pixels, the target recognition system completes the contour delineation of the fixed object and obtains the basic image.

[0069] This application's embodiments compare the matching degree with a target threshold determined by variance. The target recognition system can flexibly adjust the tightness of the judgment based on the inherent uncertainty of each Gaussian distribution. Distributions with large variance can represent background areas with significant changes, such as swaying grass, and the corresponding target threshold can be larger. This allows for greater grayscale variations to still be considered background, enhancing the model's adaptability to complex backgrounds and facilitating adaptive foreground and background segmentation.

[0070] In one optional embodiment, detecting whether the matching degree of each pixel with the i-th Gaussian distribution of that pixel is greater than a target threshold, and obtaining a detection result, includes: if the target recognition system detects that the matching degree of any pixel with the i-th Gaussian distribution of that pixel is greater than the target threshold, then it determines that the matching degree of the pixel with the i-th Gaussian distribution of that pixel has failed; if it detects that the matching degree of any pixel with the i-th Gaussian distribution of that pixel is less than or equal to the target threshold, then it determines that the matching degree of the pixel with the i-th Gaussian distribution of that pixel has succeeded.

[0071] Optionally, if the matching degree of a pixel with a certain Gaussian distribution is greater than the target threshold, then the pixel is determined to have failed to match the Gaussian distribution, indicating that the current pixel value is unlikely to be generated by that distribution. Conversely, if the matching degree is less than or equal to the target threshold, then the match is determined to be successful, indicating that the current pixel value conforms to the expected range of the distribution.

[0072] This application's embodiments transform continuous matching degree values ​​into explicit success or failure states, providing direct logical input for subsequent background model construction and background / foreground pixel differentiation. Using explicit judgment criteria enables the Gaussian mixture model to effectively distinguish between static backgrounds and dynamic targets, which is beneficial for providing clear, binary decision rules for background modeling.

[0073] In one optional embodiment, the contours of fixed objects in a visible light image are delineated based on the detection results to obtain a base image of the target region. This includes: the target recognition system calculates the ratio of the weight to the variance of each Gaussian distribution to obtain the target value for each Gaussian distribution. Furthermore, based on the target value of each Gaussian distribution, the K Gaussian distributions for each pixel are sorted in descending order, and the top B Gaussian distributions are selected as the background model for that pixel, where B needs to satisfy... T is the background threshold; then, based on the detection results, if any Gaussian distribution in the background model of each pixel successfully matches the pixel, the pixel is determined to be a background pixel; if every Gaussian distribution in the background model of each pixel fails to match the pixel, the pixel is determined to be a foreground pixel; finally, the outline of the fixed object in the visible light image is drawn based on the background pixels and the foreground pixels to obtain the basic image of the target area.

[0074] Alternatively, the Gaussian distribution can be arranged according to... Arrange the data in descending order and select the top B Gaussian distributions as the background model. B must satisfy the following conditions: , where T is the background threshold. If the gray value of the current pixel matches any Gaussian distribution in the background model, the pixel is classified as background; otherwise, the pixel is classified as foreground.

[0075] It should be noted that the background threshold T ranges from 0.7 to 0.9, and the specific threshold should be selected based on the actual situation. The specific steps for updating the parameters are as follows:

[0076] When a matching Gaussian distribution exists, and the matching Gaussian distribution is the j-th one, the weight update can refer to formula (1):

[0077] Formula (1);

[0078] in, The weights used to represent the j-th Gaussian distribution; Used to represent the learning rate, with a value between 0 and 1. The mean update can be referenced in formula (2):

[0079] Formula (2);

[0080] in, The mean of the j-th Gaussian distribution is used to represent the center position of the pixel grayscale value modeled by this distribution. Used to represent the grayscale observation value at pixel (x, y) in the current frame; Used to represent the effective learning rate for this distribution. Variance updates can be referenced using formula (3):

[0081] Formula (3);

[0082] in, The variance of the j-th Gaussian distribution is used to measure the range of fluctuation or uncertainty of the pixel grayscale values ​​modeled by this distribution. The square of the difference between the current observation and the distribution mean is used to reflect the degree of deviation of the current observation.

[0083] If no matching Gaussian distribution exists, select the Gaussian distribution with the smallest weights, replace the mean of that distribution with the gray value of the current pixel, initialize the variance to a large value, update the weights appropriately, and normalize the weights of all Gaussian distributions so that... .

[0084] This application's embodiments facilitate the construction of a multimodal background model and the achievement of accurate foreground segmentation. Distributions with high weight / variance ratios represent background patterns with high probability of occurrence and stability, and are preferentially selected into the background model. The background threshold T controls the total probability of background patterns covered by the background model, ensuring that the background is fully modeled. The logic that any successful match in the background model constitutes the background makes the model more inclusive of the background, effectively filtering out transient and minor perturbations, and extracting only pixels that consistently deviate from all background patterns (i.e., foreground). This application's embodiments, by integrating multiple factors such as weight, variance, and background threshold to construct and utilize the background model, facilitate adaptive foreground detection, effectively handle complex backgrounds such as gradual lighting changes and periodic motion, accurately separate the true fixed object contours, and greatly improve the accuracy of basic image generation and robustness to environmental changes.

[0085] In one optional embodiment, the basic image of the target region is obtained by outlining the contour of a fixed object in a visible light image based on background and foreground pixels. This includes: the target recognition system assigning a value of 255 to the detected foreground pixels and a value of 0 to the background pixels to generate a binary image; based on the binary image, using an edge detection algorithm to extract the contour of the fixed object within the target region; then swapping the background and foreground pixel values ​​in the binary image, and then using the edge detection algorithm again to extract the background edge contour to obtain the complete contour of the target region; approximating the extracted contour of the fixed object and the contour of the target region using the Douglas-Puk algorithm, then performing polygon fitting using the least squares method, and drawing the fitted polygon on a blank image to obtain the basic image of the target region.

[0086] Optionally, when performing edge detection, the binary image needs to be smoothed using a Gaussian filter to reduce the influence of noise. The gradient magnitude and direction are obtained by calculating the gradient of the image in the x and y directions. Non-maximum suppression is applied to the gradient magnitude in the gradient direction, retaining only the local maximum value to refine the edge. The edge is determined by double threshold detection.

[0087] Optionally, when performing contour approximation using the Douglas-Puk algorithm, all points on the extracted contour are used as the initial point set. The first point in the initial point set is selected as the starting point, and the last point is selected as the ending point. A straight line is formed connecting the starting and ending points. Then, the perpendicular distance from this line to all points in the initial point set except the starting and ending points is calculated. Using a pre-set distance threshold, the point farthest from the line is identified. If the distance from this point to the line is greater than the threshold, the point is retained, and the initial point set is divided into two parts using this point as the dividing point. The steps are repeated for each part of the initial point set. If the distance from this point to the line is less than or equal to... If the threshold is set, all points in the initial point set except for the starting and ending points are discarded. The above process is repeated until all points have been processed. The final point set is the approximate contour. In the polygon fitting process, the number of sides of the polygon to be fitted is predetermined according to the shape and requirements of the target area. The vertex coordinates of the polygon are randomly initialized. The sum of the squared distances from the points on the approximate contour to each side of the current polygon is calculated as an error metric. The vertex coordinates of the polygon are adjusted by the gradient descent method so that the error gradually decreases until the convergence condition is met. A coordinate system is established in the base image based on the collected base image, that is, the coordinates of each point in the base image have a one-to-one correspondence with the coordinates of each point in the base image.

[0088] This application's embodiments can transform foreground / background segmentation results into a simplified vector graphic representation that can be used for efficient spatial analysis. Generating a binary image prepares for contour extraction. Two edge detection steps obtain the object's inner edge and the region's outer edge, respectively. The Douglas-Puk algorithm can significantly reduce the number of contour points, thus reducing the data volume. Polygon fitting then regularizes and vectorizes the contours. The final base image is no longer a pixel array but a graphic composed of simple lines, which facilitates subsequent human contour projection and distance calculation, significantly reducing computational complexity.

[0089] In one optional embodiment, based on a preset human body proportion, judgment points are marked in the thermal imaging image to obtain a suspicious target frame. This includes: the target recognition system can determine a scale based on the camera parameters of the thermal imaging camera and the size of the reference object corresponding to the target area; the suspicious target is proportionally converted according to the scale to obtain the human height result of the suspicious target; and based on the human height result of the suspicious target, combined with the preset human body proportion, the judgment points of the suspicious target are marked, wherein the judgment points include the start, end, and midpoints of the forearm, upper arm, lower leg, thigh, chest cavity, waist and abdomen, and the center of the head; then, point recognition is performed on both sides of the contour of the same judgment point, and the midpoint connecting the recognized points on both sides is taken as the frame point; and all frame points are connected to obtain the suspicious target frame.

[0090] Optionally, the scale refers to the conversion relationship between image pixels and actual physical dimensions, and can be determined through camera parameters such as focal length and pixel size, and reference objects of known dimensions such as on-site calibration boards and door frames. The purpose of determining the scale is to establish a spatial mapping between the image and the real world, ensuring that all subsequent size calculations have physical meaning.

[0091] Optionally, the human height result refers to the physical height calculated by converting the pixel height of the target in the image using a scale. Judgment points can be key human body locations marked on the image based on the human height result and a preset human body proportion. For example, based on anthropometric data, the thigh length is set to approximately 0.245 times the height. The purpose of converting height and marking judgment points is to quickly and roughly locate key human body parts, avoiding complex image feature extraction. For example, the pixel coordinates of the top of the head and the bottom of the feet can be determined first through image processing such as finding the highest and lowest points of the temperature connectivity region, and the height H can be obtained by multiplying the pixel distance by the scale. Then, the theoretical positions of each judgment point in the image are calculated according to the preset proportion and marked.

[0092] Optionally, frame points refer to more precise location points obtained by taking the midpoint of feature points identified on both sides of the contour at each decision point. The purpose of identifying and connecting frame points is to correct the rough theoretical points to the central axis of the actual contour, making the frame fit the target more closely. For example, for each marked decision point, an operator can be used to detect edges within a rectangular area around it, finding the point with the greatest curvature on the edge as the two side boundaries, and taking the midpoint between these two points as the final frame point. All frame points can be connected with straight lines in the anatomical order of head-neck-torso-limbs to form a simplified human skeleton.

[0093] In one optional embodiment, determining the scale based on the camera parameters of the thermal imaging camera and the size of the reference object corresponding to the target area includes: the target recognition system can determine the proportional relationship between pixels at different positions in the thermal imaging image and the actual physical size based on the camera parameters of the thermal imaging camera and the actual size of the reference object in the thermal imaging image; and based on the proportional relationship, determine the scale according to the actual length of the reference object and the pixel length corresponding to the reference object in the thermal imaging image.

[0094] Optionally, images of the suspected target are acquired using a thermal imaging camera. Based on the known actual size of the object in the target area image, the proportional relationship between pixels at different locations in the image and the actual physical size is determined, and the actual length of the reference object is considered. Pixel length corresponding to the image Obtain the scale .

[0095] Optionally, the camera parameters of a thermal imaging camera may include internal parameters such as focal length, pixel size, and principal point coordinates, as well as external parameters such as camera height and pitch angle. The actual size of the reference object refers to an object in the monitoring scene whose size is known, easily identifiable, and whose position is fixed.

[0096] The embodiments of this application help to eliminate the effects of perspective distortion and image magnification, and achieve accurate mapping from two-dimensional images to three-dimensional physical space. This makes it easier to ensure that the basis for behavior determination has real physical meaning, rather than just the relative distance at the image pixel level.

[0097] In one optional embodiment, the suspected target is proportionally converted according to the scale to obtain the human height result of the suspected target, including: converting the pixel size into the actual physical size according to the number of pixels corresponding to the height of the suspected target in the thermal imaging image; obtaining the pixel distance between two points on the top of the suspected target's head and the bottom of its feet in the thermal imaging image; and calculating the product of the pixel distance and the scale to obtain the human height result of the suspected target.

[0098] Optionally, the pixel distance in the thermal imaging image between two points, the top of the suspected target's head and the bottom of its feet, can be used. ,pass × To obtain the actual height H of the suspected target, it should be noted that by obtaining the actual height of the suspected target, the location of the judgment point can be determined based on the human body proportion results. The pixel distance between the two points on the thermal imaging image of the suspected target's head and feet needs to be determined based on the pixels passed between the two points in the thermal imaging image.

[0099] Optionally, based on the standard proportional relationship of various parts of the human body, and according to the known height of the suspected target, the actual length and position of the forearm, upper arm, lower leg, thigh, chest cavity, waist and abdomen of various parts of the human body are calculated, and the start, end, midpoint and center of the head are marked in the image to obtain the judgment point.

[0100] It should be noted that by consulting human anatomy data, medical research literature, and relevant anthropometry standard data, the proportional coefficients between the height and the major parts of the human body, such as the forearm, upper arm, lower leg, thigh, chest cavity, waist and abdomen, are collected to obtain the standard proportional relationship of each part of the human body. Using the actual height of the suspected target obtained through proportional conversion, the actual length of each part is calculated by combining the proportional coefficients between each part and the height. The head is assumed to be round or elliptical. The boundary of the head is roughly estimated in the image, and then its geometric center is taken as the center point of the head. The starting point of the chest cavity can be approximated as starting from a certain distance downward from the top of the head. The ending point of the chest cavity is determined downward based on the converted actual length. The starting point of the waist and abdomen is immediately adjacent to the ending point of the chest cavity. The shoulder position is estimated as the starting point of the upper arm based on the width of the chest cavity and the human posture. The starting point of the forearm is the ending point of the upper arm. A certain distance below the waist and abdomen is the starting point of the thigh, and the starting point of the lower leg is the ending point of the thigh. The judgment points are marked based on the actual length of each part.

[0101] Optionally, the number of pixels corresponding to height refers to the total number of pixels occupied by the suspect in the vertical direction of the image, which is a measure of image space. The purpose of converting pixel size to actual physical size is to convert the pixel count, which has no physical unit, into a real length in meters or centimeters. The pixel distance between two points, the top of the head and the bottom of the feet, refers to the length of the line connecting the two specific points representing the top of the target's head and the bottom of the feet in the image coordinate system, which can be in pixels.

[0102] The embodiments of this application provide an accurate, physical-world benchmark for subsequent estimation of limb length and joint positions based on standard human proportions. Without a true physical height, relying solely on image pixel ratios will result in calculated limb lengths entirely dependent on the distance between the target and the camera, failing to reflect the target's true body shape. This leads to a distortion of the constructed suspicious target framework at a physical scale, consequently affecting the accuracy of behavioral analysis. For example, a child very close to the camera may appear pixel-wise taller than an adult at a distance in an image, but true physical height is the reliable basis for applying a general human proportion model.

[0103] In one optional embodiment, point identification is performed on both sides of the contour of the same determination point, and the midpoint connecting the identified points on both sides is used as the frame point. This includes: for each obtained determination point, setting a search range centered on each determination point; detecting an edge image based on the search range using an edge detection algorithm; tracking the contour in the edge image, obtaining the coordinates of each point on the contour, and calculating the curvature of each point on the contour, wherein the curvature is obtained by calculating the angle formed by three adjacent points; when the curvature of any point on the contour in the edge image is greater than the curvature threshold, the point is marked as a feature point; the coordinates of the feature points on both sides of each determination point are recorded respectively; the coordinate values ​​of the feature points on both sides are averaged to obtain the coordinate value of the midpoint connecting the points on both sides, and then the midpoint connecting the points is used as the frame point.

[0104] Optionally, the search range can refer to a local image region centered on each coarse determination point. For example, the search range can be a region with a pixel side length of [missing information]. A rectangle, pixel side length The specific image resolution and target size need to be considered during actual use, and it can be set between 10 and 50.

[0105] Optionally, contour tracking refers to extracting connected white pixels in an edge image into an ordered sequence of contour points. Curvature is used to quantify the degree of local curvature of the contour; curvature is represented by calculating the angle formed by three adjacent points, and specific steps may include:

[0106] For three adjacent points on the contour Calculate vector The angle between them is taken as a point. The curvature can be found in formula (4):

[0107] Formula (4);

[0108] in, Used to represent curvature, for each point on the contour. ,when This point is then marked as a feature point. For each decision point, the corresponding selected feature points are found on the contours on both sides. Used to represent the curvature threshold. It should be noted that the curvature threshold... It is necessary to observe the distribution of curvature on the contour and set the curvature threshold using the 3sigma principle.

[0109] Optionally, the coordinates of the feature points on both sides of the judgment point are recorded respectively. The coordinates of the frame point are then averaged according to the formula for calculating the midpoint coordinates. Feature points refer to contour points with curvature exceeding a certain threshold, typically corresponding to the boundary inflection points of limbs or torsos. The midpoint is the point obtained by taking the arithmetic mean of the coordinates of the feature points found on both sides of the contour at the judgment point.

[0110] In this embodiment, if the determination points are calculated solely based on height ratios, they may be theoretical positions that could deviate significantly due to changes in human posture such as raised arms, individual body size differences, and blurred thermal imaging contours. Directly using these determination points as skeletal nodes would result in severe frame distortion. Therefore, this embodiment utilizes the actual contour geometric features of the target in the thermal imaging image to absorb and correct these theoretical points, which helps to accurately place them on the physical center line of the human body parts, thereby constructing a realistic target frame.

[0111] In one optional embodiment, connecting all frame points to obtain a suspected target frame includes: using the coordinates of the frame points, first connecting the center point of the head to the midpoint of the chest cavity based on the human body sequence, then sequentially connecting the start and end points of the chest cavity from the midpoint of the chest cavity, then connecting the start, mid, and end points of the waist and abdomen, then connecting the start, mid, and end points of the upper arm and the forearm, and finally connecting the start, mid, and end points of the thigh and the lower leg to obtain a suspected target frame.

[0112] Optionally, since the human body's forearm, upper arm, lower leg, thigh, chest, and abdomen are all supported by bones, a suspicious target frame can be obtained by connecting the frame points with straight lines.

[0113] Optionally, discrete joint coordinates alone do not constitute a meaningful framework. Only by connecting discrete joints according to the correct physiological topology can a geometric model representing human posture, structure, and proportions be formed. This structured skeleton model is beneficial for inputting body shape matching and contour application. Without this connection step, subsequent algorithms will be unable to understand which points belong to the same limb, and it will be difficult to effectively match and analyze the overall body shape. The embodiments of this application, through a clear connection order, help ensure the consistency and repeatability of the generated framework, and can reduce framework ambiguity caused by unclear connection logic.

[0114] In one optional embodiment, based on a human body shape database, body shape matching and contour application are performed on the suspicious target frame to obtain the human body contour corresponding to the suspicious target. This includes: collecting contour images of people with different body shapes and establishing a human body shape database. For each collected human body contour image, the midpoint contour distances of the thigh, upper arm, chest, and waist are calculated respectively. The midpoint contour distances of the thigh, upper arm, chest, and waist corresponding to each human body contour image are used as a set of feature vectors and stored in the human body shape database. A judgment vector is established based on the midpoint contour distances of the thigh, upper arm, chest, and waist of the suspicious target. The cosine similarity between the judgment vector and the feature vectors in the human body shape database is calculated. The human body contour with the highest cosine similarity is selected from the human body shape database as the contour corresponding to the suspicious target.

[0115] Optionally, the distance between the midpoint contours of the thigh, upper arm, chest, and waist can be used as a similarity criterion to match and apply the human body model database to the current suspicious target frame.

[0116] Optionally, contour images of people of different body types can be collected and a human body type database can be established through a database system. The human images should include clear thighs, upper arms, chest, waist and abdomen, etc., which can be obtained from publicly available human image datasets, surveillance videos, and professional photographic works. Contour extraction of human images can be performed through edge detection algorithms.

[0117] Optionally, a judgment vector is established based on the midpoint contour distances of the thigh, upper arm, chest, and waist / abdomen of the current suspicious target contour, and the cosine similarity between the judgment vector and the feature vectors in the human body type database is calculated. The contour of the person in the human body model database with the highest cosine similarity is selected as the contour of the current suspicious target frame. These represent the decision vector and the eigenvector, respectively.

[0118] It should be noted that the closer the cosine similarity is to 1, the more similar the two feature vectors, the judgment vector and the feature vector, are. By performing human body shape library contour recognition and application on the suspicious target frame, the suspicious target frame can be constructed quickly, avoiding the complex edge extraction and contour tracking operations of each pixel in the image in the thermal imaging contour recognition process, thus improving the overall response speed. In the process of applying human body contours, the suspicious target frame is applied separately based on the forearm, upper arm, lower leg, thigh, chest, waist and abdomen and head in the human body shape library, so as to match the posture of the current suspicious target frame.

[0119] In one optional embodiment, detecting the behavioral characteristics of a suspicious target in a target region based on a target image includes: acquiring a first set of points on the human body contour corresponding to the suspicious target and a second set of points on the contour of a dangerous fixed object in the target image; for each point in the first set of points, calculating the distance from that point to each point in the second set of points, and then sorting all the calculated distances in ascending order to obtain the shortest distance from each point in the first set of points to the contour of the dangerous fixed object; determining a target point from the first set of points based on the shortest distance from each point in the first set of points to the dangerous fixed object, wherein the target point is the point in the first set of points closest to the contour of the dangerous fixed object; and determining that the suspicious target exhibits dangerous behavior in the target region when the target distance between the target point and the contour of the dangerous fixed object is greater than a dangerous distance threshold.

[0120] Optionally, the first point set refers to the set of coordinates of all vertices of the polygon representing the human silhouette of a suspicious target projected onto the base image. The second point set refers to the set of coordinates of all vertices of the polygon representing the silhouette of a fixed dangerous object, pre-drawn in the base image. Distance refers to the Euclidean distance between two points, and the shortest distance refers to the minimum distance among all points on the dangerous object silhouette for a specific point on the human silhouette, which can characterize the proximity of the human silhouette point to the dangerous object. The danger distance threshold refers to a physical distance value pre-set according to safety regulations, such as 0.5 meters. When the target distance exceeds this danger distance threshold, a dangerous behavior judgment is triggered.

[0121] In surveillance scenarios, excessive proximity is a direct precursor and core characteristic of dangerous behaviors such as intrusion, climbing, and touching. By calculating and monitoring the minimum geometrical interval between two contour polygons, target recognition systems can directly and quantitatively capture this excessive proximity feature without analyzing complex posture sequences or performing time-consuming pattern recognition, thus improving response speed.

[0122] In an optional embodiment, the suspicious target detection method based on visible light images and thermal imaging further includes: after determining that the suspicious target has dangerous behavior in the target area, taking each point in the first point set as the point to be judged, sequentially using the ray method to determine the overlap of the contours of the dangerous fixed objects, and marking the suspicious targets with overlap as dangerous.

[0123] Optionally, based on the coordinates of the suspicious target contour, the suspicious target contour is projected onto the base image, and anomaly identification is performed on the suspicious target based on the contour of a dangerous fixed object in the base image: based on the first set of points on the suspicious target contour. The second set of points on the outline of a dangerous fixed object For each point on the outline of the suspicious target ,calculate To all points on the outline of the dangerous fixed object The distance is calculated by sorting all distances in ascending order to obtain the point. To find the shortest distance to a dangerous fixed object, traverse all points on the suspicious target's outline and obtain the shortest distance from each point to the dangerous fixed object's outline. Take the minimum of all shortest distances as the target distance from the suspicious target's outline to the dangerous fixed object's outline. When the target distance is greater than the danger distance threshold, it indicates that the current suspicious target is exhibiting dangerous behavior, and the suspicious target is marked. Using each point in the suspicious target outline point set as a point to be judged, use the ray casting method to sequentially determine the overlap of the dangerous fixed object's outline and mark the suspicious targets that overlap as dangerous.

[0124] It should be noted that the outline of dangerous fixed objects needs to be pre-marked according to the actual situation of different target areas, and the danger distance threshold needs to be set specifically according to the actual situation of different dangerous fixed objects. By emitting a horizontal ray to the right from each point in the suspicious target outline point set, the number of intersections between this ray and the outline of the dangerous fixed object is counted. If the number of intersections is odd, the point is inside the outline of the dangerous fixed object; if the number of intersections is even, the point is outside the outline of the dangerous fixed object.

[0125] See Figure 2 According to another aspect of the embodiments of this application, a suspicious target detection device based on visible light images and thermal imaging is also provided, including: a first acquisition unit 201, a second acquisition unit 202, a first processing unit 203, a second processing unit 204, and a behavior feature detection unit 205.

[0126] The system comprises the following components: a first acquisition unit 201, which acquires N images of the target area using a visible light camera to obtain N visible light images, and then delineates the outline of a fixed object in the visible light images to obtain a base image of the target area, where N is an integer greater than or equal to 1; a second acquisition unit 202, which acquires thermal imaging images of the suspicious target using a thermal imaging camera, and then marks judgment points in the thermal imaging image based on a preset human body proportion to obtain a frame of the suspicious target; a first processing unit 203, which performs body shape matching and outline application on the frame of the suspicious target based on a human body shape library to obtain a human body outline corresponding to the suspicious target; a second processing unit 204, which projects the human body outline corresponding to the suspicious target onto the base image to obtain a target image; and a behavior feature detection unit 205, which detects the behavior features of the suspicious target in the target area based on the target image.

[0127] Optionally, the first acquisition unit 201 includes: a parameter information acquisition subunit, used to acquire parameter information of the target area, wherein the parameter information of the target area includes at least: the area of ​​the target area and the monitoring distance; and an image acquisition subunit, used to acquire N images of the target area using a visible light camera based on the parameter information of the target area, thereby obtaining N visible light images.

[0128] Optionally, the first acquisition unit 201 includes: an image processing subunit, used to convert each acquired visible light image from color space to grayscale space, obtain N grayscale images of the target area, and form an image sequence; a parameter processing subunit, used to initialize K Gaussian distributions for each pixel in the grayscale images of the image sequence, and set the initial mean, variance, and weight of each Gaussian distribution, where K is an integer greater than 1; a grayscale value acquisition subunit, used to acquire the grayscale value of each pixel in the grayscale image; a matching degree determination subunit, used to determine the matching degree between each pixel and each Gaussian distribution of the pixel based on the grayscale value of each pixel and the initial mean of each Gaussian distribution of the pixel; and a base image determination subunit, used to delineate the contour of a fixed object in the visible light image based on the matching degree between each pixel and each Gaussian distribution of the pixel, to obtain the base image of the target area.

[0129] Optionally, the matching degree determination subunit includes: a data calculation module, used to calculate the difference between each pixel and the initial mean of the i-th Gaussian distribution of the pixel, and calculate the absolute value of the difference to obtain the matching degree between the pixel and the i-th Gaussian distribution of the pixel, where i is a positive integer less than or equal to K.

[0130] Optionally, the base image determination subunit includes: a detection result determination module, used to detect whether the matching degree between each pixel and the i-th Gaussian distribution of that pixel is greater than a target threshold, and obtain a detection result. The target threshold is determined based on the variance of the i-th Gaussian distribution of that pixel; and a contour delineation processing module, used to delineate the contours of a fixed object in the visible light image based on the detection result, and obtain a base image of the target region.

[0131] Optionally, the detection result determination module includes: a first detection submodule, configured to determine that the matching degree between any pixel and the i-th Gaussian distribution of the pixel fails if the matching degree is greater than a target threshold; and a second detection submodule, configured to determine that the matching degree between any pixel and the i-th Gaussian distribution of the pixel is successful if the matching degree is less than or equal to the target threshold.

[0132] Optionally, the contour delineation processing module includes: a target value determination submodule, used to calculate the ratio of the weight and variance of each Gaussian distribution to obtain the target value of each Gaussian distribution; and an arrangement processing submodule, used to sort the K Gaussian distributions of each pixel in descending order according to the target value of each Gaussian distribution, and then select the top B Gaussian distributions as the background model of that pixel, where B needs to satisfy... T is the background threshold; the matching processing submodule is used to determine a pixel as a background pixel if any Gaussian distribution in the background model of each pixel matches the pixel successfully, and to determine a pixel as a foreground pixel if every Gaussian distribution in the background model of each pixel fails to match the pixel successfully; the contour drawing submodule is used to draw the contour of a fixed object in the visible light image based on the background pixels and foreground pixels to obtain the base image of the target area.

[0133] Optionally, the contour delineation submodule includes: a binary image determination submodule, used to assign the detected foreground pixels a value of 255 and the background pixels a value of 0 to generate a binary image; a contour extraction submodule, used to extract the contour of a fixed object within the target area based on the binary image using an edge detection algorithm; a complete contour determination submodule, used to swap the background pixel values ​​and foreground pixel values ​​in the binary image, and then use the edge detection algorithm again to extract the background edge contour to obtain the complete contour of the target area; and a base image determination submodule, used to approximate the extracted contour of the fixed object and the contour of the target area using the Douglas-Puk algorithm, then use the least squares method to perform polygon fitting, and draw the fitted polygon on a blank image to obtain the base image of the target area.

[0134] Optionally, the second acquisition unit 202 includes: a scale determination subunit, used to determine the scale based on the camera parameters of the thermal imaging camera and the size of the reference object corresponding to the target area; a human height result determination subunit, used to perform proportional conversion on the suspicious target according to the scale to obtain the human height result of the suspicious target; a marking processing subunit, used to mark the judgment points of the suspicious target according to the human height result of the suspicious target and in combination with the preset human proportion, wherein the judgment points include the start, end, and midpoint of the forearm, upper arm, lower leg, thigh, chest cavity, waist and abdomen, and the center of the head; a frame point determination subunit, used to identify the points on both sides of the contour of the same judgment point and take the midpoint connecting the identified points on both sides as the frame point; and a frame point connection subunit, used to connect all the frame points to obtain the frame of the suspicious target.

[0135] Optionally, the scale determination subunit includes: a scale relationship determination module, used to determine the scale relationship between pixels at different positions in the thermal imaging image and the actual physical size based on the camera parameters of the thermal imaging camera and the actual size of the reference object in the thermal imaging image; and a scale determination module, used to determine the scale based on the scale relationship and according to the actual length of the reference object and the pixel length corresponding to the reference object in the thermal imaging image.

[0136] Optionally, the human height result determination subunit includes: a size conversion module, used to convert the pixel size into the actual physical size based on the number of pixels corresponding to the height of the suspicious target in the thermal imaging image; a pixel distance acquisition module, used to acquire the pixel distance between two points on the top of the head and the bottom of the feet of the suspicious target in the thermal imaging image; and a human height result determination module, used to calculate the product of the pixel distance and the scale to obtain the human height result of the suspicious target.

[0137] Optionally, the frame point determination subunit includes: a search range setting module, used to set a search range centered on each obtained decision point; an edge image determination module, used to detect an edge image based on the search range using an edge detection algorithm; a curvature determination module, used to track the contour in the edge image, obtain the coordinates of each point on the contour, and calculate the curvature of each point on the contour, wherein the curvature is obtained by calculating the angle formed by three adjacent points; a feature point determination module, used to mark the point as a feature point when the curvature of any point on the contour in the edge image is greater than the curvature threshold; a coordinate recording processing module, used to record the coordinates of the feature points on both sides of each decision point; and a frame point determination module, used to average the coordinate values ​​of the feature points on both sides to obtain the coordinate value of the midpoint connecting the two points, and then use the midpoint as the frame point.

[0138] Optionally, the frame point connection subunit includes: a connection processing module, which uses the coordinates of the frame points to first connect the center point of the head to the midpoint of the chest cavity based on the human body sequence, then connect the starting point and ending point of the chest cavity sequentially from the midpoint of the chest cavity, then connect the starting point, midpoint and ending point of the waist and abdomen, then connect the starting point, midpoint and ending point of the upper arm and the starting point, midpoint and ending point of the forearm, and finally connect the starting point, midpoint and ending point of the thigh and the starting point, midpoint and ending point of the lower leg to obtain the frame of the suspected target.

[0139] Optionally, the first processing unit 203 includes: a human body shape database determination subunit, used to collect contour images of people with different body shapes and establish a human body shape database, wherein, for each collected human body contour image, the midpoint contour distances of the thigh, upper arm, chest, and waist are calculated respectively, and the midpoint contour distances of the thigh, upper arm, chest, and waist corresponding to each human body contour image are used as a set of feature vectors and stored in the human body shape database; a judgment vector determination subunit, used to establish a judgment vector based on the midpoint contour distances of the thigh, upper arm, chest, and waist of the suspicious target; a cosine similarity determination subunit, used to calculate the cosine similarity between the judgment vector and the feature vectors in the human body shape database; and a contour determination subunit, used to select the human body contour with the largest cosine similarity from the human body shape database as the contour corresponding to the suspicious target.

[0140] Optionally, the behavior feature detection unit 205 includes: a point set acquisition subunit, used to acquire a first point set on the human body contour corresponding to the suspicious target and a second point set on the contour of the dangerous fixed object in the target image; a distance determination subunit, used to calculate the distance from each point in the first point set to each point in the second point set, and then sort all the calculated distances in ascending order to obtain the shortest distance from each point in the first point set to the contour of the dangerous fixed object; a target point determination subunit, used to determine a target point from the first point set based on the shortest distance from each point in the first point set to the dangerous fixed object, wherein the target point is the point in the first point set that is closest to the contour of the dangerous fixed object; and a dangerous behavior determination subunit, used to determine that the suspicious target has dangerous behavior in the target area when the target distance between the target point and the contour of the dangerous fixed object is greater than a dangerous distance threshold.

[0141] Optionally, the suspicious target detection device based on visible light images and thermal imaging further includes: a hazard marking processing unit, which, after determining that a suspicious target has dangerous behavior in the target area, uses each point in the first point set as a point to be judged, sequentially uses the ray method to determine the overlap of the outline of the dangerous fixed object, and marks the suspicious target with overlap as hazard.

[0142] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, which stores a computer program, wherein when the computer program is executed, the device on which the computer-readable storage medium is located performs the above-described suspicious target detection method based on visible light images and thermal imaging.

[0143] According to another aspect of the embodiments of this application, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to perform the above-described suspicious target detection method based on visible light images and thermal imaging.

[0144] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program or instructions that, when executed by a processor, implement the above-described method for detecting suspicious targets based on visible light images and thermal imaging.

[0145] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0146] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0147] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0148] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0149] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0150] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0151] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for detecting suspicious targets based on visible light images and thermal imaging, characterized in that, include: The target area is captured N times by a visible light camera to obtain N visible light images. Then, the outline of the fixed object in the visible light images is drawn to obtain the basic image of the target area, where N is an integer greater than or equal to 1. Thermal imaging images of suspicious targets are acquired using a thermal imaging camera. Then, based on a preset human body proportion, judgment points are marked in the thermal imaging images to obtain a frame of the suspicious target. The judgment points include key parts of the human body, and the frame of the suspicious target is a human skeleton formed by marking and connecting the judgment points in the thermal imaging images. Based on a human body shape database, body shape matching and contour application are performed on the suspected target frame to obtain the human body contour corresponding to the suspected target. This includes: collecting contour images of people with different body shapes and establishing a human body shape database; for each collected human body contour image, calculating the midpoint contour distances of the thigh, upper arm, chest, and waist / abdomen; storing the midpoint contour distances of the thigh, upper arm, chest, and waist / abdomen corresponding to each human body contour image as a set of feature vectors in the human body shape database; establishing a judgment vector based on the midpoint contour distances of the thigh, upper arm, chest, and waist / abdomen of the suspected target; calculating the cosine similarity between the judgment vector and the feature vectors in the human body shape database; and selecting the human body contour with the highest cosine similarity from the human body shape database as the contour corresponding to the suspected target. The human silhouette corresponding to the suspected target is projected onto the base image to obtain the target image; The behavioral characteristics of the suspicious target in the target area are detected based on the target image.

2. The suspicious target detection method based on visible light images and thermal imaging according to claim 1, characterized in that, The target area is captured N times using a visible light camera, resulting in N visible light images, including: Obtain parameter information of the target area, wherein the parameter information of the target area includes at least: the area of ​​the target area and the monitoring distance; Based on the parameter information of the target area, the target area is captured N times by the visible light camera to obtain N visible light images.

3. The suspicious target detection method based on visible light images and thermal imaging according to claim 1, characterized in that, The outline of a fixed object in the visible light image is delineated to obtain a base image of the target region, including: Each of the acquired visible light images is converted from color space to grayscale space to obtain N grayscale images of the target area, which are then combined into an image sequence. For the grayscale image in the image sequence, initialize K Gaussian distributions for each pixel, and set the initial mean, variance, and weight of each Gaussian distribution, where K is an integer greater than 1; For each pixel in the grayscale image, obtain the grayscale value of that pixel; Based on the gray value of each pixel and the initial mean of each Gaussian distribution of that pixel, determine the matching degree between each pixel and each Gaussian distribution of that pixel; Based on the matching degree between each pixel and each Gaussian distribution of that pixel, the outline of the fixed object in the visible light image is drawn to obtain the base image of the target area.

4. The suspicious target detection method based on visible light images and thermal imaging according to claim 3, characterized in that, Based on the grayscale value of each pixel and the initial mean of each Gaussian distribution of that pixel, the matching degree between each pixel and each Gaussian distribution of that pixel is determined, including: Calculate the difference between each pixel and the initial mean of the i-th Gaussian distribution of that pixel, and calculate the absolute value of the difference to obtain the matching degree between the pixel and the i-th Gaussian distribution of that pixel, where i is a positive integer less than or equal to K.

5. The suspicious target detection method based on visible light images and thermal imaging according to claim 4, characterized in that, Based on the matching degree between each pixel and each Gaussian distribution of that pixel, the outline of a fixed object in the visible light image is delineated to obtain the base image of the target region, including: The detection result is obtained by detecting whether the matching degree between each pixel and the i-th Gaussian distribution of that pixel is greater than the target threshold, wherein the target threshold is determined according to the variance of the i-th Gaussian distribution of that pixel; Based on the detection results, the outline of the fixed object in the visible light image is drawn to obtain the basic image of the target area.

6. The suspicious target detection method based on visible light images and thermal imaging according to claim 5, characterized in that, The detection results are obtained by checking whether the matching degree between each pixel and the i-th Gaussian distribution of that pixel is greater than the target threshold, including: If the matching degree between any pixel and the i-th Gaussian distribution of that pixel is greater than the target threshold, then it is determined that the matching degree between that pixel and the i-th Gaussian distribution of that pixel has failed. If the matching degree of any pixel with the i-th Gaussian distribution of that pixel is less than or equal to the target threshold, then it is determined that the pixel has successfully matched the i-th Gaussian distribution of that pixel.

7. The suspicious target detection method based on visible light images and thermal imaging according to claim 6, characterized in that, Based on the detection results, the contours of fixed objects in the visible light image are delineated to obtain a basic image of the target region, including: Calculate the ratio of weight to variance for each Gaussian distribution to obtain the target value for each Gaussian distribution; Based on the target value of each Gaussian distribution, the K Gaussian distributions for each pixel are sorted in descending order, and then the top B Gaussian distributions are selected as the background model for that pixel, where B needs to satisfy... T is the background threshold. This represents the weight of the i-th Gaussian distribution; Based on the detection results, if any Gaussian distribution in the background model of each pixel successfully matches the pixel, then the pixel is determined to be a background pixel; if every Gaussian distribution in the background model of each pixel fails to match the pixel, then the pixel is determined to be a foreground pixel. Based on the background pixels and the foreground pixels, the outline of a fixed object in the visible light image is drawn to obtain the base image of the target area.

8. The suspicious target detection method based on visible light images and thermal imaging according to claim 7, characterized in that, Based on the background pixels and the foreground pixels, the outline of a fixed object in the visible light image is drawn to obtain a basic image of the target region, including: The detected foreground pixels are assigned a value of 255, and the background pixels are assigned a value of 0 to generate a binary image. Based on the binary image, an edge detection algorithm is used to extract the contours of fixed objects within the target area; The background pixel values ​​and foreground pixel values ​​in the binary image are swapped, and then the edge detection algorithm is used again to extract the background edge contour to obtain the complete contour of the target region. The contours of the extracted fixed object and the target region are approximated using the Douglas-Puk algorithm. Then, the least squares method is used for polygon fitting, and the fitted polygons are drawn on a blank image to obtain the base image of the target region.

9. The suspicious target detection method based on visible light images and thermal imaging according to claim 1, characterized in that, Based on a preset human body proportion, judgment points are marked in the thermal imaging image to obtain a suspicious target frame, including: The scale is determined based on the camera parameters of the thermal imaging camera and the size of the reference object corresponding to the target area; The suspected target's height is calculated by proportionally converting the scale. Based on the height of the suspected target and in conjunction with the preset human body proportions, the judgment points of the suspected target are marked. The judgment points include the beginning, end, and midpoint of the forearm, upper arm, lower leg, thigh, chest cavity, waist and abdomen, and the center of the head. Points are identified on both sides of the contour of the same judgment point, and the midpoint connecting the identified points on both sides is taken as the frame point. Connect all the frame points to obtain the suspected target frame.

10. The suspicious target detection method based on visible light images and thermal imaging according to claim 9, characterized in that, Based on the camera parameters of the thermal imaging camera and the size of the reference object corresponding to the target area, the scale is determined, including: Based on the camera parameters of the thermal imaging camera and the actual size of the reference object in the thermal imaging image, the proportional relationship between pixels at different positions in the thermal imaging image and the actual physical size is determined; Based on the aforementioned proportional relationship, the scale is determined according to the actual length of the reference object and the pixel length of the reference object in the thermal imaging image.

11. The suspicious target detection method based on visible light images and thermal imaging according to claim 9, characterized in that, Based on the aforementioned scale, the suspected target's height is calculated to obtain the target's height, including: Based on the number of pixels corresponding to the height of the suspected target in the thermal imaging image, the pixel size is converted into the actual physical size; Obtain the pixel distance between two points on the top of the head and the bottom of the feet of the suspected target in the thermal imaging image; The product of the pixel distance and the scale is calculated to obtain the height of the suspected target.

12. The suspicious target detection method based on visible light images and thermal imaging according to claim 9, characterized in that, For the same judgment point, point identification is performed on both sides of the contour, and the midpoint connecting the identified points on both sides is taken as the frame point, including: For each judgment point obtained, set the search range centered on each judgment point; An edge image is obtained by using an edge detection algorithm based on the search range; The contour in the edge image is tracked, the coordinates of each point on the contour are obtained, and the curvature of each point on the contour is calculated. The curvature is obtained by calculating the included angle formed by three adjacent points. When the curvature of any point on the contour in the edge image is greater than a curvature threshold, that point is marked as a feature point. Record the coordinates of the feature points on both sides of each determination point; The coordinate values ​​of the feature points on both sides are averaged to obtain the coordinate value of the midpoint connecting the two points, and then the midpoint connecting the two points is used as the frame point.

13. The suspicious target detection method based on visible light images and thermal imaging according to claim 9, characterized in that, Connecting all the frame points yields the suspected target frame, including: Using the coordinates of the frame points, the center point of the head is first connected to the midpoint of the chest cavity based on the human body sequence. Then, the starting point and ending point of the chest cavity are connected sequentially from the midpoint of the chest cavity. Next, the starting point, midpoint, and ending point of the waist and abdomen are connected. Then, the starting point, midpoint, and ending point of the upper arm and the starting point, midpoint, and ending point of the forearm are connected. Finally, the starting point, midpoint, and ending point of the thigh and the starting point, midpoint, and ending point of the lower leg are connected to obtain the frame of the suspected target.

14. The suspicious target detection method based on visible light images and thermal imaging according to claim 1, characterized in that, Detecting the behavioral characteristics of the suspicious target in the target region based on the target image includes: In the target image, obtain a first set of points on the human body contour corresponding to the suspicious target and a second set of points on the contour of the dangerous fixed object; For each point in the first point set, calculate the distance from that point to each point in the second point set, and then sort all the calculated distances in ascending order to obtain the shortest distance from each point in the first point set to the outline of the dangerous fixed object. Based on the shortest distance from each point in the first point set to the dangerous fixed object, a target point is determined from the first point set, wherein the target point is the point in the first point set that is closest to the outline of the dangerous fixed object; When the target distance between the target point and the outline of the dangerous fixed object is detected to be greater than the danger distance threshold, it is determined that the suspicious target is engaging in dangerous behavior in the target area.

15. The suspicious target detection method based on visible light images and thermal imaging according to claim 14, characterized in that, The suspicious target detection method based on visible light images and thermal imaging also includes: After determining that the suspicious target is engaging in dangerous behavior in the target area, each point in the first set of points is used as a point to be judged, and the ray method is used sequentially to determine the overlap of the outline of the dangerous fixed object, and the suspicious targets with overlap are marked as dangerous.

16. A suspicious target detection device based on visible light images and thermal imaging, characterized in that, include: The first acquisition unit is used to acquire N images of the target area through a visible light camera to obtain N visible light images, and then to delineate the outline of a fixed object in the visible light images to obtain a basic image of the target area, where N is an integer greater than or equal to 1. The second acquisition unit is used to acquire thermal imaging images of suspicious targets using a thermal imaging camera, and then mark judgment points in the thermal imaging image based on a preset human body proportion to obtain a suspicious target frame. The judgment points include key parts of the human body, and the suspicious target frame is a human skeleton formed by marking the judgment points in the thermal imaging image and connecting them. The first processing unit is used to perform body shape matching and contour application on the suspicious target frame based on the human body shape library to obtain the human body contour corresponding to the suspicious target. The second processing unit is used to project the human body outline corresponding to the suspicious target onto the base image to obtain the target image; A behavior feature detection unit is used to detect the behavior features of the suspicious target in the target area based on the target image; The first processing unit includes: a human body shape database determination subunit, used to collect contour images of people with different body shapes and establish a human body shape database, wherein for each collected human body contour image, the midpoint contour distance of the thigh, upper arm, chest, and waist is calculated respectively, and the midpoint contour distance of the thigh, upper arm, chest, and waist corresponding to each human body contour image is used as a set of feature vectors and stored in the human body shape database; a judgment vector determination subunit, used to establish a judgment vector based on the midpoint contour distance of the thigh, upper arm, chest, and waist of the suspected target; a cosine similarity determination subunit, used to calculate the cosine similarity between the judgment vector and the feature vectors in the human body shape database; and a contour determination subunit, used to select the human body contour with the largest cosine similarity from the human body shape database as the contour corresponding to the suspected target.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein when the computer program is executed, the device containing the computer-readable storage medium performs the suspicious target detection method based on visible light images and thermal imaging as described in any one of claims 1 to 15.

18. An electronic device, characterized in that, The device includes one or more processors and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to perform the suspicious target detection method based on visible light images and thermal imaging as described in any one of claims 1 to 15.

19. A computer program product, characterized in that, It includes a computer program or instructions that, when executed by a processor, implement the suspicious target detection method based on visible light images and thermal imaging as described in any one of claims 1 to 15.

Images