Method, device and system for detecting living objects in night vision environment

Abstract

The present invention relates to the technical field of living object recognition, and particularly relates to a method, device and system for detecting living objects in a night vision environment. The present invention obtains a suspected living object area according to the motion information difference and position distribution among pixel points in the regional infrared image, obtains the action coherence degree according to the number of changes in the motion direction and the motion condition of the same suspected living object area in the regional infrared image within the analysis period, and then screens the actual living object area; according to the action coherence degree of the actual living object area in the regional infrared image at each moment and the local gray similarity degree and texture structure of the corresponding area of the regional gray image at the same moment, a feature recognition vector is obtained, and then the types of living objects in the monitoring area in the night vision environment are detected. The present invention combines the motion information, motion mode and gray texture features of living objects for analysis, effectively reduces the influence of poor thermal imaging effect on living object recognition, and improves the accuracy of living object detection in the night vision environment.

Description

Technical Field

[0001] The present invention relates to the technical field of living object recognition, and particularly to a method, device and system for detecting living objects in a night vision environment. Background Art

[0002] At night or under low light conditions, traditional visible light imaging technology is difficult to capture clear images and cannot accurately identify objects and organisms in the environment, which poses a huge challenge to application scenarios that require real-time monitoring, security detection or biometric identification. Through the detection of living objects in a night vision environment, potential security threats can be discovered and warned in a timely manner, the security prevention ability can be improved, and the occurrence of security accidents can be reduced.

[0003] In a night vision environment, infrared images collected by an infrared detector are usually used to detect living objects. However, it may be affected by environmental factors such as environmental temperature and strong wind. When the environmental temperature is close to the body temperature of a living object, the thermal contrast between the living object and the background will be reduced. Strong wind may change the temperature distribution of the scene, resulting in poor thermal imaging effect of the living object, thereby reducing the accuracy of detecting living objects in a night vision environment. Summary of the Invention

[0004] In order to solve the technical problem of low accuracy of detecting living objects in a night vision environment due to poor thermal imaging effect of living objects, the purpose of the present invention is to provide a method, device and system for detecting living objects in a night vision environment. The specific technical solutions adopted are as follows:

[0005] In a first aspect, an embodiment of the present invention provides a method for detecting living objects in a night vision environment. The method includes:

[0006] Obtain the regional infrared image and regional grayscale image of the monitoring area at each moment in the night vision environment;

[0007] Obtain a suspected living object area according to the motion information difference and position distribution among pixel points in the regional infrared image;

[0008] Obtain the action coherence degree of each suspected living object area in the regional infrared image at each moment based on the number of changes in the motion direction and the motion condition of the same suspected living object area in the regional infrared images at adjacent moments within the analysis period at each moment; screen the actual living object areas based on the action coherence degree;

[0009] Obtain the feature recognition vector of the corresponding actual living object area according to the action coherence degree of each actual living object area in the regional infrared image at each moment and the texture structure of the corresponding area in the regional grayscale image at the same moment; detect the types of living objects in the monitoring area in the night vision environment based on the feature recognition vector.

[0010] Further, the obtaining of the action coherence degree of each suspected living object region in the regional infrared image at each moment includes:

[0011] Perform optical flow tracking on the pixel points in the regional infrared images at all moments to obtain the optical flow vectors of each pixel point in the regional infrared image at each moment;

[0012] Count the number of times the included angle between the optical flow vectors of the centroids of the same suspected living object region in the regional infrared images at adjacent moments within the analysis period at each moment is greater than the preset included angle threshold, and record it as the motion direction change value of each suspected living object region in the regional infrared image at each moment;

[0013] Calculate the cumulative sum of the magnitudes of the optical flow vectors of the centroids of the same suspected living object region in the regional infrared images at all moments within the analysis period at each moment, and use it as the motion trajectory value of each suspected living object region in the regional infrared image at each moment;

[0014] Take the distance between the centroids of the same suspected living object region in the regional infrared images at the two end moments within the analysis period at each moment as the motion displacement of each suspected living object region in the regional infrared image at each moment;

[0015] Calculate the ratio of the motion displacement to the motion trajectory value, perform a negative correlation mapping on the motion direction change value, and normalize the product of the mapping result and the ratio to obtain the action coherence degree of each suspected living object region in the regional infrared image at each moment.

[0016] Further, the method for obtaining the feature recognition vector of the actual living object region includes:

[0017] Determine the mapping region of each actual living object region in the regional infrared image at each moment in the regional grayscale image at the same moment, extract the texture features of the mapping region to obtain the gray-level co-occurrence matrix; obtain the texture feature indexes of the gray-level co-occurrence matrix, and the texture feature indexes include the correlation degree;

[0018] According to the action coherence degree and the correlation degree of each actual living object region in the regional infrared image, obtain the motion feature value of the corresponding actual living object region;

[0019] The feature recognition vector of each actual living object region is composed of the motion feature value of each actual living object region in the regional infrared image and the remaining texture feature indexes except the correlation degree.

[0020] Further, the detecting of the types of living objects in the monitoring area in the night vision environment based on the feature recognition vector includes:

[0021] Obtain the standard feature vectors of different types of living objects;

[0022] Calculate the cosine similarity between the feature recognition vectors of each actual living object region in the regional infrared image at each moment and the standard feature vectors of all types of living objects respectively, and take the type of living object corresponding to the maximum cosine similarity as the type of living object in the corresponding actual living object region.

[0023] Further, the method for obtaining the suspected living object region includes:

[0024] Perform optical flow tracking on the pixel points in the regional infrared images at all moments to obtain the moving speed of each pixel point in the regional infrared image at each moment;

[0025] Obtain the motion performance difference degree of the corresponding two pixel points according to the included angle, moving speed difference and pixel value difference of the optical flow vectors of any two pixel points in the regional infrared image;

[0026] Denote the connected domain composed of the pixel points with the motion performance difference degree less than the preset difference threshold in the regional infrared image as the local region; perform region growing on the local regions in the regional infrared image to obtain the suspected living object region.

[0027] Further, the actual living object region is the suspected living object region in the regional infrared image at each moment with the action coherence degree greater than the preset coherence threshold.

[0028] Further, the performing region growing on the local regions in the regional infrared image to obtain the suspected living object region includes:

[0029] Arbitrarily select a local region in the regional infrared image as the example region, perform region growing with the example region as the growth point, and take the other local regions that meet the preset growth conditions except the growth point as the new growth points for region growing until the growth stops when the other local regions except all the new growth points do not meet the preset growth conditions, and obtain the corresponding suspected living object region;

[0030] Denote the ratio of the maximum value to the minimum value of the areas of any two local regions as the area ratio;

[0031] The preset growth conditions include: the shortest distance to the corresponding growth point is less than the preset distance threshold and the area ratio to the corresponding growth point is greater than the preset area difference threshold.

[0032] Further, the method for performing optical flow tracking on the pixel points in the regional infrared images at all moments is the Lucas-Kanade algorithm.

[0033] In a second aspect, an embodiment of the present invention provides a living object detection device in a night vision environment. The device includes a processor, and when the processor executes, it implements the steps of the method for a living object detection device in a night vision environment as described above.

[0034] In a third aspect, another embodiment of the present invention provides a living object detection system in a night vision environment. The system includes:

[0035] A data acquisition module, configured to obtain the regional infrared image and the regional grayscale image of the monitoring area at each moment in the night vision environment;

[0036] A suspected living object area analysis module, configured to obtain a suspected living object area according to the motion information difference and position distribution between pixel points in the regional infrared image;

[0037] An actual living object area extraction module, configured to obtain the action coherence degree of each suspected living object area in the regional infrared image at each moment according to the number of times of change in the motion direction and the motion condition of the same suspected living object area in the regional infrared images at adjacent moments within the analysis period at each moment; and screen the actual living object areas based on the action coherence degree;

[0038] A living object detection module, configured to obtain a feature recognition vector of the corresponding actual living object area according to the action coherence degree of each actual living object area in the regional infrared image at each moment and the texture structure of the corresponding area in the regional grayscale image at the same moment; and detect the types of living objects in the monitoring area in the night vision environment based on the feature recognition vector.

[0039] The present invention has the following beneficial effects:

[0040] First aspect: Movement is one of the most significant features of living objects. In a complex environment, there may be a phenomenon that the temperature data of the background is close to that of the living object or there is obvious noise in the position of the living object. Analyzing the position of the living object based on the motion information difference and position distribution between pixel points in the regional infrared image and initially selecting the suspected living object area can reduce misjudgment when simply relying on the pixel value analysis of the infrared image.

[0041] Second aspect: The movement of living objects usually has coherence and regularity, while the movement of non-living objects is usually chaotic. There are significant differences in their movement patterns, especially coherence. By considering the movement coherence of the suspected living object area by analyzing the number of times of change in the movement direction and the movement condition of the same suspected living object area in the regional infrared image within the analysis period, the movement pattern and behavioral characteristics of the object can be captured, and important clues about the types and behaviors of living objects can also be provided in the night vision environment. The actual living object areas screened based on the action coherence degree can significantly improve the ability to identify living objects when the thermal imaging effect is poor.

[0042] Third aspect: Infrared images are generally rather blurred and lack details. Especially when the infrared imaging effect is not good, there may be relatively large errors in identifying living species relying on single infrared imaging data. By combining the motion coherence of the actual living area with the texture features of the corresponding area in the regional grayscale image, the poor infrared image imaging effect can be supplemented, the recognition effect in the night vision environment can be improved, the perception ability of texture features in complex scenes can be enhanced, and thus the discrimination ability of living species can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0044] Figure 1 It is a flowchart of the steps of a method for detecting living things in a night vision environment provided by an embodiment of the present invention;

[0045] Figure 2 It is a schematic diagram of a computer device of a device for detecting living things in a night vision environment provided by an embodiment of the present invention;

[0046] Figure 3 It is a system structure diagram of a system for detecting living things in a night vision environment provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0047] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the intended invention purpose, the following, in combination with the accompanying drawings and preferred embodiments, details the specific implementation manners, structures, features and their effects of a method, device and system for detecting living things in a night vision environment proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0049] The following specifically describes the specific solutions of a method, device and system for detecting living things in a night vision environment provided by the present invention with reference to the accompanying drawings.

[0050] Embodiment 1:

[0051] The present invention proposes a method for detecting living things in a night vision environment. Please refer to Figure 1, which shows a flowchart of the steps of a method for detecting living things in a night vision environment provided by an embodiment of the present invention. The method includes:

[0052] Step S1: Obtain the regional infrared image and regional grayscale image of the monitoring area at each moment in the night vision environment.

[0053] Install an infrared thermal imager and a visible light camera in the area to be monitored, that is, the monitoring area, to ensure that the two devices can fully cover the monitoring area. Use the two devices to continuously collect images of the monitoring area at each moment in the night vision environment. The image collected by the infrared thermal imager is called the regional infrared image, and the image collected by the visible light camera is called the regional RGB image. Perform grayscale processing on the regional RGB image to obtain the regional grayscale image; perform denoising and image enhancement processing on the regional infrared image and the regional grayscale image.

[0054] It should be noted that the infrared thermal imager and the visible light camera are installed closely and shoot the monitoring area at the same angle to ensure that the sizes of the regional infrared image and the regional grayscale image are equal and the pixel points correspond one by one. The sampling frequencies of the regional infrared image and the regional RGB image are the same.

[0055] In an implementation manner of the embodiment of the present invention, the sampling frequency is set to once every 0.5 seconds.

[0056] In an implementation manner of the embodiment of the present invention, a weighted average grayscale algorithm is selected for grayscale processing, Gaussian filtering is used for denoising processing, and histogram equalization is used for image enhancement processing. The specific methods of the above preprocessing are not introduced here and are all well-known technical means to those skilled in the art. Other image acquisition devices and image preprocessing algorithms can also be selected and are not limited here.

[0057] The following steps all use the preprocessed regional infrared image and regional grayscale image for analysis.

[0058] Step S2: Obtain a suspected living thing area according to the motion information difference and position distribution among pixel points in the regional infrared image.

[0059] Living things such as humans and animals will emit heat, which is shown as a highlighted part in the regional infrared image. However, factors such as high temperature and strong wind will cause the infrared imaging effect of living things to be poor, affecting the recognition of the types of living things in the monitoring area in the night vision environment. Since the movement of living things usually has continuity and the movement information of different living things is different, the position of the suspected living thing is determined based on the motion information difference and position distribution of different pixel points, improving the accuracy of recognizing and tracking living things in a complex and changeable environment.

[0060] Preferably, in some possible implementation manners of the embodiments of the present invention, the method for obtaining a suspected living object area includes: obtaining a motion performance difference degree corresponding to any two pixel points according to the included angle, moving speed difference, and pixel value difference of the optical flow vectors of the two pixel points in the regional infrared image; recording the connected domain formed by the pixel points with a motion performance difference degree less than a preset difference threshold in the regional infrared image as a local area; and performing region growing on the local area in the regional infrared image to obtain a suspected living object area.

[0061] The optical flow vector and the moving speed reflect the motion condition of the pixel point, and the pixel value of the pixel point in the infrared image presents the temperature condition of the corresponding position of the pixel point; because the motion conditions of the same living object are basically the same and the body temperatures are relatively close, if the included angle, moving speed difference, and pixel value difference of the optical flow vectors of two pixel points are smaller, it indicates that the motion conditions and temperature conditions of the corresponding positions of the two pixel points are closer, then the motion performance difference degree is smaller, and the possibility that the two pixel points are at the same position of the living object is greater. Therefore, the included angle, moving speed difference, and pixel value difference of the optical flow vectors of the two pixel points are all positively correlated with the motion performance difference degree. In the embodiments of the present invention, the product of the absolute value of the difference in the included angle of the optical flow vectors of any two pixel points in the regional infrared image, the absolute value of the difference in the moving speed, and the absolute value of the difference in the pixel value is normalized to obtain the motion performance difference degree.

[0062] In the embodiments of the present invention, the correlation relationship between the included angle of the optical flow vectors of the two pixel points, the absolute value of the difference in the moving speed, the absolute value of the difference in the pixel value, and the motion performance difference degree can also be constructed through other basic mathematical operations, which are not limited and elaborated herein.

[0063] It should be noted that in the embodiments of the present invention, the Norm function is used for normalization processing. In the embodiments of the present invention, other normalization methods can also be selected, such as function transformation, maximum-minimum normalization, and other normalization methods, which are not limited herein.

[0064] In the embodiments of the present invention, the Lucas-Kanade algorithm is used to perform optical flow tracking on the pixel points in the regional infrared image at all times to obtain the moving speed and optical flow vector of each pixel point in the regional infrared image at each time. In other embodiments, optical flow tracking algorithms such as the Kanade-Lucas-Tomasi tracking algorithm and the Horn-Schunck optical flow method can also be used.

[0065] In an implementation manner of the embodiments of the present invention, the preset difference threshold is set to 0.2.

[0066] When a living being is in motion such as walking or running, it is accompanied by the swinging of arms or legs, etc. Moreover, due to this motion, these parts may have a temperature distribution different from other parts of the body, resulting in a phenomenon of splitting and division between the swinging parts and the main body of the living being in the infrared image. Therefore, it is necessary to perform region growing and merging on the local regions belonging to the same living being in the regional infrared image to obtain a complete suspected living being region.

[0067] In an embodiment of the present invention, the method for performing region growing on a local region includes: arbitrarily selecting a local region in the regional infrared image as an example region, using the example region as a growth point to perform region growing, using other local regions that meet the preset growth conditions except the growth point as new growth points to perform region growing, and stopping the growth until other local regions except all the new growth points do not meet the preset growth conditions, so as to obtain the corresponding suspected living being region; denoting the ratio of the maximum value to the minimum value of the areas of any two local regions as the area ratio; the preset growth conditions include: the shortest distance to the corresponding growth point is less than the preset distance threshold and the area ratio to the corresponding growth point is greater than the preset area difference threshold.

[0068] Although there are significant differences in the sizes between the swinging arms or legs of a living being during motion and the main body of the living being, the distance at which the swinging part is split from the main body of the living being is shorter than the distance between different living beings. When performing local region growing and merging, it is necessary to consider the distance and area differences between different local regions. It should be noted that the shortest distance between any two edge pixel points located in two different local regions is denoted as the shortest distance between the two regions.

[0069] In an implementation manner of an embodiment of the present invention, the preset distance threshold is set to 10, and the preset area difference threshold is set to 7.

[0070] In another embodiment of the present invention, morphological dilation operation can also be performed on the local regions in the regional infrared image, and the processed region is denoted as the suspected living being region. By performing morphological dilation operation, the splitting and division that occur in the same living being region due to motion and uneven temperature distribution are filled. Among them, morphological operation is a well-known technology to those skilled in the art and will not be elaborated here.

[0071] Step S3: According to the number of changes in the motion direction and the motion situation of the same suspected living being region in the regional infrared images at adjacent moments within each analysis period at each moment, obtain the action coherence degree of each suspected living being region in the regional infrared image at each moment; based on the action coherence degree, screen the actual living being regions.

[0072] Since the movement of living things is usually coherent and regular, living things will not frequently change their movement directions in a short period of time and their movement trajectories are often relatively smooth; non-living things such as mechanical equipment may show chaotic trajectories and movement patterns with frequent direction changes in a short period of time due to external factors such as wind force and mechanical drive. Therefore, according to the number of movement direction changes and the movement conditions of the same suspected living thing area in the regional infrared images at adjacent moments within the analysis period, the movement coherence of the suspected living thing area is analyzed to obtain the action coherence degree, which is used to screen the actual living thing areas corresponding to the living things within the monitoring area.

[0073] Preferably, in some possible implementation manners of the embodiments of the present invention, the method for obtaining the action coherence degree includes: counting the number of times that the included angle between the optical flow vectors of the centroids of the same suspected living thing area in the regional infrared images at adjacent moments within each moment of the analysis period is greater than a preset included angle threshold, and recording it as the movement direction change value of each suspected living thing area in the regional infrared image at each moment; calculating the cumulative sum of the magnitudes of the optical flow vectors of the centroids of the same suspected living thing area in the regional infrared images at all moments within each moment of the analysis period as the movement trajectory value of each suspected living thing area in the regional infrared image at each moment; taking the distance between the centroids of the same suspected living thing area in the regional infrared images at the two end moments within each moment of the analysis period as the movement displacement of each suspected living thing area in the regional infrared image at each moment; calculating the ratio of the movement displacement to the movement trajectory value, performing a negative correlation mapping on the movement direction change value, and normalizing the product of the mapping result and the ratio to obtain the action coherence degree of each suspected living thing area in the regional infrared image at each moment.

[0074] The optical flow vector reflects the movement direction of the suspected living thing area, and the centroid of the suspected living thing area represents the main body part of the object, and the movement direction of the centroid will not be affected by the leg swing; the included angle between the optical flow vectors of the centroids of the same suspected living thing area in the regional infrared images at adjacent moments reflects the movement direction change situation of the suspected living thing area. The larger the included angle, the greater the possibility of movement direction change. An included angle greater than the preset included angle threshold is regarded as the movement direction change of the suspected living thing area; if the movement direction change value is larger and the number of movement direction changes of the suspected living thing area is more, the movement coherence is smaller. Non-living things have longer movement trajectories and shorter movement displacements in a period of time due to frequent movement direction changes, while living things have longer movement trajectories and movement displacements in a period of time due to their usually coherent movement. In the case of equal movement trajectory lengths, non-living things have shorter movement displacements than living things. The magnitude of the optical flow vector reflects the length of the movement trajectory. If the ratio of the movement displacement to the movement trajectory value is larger, the movement coherence of the suspected living thing area within the analysis period is better, and the possibility that the suspected living thing area is a living thing is greater.

[0075] In a specific implementation manner of the embodiment of the present invention, for each suspected living object region in the regional infrared image at each moment, the motion coherence degree LG of the suspected living object region is expressed by the formula:

[0076]

[0077] In the formula, N is the motion direction change value of the suspected living object region; L is the motion displacement of the suspected living object region; G is the motion trajectory value of the suspected living object region; Norm is the normalization function; exp is the exponential function with the natural constant as the base. It should be noted that since the analysis object of this solution is a moving object, the motion trajectory value is greater than zero.

[0078] In an implementation manner of the embodiment of the present invention, the analysis period contains 10 moments, and each moment is the last moment in its analysis period.

[0079] In an implementation manner of the embodiment of the present invention, the preset included angle threshold is set to 90 degrees. The range of the included angle between two vectors is from 0 degrees to 180 degrees. When the included angle is greater than 90 degrees, it is considered that the motion direction has a serious deviation and the motion direction changes.

[0080] Since the movement of a living object is coherent, the greater the motion coherence degree of the suspected living object region, the greater the possibility of the corresponding living object; the suspected living object regions in the regional infrared image at each moment with an action coherence degree greater than the preset coherence threshold are used as the actual living object regions, and the corresponding positions of the actual activity regions in the monitoring region are moving living objects.

[0081] In an implementation manner of the embodiment of the present invention, the preset coherence threshold is set to 0.7.

[0082] In the embodiment of the present invention, the optical flow method is used to perform region matching on the suspected living object regions in the continuous frame regional infrared images; the same suspected living object region in the continuous frame regional infrared images refers to the suspected living object region with successful region matching, representing the same moving object.

[0083] Step S4: According to the action coherence degree of each actual living object region in the regional infrared image at each moment and the texture structure of the corresponding region in the regional grayscale image at the same moment, obtain the feature recognition vector of the corresponding actual living object region; detect the types of living objects in the monitoring region in the night vision environment based on the feature recognition vector.

[0084] In some possible implementation manners of the embodiments of the present invention, the method for obtaining the feature recognition vector includes: determining the mapping region of each actual living object region in the regional infrared image at each moment in the regional gray-scale image at the same moment, and extracting texture features from the mapping region to obtain a gray-level co-occurrence matrix; obtaining texture feature indexes of the gray-level co-occurrence matrix, where the texture feature indexes include correlation; obtaining the motion feature value of the corresponding actual living object region according to the motion coherence degree and the correlation of each actual living object region in the regional infrared image; and forming the feature recognition vector of the corresponding actual living object region from the motion feature value of each actual living object region in the regional infrared image and the remaining texture feature indexes except the correlation. Among them, the method for obtaining the gray-level co-occurrence matrix and its texture feature indexes is a well-known technology to those skilled in the art and will not be elaborated here.

[0085] The correlation presents the linear relationship of different pixel gray values in the mapping region of the actual living object region in the regional gray-scale image, indicating the relationship strength between different texture features. The motion coherence degree reflects the consistency and regularity of the motion of the actual living object region in the time dimension. The contrast, uniformity, and energy mainly capture the static features of the image. When identifying a moving living object, the correlation can reflect the dynamic changes and motion patterns of the living object, and combine it with the motion coherence degree presenting motion information for analysis, thereby making up for the lack of clear texture and details in the image, being able to better capture the spatial correlation of the corresponding region of the moving living object, and enhancing the dynamic recognition ability. The greater the correlation and the motion coherence degree, the stronger the feature recognition ability of the moving living object, and the greater the motion feature value. In the embodiments of the present invention, the product of the motion coherence degree of each actual living object region in the regional infrared image at each moment and the correlation of the gray-level co-occurrence matrix of the mapping region of the actual living object region in the regional gray-scale image at the same moment is used as the motion feature value of the corresponding actual living object region.

[0086] Introducing the motion feature value in the process of constructing the feature recognition vector enables it to more sensitively recognize the unique motion laws, structural features, etc. of living objects, and the texture feature indexes can extract multi-level texture information of the region, which can enhance the perception ability of texture features in complex scenarios, and further improve the discrimination ability of living object types.

[0087] In the embodiments of the present invention, the texture feature indexes include: correlation, contrast, energy, and uniformity. Other texture feature indexes can also be used and are not limited here.

[0088] In one implementation manner of the embodiments of the present invention, in the process of obtaining the gray-level co-occurrence matrix, the selected direction is 90 degrees, and the distance d is set to 2.

[0089] Under the condition that there is no influence of environmental factors, such as the environmental temperature is 20 degrees Celsius and there is no wind, according to the same method as the feature recognition vector, obtain the feature recognition vector of each type of living thing, which is denoted as the standard feature vector of the corresponding type of living thing; the standard feature vector presents the living thing features of each type of living thing under the condition of better thermal imaging effect. Calculate the cosine similarity between the feature recognition vector of each actual living thing area in the regional infrared image at each moment and the standard feature vectors of all types of living things respectively, and take the living thing type corresponding to the maximum cosine similarity as the living thing type of the corresponding actual living thing area.

[0090] It should be noted that the greater the cosine similarity between the feature recognition vector of the actual living thing area and the standard feature vector of a certain type of living thing, the more similar the actual living thing area is to the living thing features of that type of living thing, and the greater the possibility that the actual living thing area represents that type of living thing.

[0091] In other embodiments, the feature recognition vector of the suspected living thing area can be input into the trained neural network, and the neural network outputs the living thing type corresponding to the suspected living thing area.

[0092] So far, the present invention is completed.

[0093] Embodiment 2:

[0094] Figure 2 It is a schematic diagram of a computer device for a living thing detection device in a night vision environment provided by an embodiment of the present invention. Exemplarily, as Figure 2 shown, the computer device includes: a memory 501, a processor 502, and a computer program 503 stored in the memory 501 and running on the processor 502. Among them, when the processor 502 executes the computer program 503, the computer device can execute any one of the living thing detection methods in the night vision environment described above.

[0095] In addition, an embodiment of the present application also protects a device, which may include a memory and a processor. Among them, an executable program code is stored in the memory, and the processor is used to call and execute the executable program code to execute a living thing detection method provided by an embodiment of the present application.

[0096] This embodiment can divide the functions of the device according to the above method examples. For example, it can correspond to each functional module, or integrate two or more functions into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is illustrative, only a logical function division, and there may be other division methods in actual implementation.

[0097] It should be understood that the device provided in this embodiment is used to execute the above-mentioned method for detecting living things in a night vision environment, so the same effects as those of the above-mentioned implementation method can be achieved.

[0098] In the case of adopting an integrated unit, the device may include a processing module and a storage module. Among them, when the device is applied to a device, the processing module may be used to control and manage the actions of the device. The storage module may be used to support the device to execute mutual program codes, etc.

[0099] Among them, the processing module may be a processor or a controller, which can implement or execute various exemplary logic blocks, modules, and circuits included in combination with the disclosure of the present application. The processor may also be a combination that realizes computing functions, such as including a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module may be a memory.

[0100] Embodiment 3:

[0101] The present invention proposes a living thing detection system in a night vision environment. Please refer to Figure 3 , which shows the system structure diagram of a living thing detection system in a night vision environment provided by an embodiment of the present invention. The system includes:

[0102] A data acquisition module 610, configured to acquire the regional infrared image and the regional grayscale image of the monitoring area at each moment in the night vision environment;

[0103] A suspected living thing area analysis module 620, configured to obtain a suspected living thing area according to the motion information difference and position distribution between pixel points in the regional infrared image;

[0104] An actual living thing area extraction module 630, configured to obtain the action coherence degree of each suspected living thing area in the regional infrared image at each moment according to the number of changes in the motion direction and the motion condition of the same suspected living thing area in the regional infrared images at adjacent moments within the analysis period of each moment; and screen the actual living thing areas based on the action coherence degree;

[0105] A living thing detection module 640, configured to obtain a feature recognition vector of the corresponding actual living thing area according to the action coherence degree of each actual living thing area in the regional infrared image at each moment and the texture structure of the corresponding area in the regional grayscale image at the same moment; and detect the types of living things in the monitoring area in the night vision environment based on the feature recognition vector.

[0106] It should be noted that: For the device provided in the above embodiments, only the division of the above functional modules is used for illustration. In actual applications, the above functions can be allocated to different functional modules as needed, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above. In addition, a live object detection system and a live object detection method embodiment provided in the above embodiments belong to the same concept. For the specific implementation process, please refer to the method embodiment, which will not be elaborated here.

[0107] Embodiment 4:

[0108] This embodiment also provides a computer-readable storage medium, in which computer program code is stored. When the computer program code runs on a computer, it causes the computer to execute the above-related method steps to implement a live object detection method provided in the above embodiments.

[0109] Embodiment 5:

[0110] This embodiment also provides a computer program product. When the computer program product runs on a computer, it causes the computer to execute the above-related steps to implement a live object detection method provided in the above embodiments.

[0111] Among them, the device, computer-readable storage medium, computer program product, or chip provided in this embodiment are all used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, which will not be elaborated here.

[0112] In the embodiments provided in the present application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical or other form.

[0113] It should be noted that: The above sequence of the embodiments of the present invention is only for description and does not represent the superiority or inferiority of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0114] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other, and the differences between each embodiment and other embodiments are emphasized respectively.

Claims

1. A method for detecting living things in a night vision environment, characterized in that: The method includes: Obtain regional infrared images and regional grayscale images of the monitoring area at each moment in the night vision environment; Obtain the suspected living object area based on the motion information difference and position distribution between pixels in the regional infrared image; Obtaining the motion coherence of each suspected living creature region in the regional infrared image at each moment according to the number of changes in motion direction and motion conditions of the same suspected living creature region in the regional infrared image at adjacent moments within the analysis period at each moment; and screening the actual living creature region based on the motion coherence; According to the motion coherence of each actual living creature region in the regional infrared image at each moment and the texture structure of the corresponding region in the regional grayscale image at the same moment, a feature recognition vector corresponding to the actual living creature region is obtained; based on the feature recognition vector, the type of living creature in the monitoring area under the night vision environment is detected; The step of obtaining the motion coherence of each suspected living creature region in the regional infrared image at each moment includes: Perform optical flow tracking on the pixel points in the regional infrared image at all times to obtain the optical flow vector of each pixel point in the regional infrared image at each time; Count the number of times that the angle between the optical flow vectors of the centroid of the same suspected living object region in the regional infrared images at adjacent moments within the analysis period at each moment is greater than a preset angle threshold, and record it as the change value of the movement direction of each suspected living object region in the regional infrared images at each moment; Calculate the cumulative sum of the modulus of the optical flow vector of the centroid of the same suspected living object region in the regional infrared image at all times within the analysis period at each moment as the motion trajectory value of each suspected living object region in the regional infrared image at each moment; The distance between the centroids of the same suspected living thing region in the regional infrared image at two end points in the analysis period at each moment is taken as the motion displacement of each suspected living thing region in the regional infrared image at each moment; The ratio of the motion displacement to the motion trajectory value is calculated, the motion direction change value is negatively correlated and mapped, and the product of the mapping result and the ratio is normalized to obtain the motion coherence of each suspected living thing area in the regional infrared image at each moment.

2. The method for detecting living things in a night vision environment according to claim 1, characterized in that: The method for acquiring the feature recognition vector of the actual living object area includes: Determine the mapping area of ​​each actual living object area in the regional infrared image at each moment in the regional grayscale image at the same moment, extract texture features of the mapping area to obtain a gray-level co-occurrence matrix; obtain texture feature indicators of the gray-level co-occurrence matrix, wherein the texture feature indicators include correlation; According to the motion continuity and the correlation of each actual living thing region in the regional infrared image, obtaining a motion feature value corresponding to the actual living thing region; The motion feature value of each actual living thing region in the regional infrared image and the remaining texture feature indicators except the correlation form a feature recognition vector corresponding to the actual living thing region.

3. The method for detecting living things in a night vision environment according to claim 1, characterized in that: The detecting the type of living thing in the monitoring area in the night vision environment based on the feature recognition vector includes: Obtain standard feature vectors for different types of living things; The cosine similarities between the feature recognition vector of each actual living thing region in the regional infrared image at each moment and the standard feature vectors of all types of living things are calculated respectively, and the living thing type corresponding to the maximum cosine similarity is used as the living thing type corresponding to the actual living thing region.

4. The method for detecting living things in a night vision environment according to claim 1, characterized in that: The method for obtaining the suspected living thing area includes: Perform optical flow tracking on the pixel points in the regional infrared image at all times to obtain the moving speed of each pixel point in the regional infrared image at each time; According to the angle of the optical flow vectors, the moving speed difference and the pixel value difference of any two pixel points in the regional infrared image, the difference in motion performance of the corresponding two pixel points is obtained; The connected domain formed by the pixel points corresponding to the motion performance difference less than the preset difference threshold in the regional infrared image is recorded as a local area; the local area in the regional infrared image is subjected to regional growth to obtain a suspected living thing area.

5. The method for detecting living things in a night vision environment according to claim 1, characterized in that: The actual living creature region is a suspected living creature region in which the action continuity in the regional infrared image at each moment is greater than a preset continuity threshold.

6. The method for detecting living things in a night vision environment according to claim 4, characterized in that: The step of performing regional growth on the local region in the regional infrared image to obtain a suspected living object region includes: In the regional infrared image, a local area is selected as a sample area, the sample area is used as a growth point for regional growth, and other local areas except the growth point that meet the preset growth conditions are used as new growth points for regional growth, until the growth is stopped when other local areas except all new growth points do not meet the preset growth conditions, and the corresponding suspected living thing area is obtained; The ratio of the maximum value to the minimum value of the area of ​​any two local regions is recorded as the area ratio; The preset growth conditions include: the shortest distance to the corresponding growth point is less than a preset distance threshold and the area ratio to the corresponding growth point is greater than a preset area difference threshold.

7. The method for detecting living things in a night vision environment according to claim 4, characterized in that: The method for performing optical flow tracing on pixel points in regional infrared images at all times is the Lucas-Kanade algorithm.

8. A device for detecting living things in a night vision environment, characterized in that: The device includes a processor, and when the processor is executed, the steps of a method for detecting living things in a night vision environment according to any one of claims 1 to 7 are implemented.

9. A system for detecting living things in a night vision environment, characterized in that: The system includes: A data acquisition module is used to obtain regional infrared images and regional grayscale images of the monitoring area at each moment in a night vision environment; A suspected living thing area analysis module is used to obtain a suspected living thing area based on the motion information difference and position distribution between pixels in the regional infrared image; The actual living creature region extraction module is used to obtain the motion coherence of each suspected living creature region in the regional infrared image at each moment according to the number of motion direction changes and motion conditions of the same suspected living creature region in the regional infrared image at adjacent moments within the analysis period at each moment; and to screen the actual living creature region based on the motion coherence; A living thing detection module, for obtaining a feature recognition vector corresponding to each actual living thing region according to the motion continuity of each actual living thing region in the regional infrared image at each moment and the texture structure of the corresponding region in the regional grayscale image at the same moment; and detecting the type of living thing in the monitoring area under the night vision environment based on the feature recognition vector; The step of obtaining the motion coherence of each suspected living creature region in the regional infrared image at each moment includes: Perform optical flow tracking on the pixel points in the regional infrared image at all times to obtain the optical flow vector of each pixel point in the regional infrared image at each time; Count the number of times that the angle between the optical flow vectors of the centroid of the same suspected living object region in the regional infrared images at adjacent moments within the analysis period at each moment is greater than a preset angle threshold, and record it as the change value of the movement direction of each suspected living object region in the regional infrared images at each moment; Calculate the cumulative sum of the modulus of the optical flow vector of the centroid of the same suspected living object region in the regional infrared image at all times within the analysis period at each moment as the motion trajectory value of each suspected living object region in the regional infrared image at each moment; The distance between the centroids of the same suspected living thing region in the regional infrared image at two end points in the analysis period at each moment is taken as the motion displacement of each suspected living thing region in the regional infrared image at each moment; The ratio of the motion displacement to the motion trajectory value is calculated, the motion direction change value is negatively correlated and mapped, and the product of the mapping result and the ratio is normalized to obtain the motion coherence of each suspected living thing area in the regional infrared image at each moment.

Images