Living object detection method, device and system in night vision environment

Abstract

The invention relates to the technical field of living thing identification, in particular to a living thing detection method, device and system in a night vision environment. The method comprises the following steps of: acquiring a suspected living object region according to motion information difference and position distribution among pixel points in a regional infrared image, acquiring motion coherence according to motion direction change times and motion conditions of the same suspected living object region in the regional infrared image in an analysis time period, and further screening an actual living object region; and according to the motion coherence degree of the actual living object region in the regional infrared image at each moment and the local gray level similarity degree and texture structure of the region corresponding to the regional gray level image at the same moment, obtaining a feature recognition vector, and further detecting the type of the living object in the monitoring region in the night vision environment. According to the method, the motion information, the motion mode and the gray texture features of the living things are combined for analysis, the influence of a poor thermal imaging effect on living thing recognition is effectively reduced, and the accuracy of living thing detection in the night vision environment is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of living object recognition, and in particular to a method, device and system for detecting living objects in a night vision environment. Background Art

[0002] At night or in low light conditions, traditional visible light imaging technology has difficulty capturing clear images and cannot accurately identify objects and organisms in the environment, which is a huge challenge for application scenarios that require real-time monitoring, security detection or biometrics. Through the detection of living things in night vision environments, potential security threats can be discovered and warned in a timely manner, improving security prevention capabilities and reducing the occurrence of security accidents.

[0003] Infrared images collected by infrared detectors are usually used to detect living things in night vision environments. However, this may be affected by environmental factors such as ambient temperature and strong winds. Ambient temperature close to the body temperature of living things will cause the thermal contrast between living things and the background to decrease. Strong winds may change the temperature distribution of the scene, resulting in poor thermal imaging of living things, thereby reducing the accuracy of living thing detection in night vision environments. Summary of the invention

[0004] In order to solve the technical problem that the accuracy of detecting living things in a night vision environment is low due to the poor thermal imaging effect of living things, the purpose of the present invention is to provide a method, device and system for detecting living things in a night vision environment. The technical solutions adopted are as follows: In a first aspect, an embodiment of the present invention provides a method for detecting living things in a night vision environment, the method comprising: Obtain regional infrared images and regional grayscale images of the monitoring area at each moment in the night vision environment; Obtain the suspected living object area based on the motion information difference and position distribution between pixels in the regional infrared image; Obtaining the motion coherence of each suspected living creature region in the regional infrared image at each moment according to the number of changes in motion direction and motion conditions of the same suspected living creature region in the regional infrared image at adjacent moments within the analysis period at each moment; and screening the actual living creature region based on the motion coherence; According to the motion continuity of each actual living thing area in the regional infrared image at each moment and the texture structure of the corresponding area in the regional grayscale image at the same moment, a feature recognition vector corresponding to the actual living thing area is obtained; based on the feature recognition vector, the type of living thing in the monitoring area under the night vision environment is detected.

[0005] Furthermore, the step of obtaining the motion coherence of each suspected living creature region in the regional infrared image at each moment includes: Perform optical flow tracking on the pixel points in the regional infrared image at all times to obtain the optical flow vector of each pixel point in the regional infrared image at each time; Count the number of times that the angle between the optical flow vectors of the centroid of the same suspected living object region in the regional infrared images at adjacent moments within the analysis period at each moment is greater than a preset angle threshold, and record it as the change value of the movement direction of each suspected living object region in the regional infrared images at each moment; Calculate the cumulative sum of the modulus of the optical flow vector of the centroid of the same suspected living object region in the regional infrared image at all times within the analysis period at each moment as the motion trajectory value of each suspected living object region in the regional infrared image at each moment; The distance between the centroids of the same suspected living thing region in the regional infrared image at two end points in the analysis period at each moment is taken as the motion displacement of each suspected living thing region in the regional infrared image at each moment; The ratio of the motion displacement to the motion trajectory value is calculated, the motion direction change value is negatively correlated and mapped, and the product of the mapping result and the ratio is normalized to obtain the motion coherence of each suspected living thing area in the regional infrared image at each moment.

[0006] Furthermore, the method for acquiring the feature recognition vector of the actual living object area includes: Determine the mapping area of ​​each actual living object area in the regional infrared image at each moment in the regional grayscale image at the same moment, extract texture features of the mapping area to obtain a gray-level co-occurrence matrix; obtain texture feature indicators of the gray-level co-occurrence matrix, wherein the texture feature indicators include correlation; According to the motion continuity and the correlation of each actual living thing region in the regional infrared image, obtaining a motion feature value corresponding to the actual living thing region; The motion feature value of each actual living thing region in the regional infrared image and the remaining texture feature indicators except the correlation form a feature recognition vector corresponding to the actual living thing region.

[0007] Furthermore, the detecting the type of living thing in the monitoring area under the night vision environment based on the feature recognition vector includes: Obtain standard feature vectors for different types of living things; The cosine similarities between the feature recognition vector of each actual living thing region in the regional infrared image at each moment and the standard feature vectors of all types of living things are calculated respectively, and the living thing type corresponding to the maximum cosine similarity is used as the living thing type corresponding to the actual living thing region.

[0008] Furthermore, the method for obtaining the suspected living thing area includes: Perform optical flow tracking on the pixels in the regional infrared image at all times to obtain the moving speed of each pixel in the regional infrared image at each time; According to the angle of the optical flow vectors, the moving speed difference and the pixel value difference of any two pixel points in the regional infrared image, the difference in motion performance of the corresponding two pixel points is obtained; The connected domain formed by the pixel points corresponding to the motion performance difference less than the preset difference threshold in the regional infrared image is recorded as a local area; the local area in the regional infrared image is subjected to regional growth to obtain a suspected living thing area.

[0009] Furthermore, the actual living creature region is a suspected living creature region in which the motion continuity in the regional infrared image at each moment is greater than a preset continuity threshold.

[0010] Furthermore, the performing of region growing on the local region in the regional infrared image to obtain the suspected living object region includes: In the regional infrared image, a local area is selected as a sample area, the sample area is used as a growth point for regional growth, and other local areas except the growth point that meet the preset growth conditions are used as new growth points for regional growth, until the growth is stopped when other local areas except all new growth points do not meet the preset growth conditions, and the corresponding suspected living thing area is obtained; The ratio of the maximum value to the minimum value of the area of ​​any two local regions is recorded as the area ratio; The preset growth conditions include: the shortest distance to the corresponding growth point is less than a preset distance threshold and the area ratio to the corresponding growth point is greater than a preset area difference threshold.

[0011] Furthermore, the method for performing optical flow tracing on pixel points in the regional infrared image at all times is a Lucas-Kanade algorithm.

[0012] In a second aspect, an embodiment of the present invention provides a device for detecting living things in a night vision environment. The device includes a processor, and when the processor is executed, the steps of a method for detecting living things in a night vision environment are implemented as described above.

[0013] In a third aspect, another embodiment of the present invention provides a system for detecting living things in a night vision environment, the system comprising: A data acquisition module is used to obtain regional infrared images and regional grayscale images of the monitoring area at each moment in a night vision environment; A suspected living thing area analysis module is used to obtain a suspected living thing area based on the motion information difference and position distribution between pixels in the regional infrared image; The actual living creature region extraction module is used to obtain the motion continuity of each suspected living creature region in the regional infrared image at each moment according to the number of motion direction changes and motion conditions of the same suspected living creature region in the regional infrared image at adjacent moments within the analysis period at each moment; and screen the actual living creature region based on the motion continuity; The living thing detection module is used to obtain the feature recognition vector corresponding to the actual living thing area according to the motion continuity of each actual living thing area in the regional infrared image at each moment and the texture structure of the corresponding area in the regional grayscale image at the same moment; based on the feature recognition vector, detect the type of living thing in the monitoring area under the night vision environment.

[0014] The present invention has the following beneficial effects: First, movement is one of the most prominent characteristics of living things. In a complex environment, the temperature data of the background and the living things may be close or there may be obvious noise at the location of the living things. The location of the living things is analyzed based on the difference in motion information and position distribution between pixels in the regional infrared image, and the suspected living things area is preliminarily selected, which can reduce misjudgment when relying solely on pixel value analysis of infrared images.

[0015] The second aspect: the movement of living things is usually coherent and regular, while the movement of non-living things is usually chaotic. There is a big difference in the movement patterns, especially the coherence, of the two. By analyzing the number of changes in the movement direction and the movement conditions of the same suspected living area in the regional infrared image within a time period, the movement coherence of the suspected living area is considered, and the movement pattern and behavioral characteristics of the object are captured. In the night vision environment, it can also provide important clues about the type and behavior of living things. The actual living area screened based on the motion continuity can significantly improve the ability to identify living things when the thermal imaging effect is poor.

[0016] The third aspect: Infrared images are generally blurry and lack details. Especially when the infrared imaging effect is poor, relying on a single infrared imaging data to identify the species of living things may result in large errors. Combining the motion continuity of the actual living area with the texture features of the corresponding area in the regional grayscale image can supplement the poor imaging effect of infrared images, improve the recognition effect in night vision environments, enhance the perception of texture features in complex scenes, and thus improve the ability to distinguish living species. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A flowchart of a method for detecting living things in a night vision environment provided by an embodiment of the present invention; Figure 2 A schematic diagram of a computer device for a living thing detection device in a night vision environment provided by an embodiment of the present invention; Figure 3 A system structure diagram of a living thing detection system in a night vision environment provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the method, device and system for detecting living things in a night vision environment proposed by the present invention, its specific implementation method, structure, features and effects in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0020] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0021] The following is a detailed description of a method, device and system for detecting living things in a night vision environment provided by the present invention in conjunction with the accompanying drawings.

[0022] Embodiment 1: The present invention proposes a method for detecting living things in a night vision environment. Figure 1 , which shows a flowchart of a method for detecting living things in a night vision environment provided by an embodiment of the present invention, the method comprising: Step S1: Obtain regional infrared images and regional grayscale images of the monitoring area at each moment in a night vision environment.

[0023] Install an infrared thermal imager and a visible light camera in the area that needs to be monitored, that is, the monitoring area, to ensure that the two devices can fully cover the monitoring area. Use the two devices to continuously collect images of the monitoring area at each moment under the night vision environment. The images collected by the infrared thermal imager are called regional infrared images, and the images collected by the visible light camera are called regional RGB images. Perform grayscale processing on the regional RGB image to obtain a regional grayscale image; perform denoising and image enhancement processing on the regional infrared image and the regional grayscale image.

[0024] It should be noted that the infrared thermal imager and the visible light camera are installed in close proximity and use the same angle to shoot the monitoring area, ensuring that the size of the regional infrared image and the regional grayscale image are equal and the pixels correspond one to one. The sampling frequency of the regional infrared image and the regional RGB image is the same.

[0025] In an implementation of the embodiment of the present invention, the sampling frequency is set to once every 0.5 seconds.

[0026] In one implementation of the embodiment of the present invention, a weighted average grayscale algorithm is selected for grayscale processing, a Gaussian filter is used for denoising, and a histogram equalization is used for image enhancement processing. The specific methods of the above preprocessing are not introduced here, and they are all technical means well known to those skilled in the art. Other image acquisition devices and image preprocessing algorithms can also be selected, which are not limited here.

[0027] The following steps all use the preprocessed regional infrared image and regional grayscale image for analysis.

[0028] Step S2: Obtain the suspected living thing area according to the motion information difference and position distribution between the pixels in the regional infrared image.

[0029] Living things such as humans and animals emit heat, which is displayed as a highlight in the regional infrared image. However, factors such as high temperature and strong wind will lead to poor infrared imaging of living things, affecting the identification of living things in the monitoring area under night vision environment. Since the movement of living things is usually continuous and the movement information of different living things is different, the position of suspected living things is determined based on the difference in movement information and position distribution of different pixel points, which improves the accuracy of identifying and tracking living things in complex and changing environments.

[0030] Preferably, in some possible implementation modes of the embodiments of the present invention, the method for obtaining the suspected living thing area includes: obtaining the motion performance difference corresponding to any two pixel points in the regional infrared image according to the angle of the optical flow vectors, the moving speed difference and the pixel value difference between the two pixel points in the regional infrared image; recording the connected domain formed by the pixel points corresponding to the motion performance difference less than a preset difference threshold in the regional infrared image as a local area; performing regional growth on the local area in the regional infrared image to obtain the suspected living thing area.

[0031] The optical flow vector and the moving speed reflect the movement of the pixel point, and the pixel value of the pixel point in the infrared image presents the temperature of the corresponding position of the pixel point; because the movement of the same living thing is basically the same and the body temperature is relatively close, if the angle of the optical flow vector of two pixels, the difference in moving speed and the difference in pixel value are smaller, it means that the movement and temperature of the corresponding positions of the two pixels are closer, then the smaller the difference in motion performance, the greater the possibility that the two pixels are in the same position of the living thing. Therefore, the angle of the optical flow vector of two pixels, the difference in moving speed and the difference in pixel value are all positively correlated with the difference in motion performance. In an embodiment of the present invention, the product of the absolute value of the difference in the angle and moving speed of the optical flow vector of any two pixels in the regional infrared image and the absolute value of the difference in pixel value is normalized to obtain the difference in motion performance.

[0032] In the embodiment of the present invention, other basic mathematical operations can also be used to construct the correlation between the angle of the optical flow vector of two pixels, the absolute value of the difference in moving speed and the absolute value of the difference in pixel values ​​and the difference in motion performance, which is not limited or elaborated here.

[0033] It should be noted that the Norm function is used to perform normalization in the embodiment of the present invention. Other normalization methods may also be selected in the embodiment of the present invention, such as function conversion, maximum and minimum normalization, etc., which are not limited here.

[0034] In the embodiment of the present invention, the Lucas-Kanade algorithm is used to perform optical flow tracking on the pixels in the regional infrared image at all times to obtain the moving speed and optical flow vector of each pixel in the regional infrared image at each time. In other embodiments, optical flow tracking algorithms such as the Kanade-Lucas-Tomasi tracking algorithm and the Horn-Schunck optical flow method can also be used.

[0035] In an implementation of the embodiment of the present invention, the preset difference threshold is set to 0.2.

[0036] When living things walk or run, their arms or legs swing, and these parts may produce a temperature distribution different from other parts of the body due to movement, resulting in the swinging parts and the main body of the living thing being separated in the infrared image. Therefore, it is necessary to grow and merge the local areas belonging to the same living thing in the regional infrared image to obtain a complete suspected living thing area.

[0037] In an embodiment of the present invention, a method for performing regional growth on a local area includes: randomly selecting a local area in a regional infrared image as an example area, performing regional growth on the example area as a growth point, performing regional growth on other local areas except the growth point that meet preset growth conditions as new growth points, and stopping the growth when other local areas except all new growth points do not meet the preset growth conditions, thereby obtaining a corresponding suspected living thing area; recording the ratio of the maximum value to the minimum value of the area of ​​any two local areas as the area ratio; the preset growth conditions include: the shortest distance between the corresponding growth point is less than a preset distance threshold and the area ratio with the corresponding growth point is greater than a preset area difference threshold.

[0038] Although the size of the swinging arms or legs of a living creature is quite different from that of the main body of the living creature, the distance between the swinging parts and the main body of the living creature is shorter than the distance between different living creatures. When growing and merging local regions, the distance and area differences of different local regions need to be considered. It should be noted that the shortest distance between any two edge pixels located in two local regions is recorded as the shortest distance between the two regions.

[0039] In an implementation of the embodiment of the present invention, the preset distance threshold is set to 10, and the preset area difference threshold is set to 7.

[0040] In another embodiment of the present invention, a morphological expansion operation may be performed on a local area in the regional infrared image, and the processed area is recorded as a suspected living object area. The morphological expansion operation is used to fill the split divisions in the same living object area caused by motion and uneven temperature distribution. The morphological operation is a well-known technology for those skilled in the art and will not be described in detail here.

[0041] Step S3: Obtain the motion continuity of each suspected living creature region in the regional infrared image at each moment according to the number of motion direction changes and motion conditions of the same suspected living creature region in the regional infrared image at adjacent moments within the analysis period at each moment; and select the actual living creature region based on the motion continuity.

[0042] Since the movement of living things is usually coherent and regular, they will not change their movement direction frequently in a short period of time and their movement trajectories are often relatively smooth; non-living things such as mechanical equipment may show chaotic trajectories and frequent changes of direction in a short period of time due to external factors such as wind and mechanical drive. Therefore, according to the number of changes in the movement direction and movement conditions of the same suspected living area in the regional infrared images at adjacent moments during the analysis period, the movement coherence of the suspected living area is analyzed to obtain the motion coherence, which is used to screen the actual living area corresponding to the living things in the monitoring area.

[0043] Preferably, in some possible implementations of the embodiments of the present invention, the method for obtaining motion coherence includes: counting the number of times that the angle between the optical flow vectors of the centroid of the same suspected living thing area in the regional infrared images at adjacent moments within the analysis period of each moment is greater than a preset angle threshold, and recording it as the motion direction change value of each suspected living thing area in the regional infrared image at each moment; calculating the cumulative sum of the moduli of the optical flow vectors of the centroid of the same suspected living thing area in the regional infrared images at all moments within the analysis period of each moment as the motion trajectory value of each suspected living thing area in the regional infrared image at each moment; taking the distance between the centroids of the same suspected living thing area in the regional infrared images at two end moments within the analysis period of each moment as the motion displacement of each suspected living thing area in the regional infrared image at each moment; calculating the ratio of the motion displacement to the motion trajectory value, performing negative correlation mapping on the motion direction change value, and normalizing the product of the mapping result and the ratio to obtain the motion coherence of each suspected living thing area in the regional infrared image at each moment.

[0044] The optical flow vector reflects the direction of movement of the suspected living area. The center of mass of the suspected living area represents the main part of the object, and the direction of movement of the center of mass will not be affected by the swing of the legs. The angle of the optical flow vector of the center of mass of the same suspected living area in the regional infrared image at adjacent moments reflects the change in the direction of movement of the suspected living area. The larger the angle, the greater the possibility of the change in the direction of movement. The angle greater than the preset angle threshold is regarded as the change in the direction of movement of the suspected living area. If the change value of the direction of movement is larger, the more times the direction of movement of the suspected living area changes, the smaller the movement coherence. The non-living objects have a longer movement trajectory and a shorter movement displacement within a period of time due to the frequent changes in the movement direction. The movement of living objects is usually coherent, which leads to a longer movement trajectory and movement displacement within a period of time. When the length of the movement trajectory is equal, the movement displacement of the non-living object is shorter than that of the living object. The modulus of the optical flow vector reflects the length of the movement trajectory. If the ratio of the movement displacement to the movement trajectory value is larger, the movement coherence of the suspected living area in the analysis period is better, and the possibility that the suspected living area is a living object is greater.

[0045] In a specific implementation of the embodiment of the present invention, for each suspected living object region in the regional infrared image at each moment, the motion coherence LG of the suspected living object region is expressed by the formula: In the formula, N is the change in the direction of movement of the suspected living thing area; L is the movement displacement of the suspected living thing area; G is the movement trajectory value of the suspected living thing area; Norm is the normalization function; exp is an exponential function with a natural constant as the base. It should be noted that since the analysis object of this scheme is a moving object, the movement trajectory value is greater than zero.

[0046] In an implementation of the embodiment of the present invention, the analysis period includes 10 moments, and each moment is the last moment in the analysis period.

[0047] In one implementation of the embodiment of the present invention, the preset angle threshold is set to 90 degrees. The angle between the two vectors ranges from 0 degrees to 180 degrees. When the angle is greater than 90 degrees, it is considered that the motion direction is seriously offset and the motion direction is changed.

[0048] Since the movement of living things is continuous, the greater the movement continuity of the suspected living thing area, the greater the possibility that it corresponds to a living thing; the suspected living thing area with a movement continuity greater than a preset continuity threshold in the regional infrared image at each moment is regarded as the actual living thing area, and the actual activity area at the corresponding position in the monitoring area is the moving living thing.

[0049] In an implementation of the embodiment of the present invention, the preset coherence threshold is set to 0.7.

[0050] In the embodiment of the present invention, the optical flow method is used to perform regional matching on the suspected living object regions in the continuous frame region infrared images; the same suspected living object region in the continuous frame region infrared images refers to the suspected living object region with successful regional matching, representing the same moving object.

[0051] Step S4: According to the motion continuity of each actual living thing area in the regional infrared image at each moment and the texture structure of the corresponding area in the regional grayscale image at the same moment, obtain the feature recognition vector corresponding to the actual living thing area; based on the feature recognition vector, detect the type of living thing in the monitoring area under the night vision environment.

[0052] In some possible implementations of the present invention, the method for obtaining the feature recognition vector includes: determining the mapping area of ​​each actual living thing area in the regional infrared image at each moment in the regional grayscale image at the same moment, extracting the texture features of the mapping area to obtain a grayscale co-occurrence matrix; obtaining the texture feature index of the grayscale co-occurrence matrix, the texture feature index includes the correlation; according to the action coherence and correlation of each actual living thing area in the regional infrared image, obtaining the motion feature value of the corresponding actual living thing area; the motion feature value of each actual living thing area in the regional infrared image and the remaining texture feature indexes except the correlation constitute the feature recognition vector of the corresponding actual living thing area. Among them, the method for obtaining the grayscale co-occurrence matrix and its texture feature index is a well-known technology for those skilled in the art, and will not be repeated here.

[0053] The correlation presents the linear relationship between the grayscale values ​​of different pixels in the mapping area of ​​the regional grayscale image of the actual living creature area, indicating the strength of the relationship between different texture features, and the motion coherence reflects the consistency and regularity of the motion of the actual living creature area in the time dimension. The contrast, uniformity and energy mainly capture the static features of the image; when identifying moving living creatures, the correlation can reflect the dynamic changes and motion patterns of the living creatures, and combine it with the motion coherence that presents motion information for analysis, thereby making up for the lack of clear texture and details in the image, and can better capture the spatial correlation of the corresponding area of ​​the moving living creature, and enhance the dynamic recognition ability. The greater the correlation and the motion coherence, the stronger the feature ability to identify moving living creatures, and the greater the motion feature value. In an embodiment of the present invention, the product of the motion coherence of each actual living creature area in the regional infrared image at each moment and the correlation of the grayscale co-occurrence matrix of the mapping area of ​​the actual living creature area in the regional grayscale image at the same moment is used as the motion feature value of the corresponding actual living creature area.

[0054] In the process of constructing feature recognition vectors, motion feature values ​​are introduced to make it more sensitive to identify the unique movement patterns and structural characteristics of living things. The texture feature indicators can extract multi-level texture information of the region, which can enhance the perception of texture features in complex scenes and thus improve the ability to distinguish species of living things.

[0055] In the embodiment of the present invention, the texture feature indicators include: correlation, contrast, energy and uniformity. Other texture feature indicators may also be used and are not limited here.

[0056] In an implementation of the embodiment of the present invention, in the process of obtaining the gray level co-occurrence matrix, the direction is selected as 90 degrees and the distance d is set to 2.

[0057] Under the condition that there is no influence of environmental factors, such as the ambient temperature is 20 degrees Celsius and there is no wind, the feature recognition vector of each type of living creature is obtained in the same way as the feature recognition vector, and recorded as the standard feature vector of the corresponding type of living creature; the standard feature vector presents the living creature characteristics of each type of living creature under the condition of better thermal imaging effect. The cosine similarity between the feature recognition vector of each actual living creature area in the regional infrared image at each moment and the standard feature vector of all types of living creatures is calculated respectively, and the living creature type corresponding to the maximum cosine similarity is taken as the living creature type corresponding to the actual living creature area.

[0058] It should be noted that the greater the cosine similarity between the feature recognition vector of the actual living thing area and the standard feature vector of a certain type of living thing, the more similar the living thing characteristics of the actual living thing area are to the living thing of that type, and the greater the possibility that the actual living thing area represents the living thing of that type.

[0059] In other embodiments, the feature recognition vector of the suspected living thing region may be input into a trained neural network, and the neural network outputs the living thing type corresponding to the suspected living thing region.

[0060] So far, the present invention is completed.

[0061] Embodiment 2: Figure 2 A computer device schematic diagram of a living thing detection device in a night vision environment provided by an embodiment of the present invention. For example, Figure 2 As shown, the computer device includes: a memory 501, a processor 502, and a computer program 503 stored in the memory 501 and running on the processor 502, wherein when the processor 502 executes the computer program 503, the computer device can execute any one of the living thing detection methods in a night vision environment introduced above.

[0062] In addition, an embodiment of the present application also protects a device, which may include a memory and a processor, wherein the memory stores an executable program code, and the processor is used to call and execute the executable program code to execute a method for detecting living things in a night vision environment provided by an embodiment of the present application.

[0063] In this embodiment, the functional modules of the device can be divided according to the above method example. For example, each functional module can be corresponded, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0064] It should be understood that the device provided in this embodiment is used to execute the above-mentioned method for detecting living things in a night vision environment, and thus can achieve the same effect as the above-mentioned implementation method.

[0065] In the case of an integrated unit, the device may include a processing module and a storage module. When the device is applied to a device, the processing module may be used to control and manage the actions of the device. The storage module may be used to support the device to execute mutual program codes, etc.

[0066] The processing module may be a processor or a controller, which may implement or execute various exemplary logic blocks, modules and circuits included in the disclosure of this application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module may be a memory.

[0067] Embodiment 3: The present invention proposes a system for detecting living things in a night vision environment. Figure 3 , which shows a system structure diagram of a living thing detection system in a night vision environment provided by an embodiment of the present invention, the system comprising: The data acquisition module 610 is used to obtain the regional infrared image and regional grayscale image of the monitoring area at each moment in the night vision environment; A suspected living thing region analysis module 620 is used to obtain a suspected living thing region based on the motion information difference and position distribution between pixels in the regional infrared image; The actual living creature region extraction module 630 is used to obtain the motion continuity of each suspected living creature region in the regional infrared image at each moment according to the number of motion direction changes and motion conditions of the same suspected living creature region in the regional infrared image at adjacent moments within the analysis period at each moment; and to select the actual living creature region based on the motion continuity; The living thing detection module 640 is used to obtain the feature recognition vector corresponding to the actual living thing area according to the motion continuity of each actual living thing area in the regional infrared image at each moment and the texture structure of the corresponding area in the regional grayscale image at the same moment; based on the feature recognition vector, detect the type of living thing in the monitoring area under the night vision environment.

[0068] It should be noted that the device provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above. In addition, the system for detecting living things in a night vision environment and the method for detecting living things in a night vision environment provided in the above embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0069] Embodiment 4: This embodiment also provides a computer-readable storage medium, which stores a computer program code. When the computer program code runs on a computer, the computer executes the above-mentioned related method steps to implement a method for detecting living things in a night vision environment provided by the above embodiment.

[0070] Embodiment 5: This embodiment also provides a computer program product. When the computer program product runs on a computer, the computer executes the above-mentioned related steps to implement a method for detecting living things in a night vision environment provided by the above embodiment.

[0071] Among them, the device, computer-readable storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0072] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0073] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0074] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A method for detecting living things in a night vision environment, characterized in that: The method includes: Obtain regional infrared images and regional grayscale images of the monitoring area at each moment in the night vision environment; Obtain the suspected living object area based on the motion information difference and position distribution between pixels in the regional infrared image; Obtaining the motion coherence of each suspected living creature region in the regional infrared image at each moment according to the number of changes in motion direction and motion conditions of the same suspected living creature region in the regional infrared image at adjacent moments within the analysis period at each moment; and screening the actual living creature region based on the motion coherence; According to the motion continuity of each actual living thing area in the regional infrared image at each moment and the texture structure of the corresponding area in the regional grayscale image at the same moment, a feature recognition vector corresponding to the actual living thing area is obtained; based on the feature recognition vector, the type of living thing in the monitoring area under the night vision environment is detected.

2. The method for detecting living things in a night vision environment according to claim 1, characterized in that: The step of obtaining the motion coherence of each suspected living creature region in the regional infrared image at each moment includes: Perform optical flow tracking on the pixel points in the regional infrared image at all times to obtain the optical flow vector of each pixel point in the regional infrared image at each time; Count the number of times that the angle between the optical flow vectors of the centroid of the same suspected living object region in the regional infrared images at adjacent moments within the analysis period at each moment is greater than a preset angle threshold, and record it as the change value of the movement direction of each suspected living object region in the regional infrared images at each moment; Calculate the cumulative sum of the modulus of the optical flow vector of the centroid of the same suspected living object region in the regional infrared image at all times within the analysis period at each moment as the motion trajectory value of each suspected living object region in the regional infrared image at each moment; The distance between the centroids of the same suspected living thing region in the regional infrared image at two end points in the analysis period at each moment is taken as the motion displacement of each suspected living thing region in the regional infrared image at each moment; The ratio of the motion displacement to the motion trajectory value is calculated, the motion direction change value is negatively correlated and mapped, and the product of the mapping result and the ratio is normalized to obtain the motion coherence of each suspected living thing area in the regional infrared image at each moment.

3. The method for detecting living things in a night vision environment according to claim 1, characterized in that: The method for acquiring the feature recognition vector of the actual living object area includes: Determine the mapping area of ​​each actual living object area in the regional infrared image at each moment in the regional grayscale image at the same moment, extract texture features of the mapping area to obtain a gray-level co-occurrence matrix; obtain texture feature indicators of the gray-level co-occurrence matrix, wherein the texture feature indicators include correlation; According to the motion continuity and the correlation of each actual living thing region in the regional infrared image, obtaining a motion feature value corresponding to the actual living thing region; The motion feature value of each actual living thing region in the regional infrared image and the remaining texture feature indicators except the correlation form a feature recognition vector corresponding to the actual living thing region.

4. The method for detecting living things in a night vision environment according to claim 1, characterized in that: The detecting the type of living thing in the monitoring area in the night vision environment based on the feature recognition vector includes: Obtain standard feature vectors for different types of living things; The cosine similarities between the feature recognition vector of each actual living thing region in the regional infrared image at each moment and the standard feature vectors of all types of living things are calculated respectively, and the living thing type corresponding to the maximum cosine similarity is used as the living thing type corresponding to the actual living thing region.

5. The method for detecting living things in a night vision environment according to claim 2, characterized in that: The method for obtaining the suspected living thing area includes: Perform optical flow tracking on the pixel points in the regional infrared image at all times to obtain the moving speed of each pixel point in the regional infrared image at each time; According to the angle of the optical flow vectors, the moving speed difference and the pixel value difference of any two pixel points in the regional infrared image, the difference in motion performance of the corresponding two pixel points is obtained; The connected domain formed by the pixel points corresponding to the motion performance difference less than the preset difference threshold in the regional infrared image is recorded as a local area; the local area in the regional infrared image is subjected to regional growth to obtain a suspected living thing area.

6. The method for detecting living things in a night vision environment according to claim 1, characterized in that: The actual living creature region is a suspected living creature region in which the action continuity in the regional infrared image at each moment is greater than a preset continuity threshold.

7. The method for detecting living things in a night vision environment according to claim 5, characterized in that: The step of performing regional growth on the local region in the regional infrared image to obtain a suspected living object region includes: In the regional infrared image, a local area is selected as a sample area, the sample area is used as a growth point for regional growth, and other local areas except the growth point that meet the preset growth conditions are used as new growth points for regional growth, until the growth is stopped when other local areas except all new growth points do not meet the preset growth conditions, and the corresponding suspected living thing area is obtained; The ratio of the maximum value to the minimum value of the area of ​​any two local regions is recorded as the area ratio; The preset growth conditions include: the shortest distance to the corresponding growth point is less than a preset distance threshold and the area ratio to the corresponding growth point is greater than a preset area difference threshold.

8. The method for detecting living things in a night vision environment according to claim 5, characterized in that: The method for performing optical flow tracing on pixel points in regional infrared images at all times is the Lucas-Kanade algorithm.

9. A device for detecting living things in a night vision environment, characterized in that: The device includes a processor, and when the processor is executed, the steps of a method for detecting living things in a night vision environment according to any one of claims 1 to 8 are implemented.

10. A system for detecting living things in a night vision environment, characterized in that: The system includes: A data acquisition module is used to obtain regional infrared images and regional grayscale images of the monitoring area at each moment in a night vision environment; A suspected living thing area analysis module is used to obtain a suspected living thing area based on the motion information difference and position distribution between pixels in the regional infrared image; The actual living creature region extraction module is used to obtain the motion continuity of each suspected living creature region in the regional infrared image at each moment according to the number of motion direction changes and motion conditions of the same suspected living creature region in the regional infrared image at adjacent moments within the analysis period at each moment; and screen the actual living creature region based on the motion continuity; The living thing detection module is used to obtain the feature recognition vector corresponding to the actual living thing area according to the motion continuity of each actual living thing area in the regional infrared image at each moment and the texture structure of the corresponding area in the regional grayscale image at the same moment; based on the feature recognition vector, detect the type of living thing in the monitoring area under the night vision environment.

Images