A method and device for recognizing personnel behavior based on thermal imaging

Abstract

The application discloses a personnel behavior recognition method and device based on thermal imaging, and the method comprises the following steps: obtaining a to-be-recognized thermal imaging image, wherein the to-be-recognized thermal imaging image is a two-dimensional image; extracting an abnormal pixel point set and a personnel position according to pixel values of each pixel point in the to-be-recognized thermal imaging image; and analyzing the abnormal pixel point set according to the positional relationship between the abnormal pixel point set and the personnel position to obtain a recognition result. The to-be-recognized thermal imaging image is obtained by using the thermal imaging technology, the pixel values of each pixel point in the to-be-recognized thermal imaging image are analyzed and judged based on the characteristics of the thermal imaging, the abnormal pixel point set and the personnel position are obtained, the abnormal pixel point set is further determined in combination with the positional relationship between the abnormal pixel point set and the personnel position, and finally the recognition result is obtained, so that the behavior detection precision of the staff is improved, the misjudgment of the behavior of the staff is reduced, and the accuracy of the recognition result is ensured to a certain extent.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of behavior detection, and particularly relates to a personnel behavior recognition method and device based on thermal imaging. BACKGROUND

[0002] The power distribution station is an important part of the power system, located at the end of the power grid, connected to the transformer substation and connected to various electrical equipment. The power distribution station needs to convert high-voltage power into low-voltage power and deliver it to various power users through power distribution facilities, lines and equipment. The power distribution station also plays a role in stabilizing power supply and ensuring power safety.

[0003] The various behaviors of the staff in the power distribution room may affect the equipment in the power distribution room and cause accidents. How to timely detect the abnormal behavior of the staff is a current problem.

[0004] Currently, the behavior detection of the staff in the power distribution room mainly detects smoking. The related technology detects the staff's smoking by using a traditional camera to sample and using artificial intelligence technology to detect the target and detect cigarettes. However, this detection method has some problems. First, the traditional camera has low pixels due to its own hardware parameters and business data transmission requirements, resulting in a small proportion of pixels of the cigarette in the image. Second, the light environment of the power distribution station is unstable, which affects the imaging effect of the traditional camera. At the same time, the staff's cigarette may be similar to the background features, making it difficult to extract features and separate the foreground and background.

[0005] Patent CN118865248A discloses a bank smoking detection method based on multi-vision data fusion, which includes: training a target detection algorithm model based on deep learning; using the target detection algorithm model to detect whether there is a cigarette in the real-time image containing an effective detection object, when there is, determining whether it intersects or overlaps with the human hand or mouth, and when there is intersection or overlap, marking the effective detection object as a suspicious detection object; obtaining an abnormal area with a temperature higher than a preset temperature in the area containing the effective detection object, aligning the coordinates of the abnormal area and the real-time image determined to have a suspicious detection object, obtaining a fusion image, and then determining whether the suspicious detection object has a smoking behavior. By coordinate alignment, the detection results of the video monitoring camera and the infrared temperature camera are associated and processed, and the depth sorting and tracking algorithm is combined to significantly improve the accuracy of identifying the smoking behavior and reduce false positives.

[0006] In the related technology, how to improve the detection accuracy of staff smoking detection is a current problem to be solved. SUMMARY

[0007] In view of the defects in the prior art, the personnel behavior recognition method based on thermal imaging is provided, comprising:

[0008] Obtain a to-be-recognized thermal imaging image, wherein the to-be-recognized thermal imaging image is a two-dimensional image;

[0009] According to the pixel values of each pixel point in the to-be-recognized thermal imaging image, an abnormal pixel point set and a personnel position are extracted;

[0010] According to the positional relationship between the abnormal pixel point set and the personnel position, the abnormal pixel point set is analyzed to obtain a recognition result.

[0011] According to the above technical solution, the to-be-recognized thermal imaging image is obtained by using thermal imaging technology, the pixel values of each pixel point in the to-be-recognized thermal imaging image are analyzed and judged based on the characteristics of thermal imaging, the abnormal pixel point set and the personnel position are obtained, and then the abnormal pixel point set is further determined in combination with the positional relationship between the abnormal pixel point set and the personnel position, so as to obtain the recognition result. The work personnel behavior detection precision is improved, the misjudgment of the work personnel behavior is reduced, and the accuracy of the recognition result is ensured to a certain extent.

[0012] In a possible implementation, an initial thermal imaging image is obtained;

[0013] According to a preset compression multiple, the initial thermal imaging image is compressed to obtain the to-be-recognized thermal imaging image, which is specifically represented as:

[0014]

[0015] Wherein, O is the compressed to-be-recognized thermal imaging image, O (i,j) is the pixel value of the i-th row and j-th column pixel point in the to-be-recognized thermal imaging image, N is the compression multiple, w is the number of rows in the initial thermal imaging image, h is the number of columns in the initial thermal imaging image, I (i,j) is the pixel value of the i-th row and j-th column pixel point in the initial thermal imaging image.

[0016] In a possible implementation, according to the pixel values of each pixel point in the to-be-recognized thermal imaging image, the abnormal pixel point set and the personnel position are extracted, which specifically includes:

[0017] The pixel values of each pixel point in the to-be-recognized thermal imaging image are obtained in sequence;

[0018] When the pixel value exceeds a preset abnormal threshold value, the pixel point corresponding to the pixel value is an abnormal pixel point in the abnormal pixel point set;

[0019] When the pixel value is within a preset body temperature pixel range, the pixel point corresponding to the pixel value is a pixel point in the personnel position.

[0020] In a possible implementation, the abnormal pixel point set is analyzed according to the positional relationship between the abnormal pixel point set and the personnel position to obtain a recognition result, and the method specifically comprises the following steps.

[0021] The horizontal distances between each abnormal pixel point in the abnormal pixel point set and the personnel position are compared in sequence, and the comparison is specifically represented as:

[0022] Δ = abs(x2-x3)

[0023] Dis m = min(abs(x 1,m -x2), abs(x1-x3))

[0024] wherein Δ is the width of the personnel position in the horizontal position, Dis m is the horizontal distance between the mth abnormal pixel point in the abnormal pixel point set and the personnel position, x 1,m is the coordinate value of the mth abnormal pixel point in the abnormal pixel point set on the x axis, and x2 and x3 are the maximum value and the minimum value of the coordinate values of the personnel position on the x axis, respectively.

[0025] When Dis m < Δ, the mth abnormal pixel point is added to the related point set.

[0026] The recognition result is determined according to the number of related points in the related point set.

[0027] In a possible implementation, the method further comprises the following steps.

[0028] Based on the recognition result, an image group corresponding to the to-be-recognized thermal image is obtained, wherein each image in the image group has a time sequence relationship.

[0029] Abnormal pixel points in each image in the image group are extracted respectively.

[0030] The running track of the abnormal pixel points is obtained in combination with the time sequence relationship of each image in the image group.

[0031] The recognition result is determined or adjusted by analyzing the running track.

[0032] In a possible implementation, the running track of the abnormal pixel points is obtained in combination with the time sequence relationship of each image in the image group, and the method specifically comprises the following steps.

[0033] According to the time sequence relationship of each image in the image group, the coordinates of the abnormal pixel points in the images are normalized in sequence to obtain standard coordinates.

[0034] The standard coordinates of each image in the image group are fused to obtain the running track of the abnormal pixel points.

[0035] In a possible implementation, the running track is analyzed to determine or adjust the recognition result, specifically including:

[0036] The running track is input into a preset track analysis model to obtain an analysis result.

[0037] Based on the analysis result, the recognition result is determined or adjusted.

[0038] In a possible implementation, the track analysis model is determined in the following manner:

[0039] A training data set is obtained, wherein the training data set includes a plurality of running tracks of abnormal pixel points.

[0040] Based on the training data set, a preset classification model is trained to obtain the track analysis model.

[0041] In a possible implementation, the personnel position is obtained in the following manner:

[0042] The to-be-recognized thermal imaging image is convolved and pooled to obtain a first feature map.

[0043] The first feature map is subjected to feature extraction to generate a candidate region.

[0044] The first feature map and the candidate region are analyzed to obtain a region of interest.

[0045] The region of interest is classified by using a classifier to obtain the personnel position.

[0046] The application further discloses a personnel behavior recognition device based on thermal imaging, comprising:

[0047] An image acquisition module is configured to acquire a to-be-recognized thermal imaging image, wherein the to-be-recognized thermal imaging image is a two-dimensional image.

[0048] An abnormality extraction module is configured to extract an abnormal pixel point set and a personnel position according to pixel values of each pixel point in the to-be-recognized thermal imaging image.

[0049] An analysis and recognition module is configured to analyze the abnormal pixel point set according to a positional relationship between the abnormal pixel point set and the personnel position to obtain a recognition result.

[0050] The personnel behavior recognition method and device based on thermal imaging provided by the application have at least the following beneficial effects:

[0051] The heat imaging technology is adopted to obtain a to-be-recognized heat imaging image, pixel values of each pixel point in the to-be-recognized heat imaging image are analyzed and judged based on the characteristics of heat imaging, an abnormal pixel point set and a personnel position are obtained, the abnormal pixel point set is further determined in combination with the positional relationship between the abnormal pixel point set and the personnel position, and a recognition result is obtained, so that the behavior detection precision of the staff is improved, the misjudgment on the behavior of the staff is reduced, and the accuracy of the recognition result is ensured to a certain extent. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 A flowchart of a personnel behavior recognition method based on heat imaging is provided for the embodiment of the present application.

[0053] Figure 2 A flowchart of a personnel behavior recognition method based on heat imaging is provided for the embodiment of the present application.

[0054] Figure 3 A structural block diagram of a personnel behavior recognition method device based on heat imaging is provided for the embodiment of the present application.

[0055] Among them, 201, image acquisition module;202, abnormal extraction module;203, analysis and identification module. DETAILED DESCRIPTION

[0056] In order to better understand the above technical solutions, the above technical solutions will be described in detail in combination with the drawings of the specification and specific embodiments. Obviously, the described embodiments are only part of the embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor belong to the scope of protection of the present application.

[0057] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates otherwise. "Multiple" generally includes at least two.

[0058] It should also be noted that the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the goods or devices including a series of elements not only include those elements, but also include other elements not explicitly listed, or include elements inherent to such goods or devices. Without more limitation, the element defined by the sentence "including a" does not exclude the presence of other identical elements in the goods or devices including the element.

[0059] The power distribution station plays an extremely important role in the power system. The power distribution station converts high-voltage electric energy into low-voltage electric energy suitable for user use through power transformation equipment, and ensures that users can use electricity normally. In modern society, various facilities such as hospitals, schools, transportation hubs and the like all need power supply, and the power distribution station is an important part of urban infrastructure construction. In addition, the power distribution station can automatically adjust the voltage to ensure the stability and quality of power supply when the user power consumption changes dramatically. In addition, the protection device in the power distribution station can detect power system faults and take timely measures to avoid power accidents, thereby ensuring the safety of electricity use.

[0060] The safety of the power distribution station house where the power distribution station is located not only directly affects the safety of the power distribution facility, but also relates to the user's electricity experience and electricity safety. The security detection of the power distribution station house has always been the focus of security in the power system, and the increasingly improved security detection system ensures the safety of the equipment and personnel in the power distribution room at all times.

[0061] With the development of the monitoring field, various monitoring devices are arranged in the power distribution station house to monitor the conditions of the equipment and staff in the room, and the monitoring devices are used to monitor the abnormal behavior of the staff. Smoking in the power distribution room by staff is a big safety hazard, and how to improve the detection accuracy of the smoking behavior of staff in the power distribution station house based on the existing ordinary camera is a problem to be solved at present.

[0062] The problems affecting the detection accuracy of smoking behavior mainly focus on the following three aspects:

[0063] The first aspect is that the detection target of the smoking behavior accounts for a small proportion in the image, which may cause low detection precision. In related technologies, the feature extraction network is often modified to increase the network depth to improve the feature extraction effect or modify the input picture size to improve the target pixel, thereby improving the detection precision of the target cigarette. However, this way has high limitations and increases the model parameter amount, which significantly reduces the detection speed.

[0064] The second aspect is that the detection image effect of personnel smoking is easily affected by visible light, causing inconsistent imaging effects of each power distribution station house, and thus the recognition precision of the algorithm for the detection images with different imaging effects is inconsistent. Related technologies mainly increase pictures in different light conditions as training data, learn the target features under different lights in the model training process, to improve the generalization ability of the model, or modify the model structure to make the model obtain better generalization ability. However, the method of improving sample diversity has certain limitations. In the power distribution station house, various situations may occur, and data collection is difficult to cover everything, so the scheme of improving sample diversity has too strong limitations and data collection often requires a large manual cost.

[0065] In a third aspect, the detection target of a person smoking is easily fused with the background based on the influence of visible light or other factors, and the detection precision is low due to the few characteristics that can be used. In related technologies, this problem is often ignored, resulting in extremely poor detection effect in some scenarios.

[0066] To solve the above problems, the embodiment of the present application provides a personnel behavior recognition method based on thermal imaging. By detecting the different temperatures of the target in the thermal imaging image, different color characteristics are reflected, and temperature abnormal points are recognized, so as to find the position of the cigarette ignition in the image. Similarly, by using the color difference of different temperatures in imaging, the image feature difference of the foreground and background can be ignored, so as to extract the position of the cigarette ignition.

[0067] In addition, for the false detection problem of temperature abnormal points caused by non-personal smoking in the thermal imaging image, the present application determines whether the temperature abnormality is caused by smoking by detecting whether there is a person in the adjacent position of the temperature abnormal point. Since the temperature of the power station room is stable and lower than the body temperature of the human body all year round, the shape of the human body will be reflected in the thermal imaging image. Therefore, the detection of whether there is a person in the adjacent position of the temperature abnormal point can be used to preliminarily screen the false detection points and reduce the false alarm rate of the detection system. At the same time, the present application constructs the trajectory data of the temperature abnormal points by the positions of the temperature abnormal points in the continuous time period, and determines whether the temperature abnormal point is caused by personal smoking by the trajectory data of the temperature abnormal points.

[0068] The present application solves the problem of low detection precision of personal smoking in the power station room caused by unstable imaging effect, small detection target, and unobvious difference between the detection target feature and the background feature. At the same time, by determining the correlation between the temperature abnormal point and the person and classifying the trajectory of the temperature abnormal point, the probability of false detection of personal smoking in the power station room is reduced.

[0069] As shown in Figure 1 The embodiment of the present application provides a personnel behavior recognition method based on thermal imaging, and the specific steps are as follows:

[0070] S101: acquiring a thermal imaging image to be recognized.

[0071] Specifically, the thermal imaging image to be recognized is a two-dimensional image acquired in a target space. In the embodiment provided by the present application, the target space is a power station room, and in other embodiments, it can be other spaces or areas where personnel behavior needs to be recognized, which is not limited. In this example, the thermal imaging image to be recognized is acquired by a fixed camera in the power station room. The thermal imaging image to be recognized is determined by the following method:

[0072] An initial thermal imaging image is acquired; the initial thermal imaging image is compressed according to a preset compression multiple to obtain the thermal imaging image to be recognized, which is specifically represented as:

[0073]

[0074] wherein, O is the compressed to-be-recognized thermal imaging image, O (i,j) is a pixel value of an i-th row and a j-th column pixel point in the to-be-recognized thermal imaging image, N is a compression multiple, w is a number of rows in the initial thermal imaging image, h is a number of columns in the initial thermal imaging image, I (i,j) is a pixel value of an i-th row and a j-th column pixel point in the initial thermal imaging image.

[0075] It can be understood that if the pixel scanning filtering on the complete initial thermal imaging image occupies a large amount of computing resources and reduces the detection efficiency. In order to improve the detection efficiency, the initial thermal imaging image is compressed. In a specific example, the compression multiple is set to 3, that is, 9 pixel points in a 3x3 matrix in the initial thermal imaging image are compressed into 1 pixel, which can theoretically compress the detection time to 1 / 9 of the original. The specific compression process is as follows:

[0076]

[0077] wherein, O is the compressed to-be-recognized thermal imaging image, O (i,j is a pixel value of an i-th row and a j-th column pixel point in the to-be-recognized thermal imaging image, w is a number of rows in the initial thermal imaging image, h is a number of columns in the initial thermal imaging image, I (i,j is a pixel value of an i-th row and a j-th column pixel point in the initial thermal imaging image.

[0078] The above compression process takes the maximum pixel value of 9 pixel points in a 3x3 matrix in the initial thermal imaging image as the pixel value of the compressed 1 pixel.

[0079] S102: According to the pixel values of each pixel point in the to-be-recognized thermal imaging image, an abnormal pixel point set and a personnel position are extracted.

[0080] Specifically, the pixel values of each pixel point in the to-be-recognized thermal imaging image are obtained in sequence. When the pixel value exceeds a preset abnormal threshold, the pixel point corresponding to the pixel value is an abnormal pixel point in the abnormal pixel point set. When the pixel value is within a preset body temperature pixel range, the pixel point corresponding to the pixel value is a pixel point in the personnel position.

[0081] In a specific embodiment, due to the characteristics of the image formed by thermal imaging, the higher the temperature, the higher the area brightness of the pixel point on the to-be-identified thermal image, and the closer the pixel value of the pixel point to 255. The surface temperature of the lit cigarette end is between 200° and 300°, and the center temperature can reach 700° to 800°. The temperature difference makes the pixel value of the pixel point corresponding to the burning cigarette in the to-be-identified thermal image much larger than the pixel value of the pixel point in other areas, so that the abnormal threshold can be set, and when the pixel value of the pixel point on the to-be-identified thermal image is greater than the set abnormal threshold, the pixel point is considered as a temperature abnormal point, i.e. an abnormal pixel point.

[0082] It can be understood that the detection of abnormal pixel points can be completed on the to-be-identified thermal image, or on the initial thermal image. If it is completed on the to-be-identified thermal image, the position of the pixel abnormal point on the initial thermal image needs to be determined according to the corresponding relationship between the pixel points in the initial thermal image and the to-be-identified thermal image.

[0083] The corresponding relationship between the pixel points in the initial thermal image and the to-be-identified thermal image is specifically represented as:

[0084] I (i,j) (i′ , j′)

[0085] i∈(i′*3,i′*3+1,i′*3+2),j∈(j′*3,j′*3+1,j′*3+2)

[0086] i′∈(0,1,…w),j′∈(0,1,…,h)

[0087] wherein, I (i,j) is the pixel point in the i-th row and the j-th column of the initial thermal image, O (i′,j′) is the pixel point in the i'-th row and the j'-th column of the to-be-identified thermal image, w is the number of rows in the initial thermal image, and h is the number of columns in the initial thermal image.

[0088] In a specific embodiment, because the temperature of the power distribution station room is kept below 35 degrees Celsius all year round, there is a certain difference with the body temperature, so in general cases, the to-be-identified thermal image can clearly show the shape of the personnel by judging the pixel value on the pixel point.

[0089] ​In another specific embodiment, the personnel position can be determined by target detection on the to-be-identified thermal imaging image using a target detection model. The specific detection process of the target detection model is as follows: the to-be-identified thermal imaging image is convolved and pooled to obtain a first feature map; the first feature map is feature-extracted to generate a candidate region; the first feature map and the candidate region are analyzed to obtain a region of interest; and a classifier is used to classify the region of interest to obtain the personnel position.

[0090] In one specific example, the above-mentioned target detection model is a regions with CNN features (R-CNN), and the target detection model includes a feature extraction network, a recommended candidate region, a region of interest pooling layer, and a classification regression layer. The above-mentioned feature extraction network convolves and pools the to-be-identified thermal imaging image to obtain a first feature map, and transmits the first feature map to the recommended candidate region and the region of interest pooling layer. The above-mentioned recommended candidate region feature-extracts the received first feature map to generate a candidate region, and transmits the generated candidate region to the region of interest pooling layer. The above-mentioned region of interest pooling layer is used to receive the first feature map and the candidate region to obtain a region of interest with a fixed size, and transmits the obtained region of interest with a fixed size to the classification regression layer. The above-mentioned classification regression layer is used to classify the received region of interest to obtain the personnel position. The classification regression layer is a kind of classifier.

[0091] The above-mentioned target detection model can be obtained by collecting data including to-be-identified thermal imaging images of workers from a power distribution station, manually labeling the data, and then iteratively training the target detection model.

[0092] S103: According to the positional relationship between the abnormal pixel point set and the personnel position, the abnormal pixel point set is analyzed to obtain an identification result.

[0093] The horizontal distances between each abnormal pixel point in the abnormal pixel point set and the personnel position are compared in sequence, and are specifically represented as follows:

[0094] Δ = abs(x2-x3)

[0095] Dis m = min(abs(x 1,m -x2), abs(x1-x3))

[0096] Where Δ is the width of the personnel position in the horizontal position, Dis m is the horizontal distance between the mth abnormal pixel point in the abnormal pixel point set and the personnel position, x 1,m is the coordinate value of the mth abnormal pixel point in the abnormal pixel point set on the x-axis, and x2 and x3 are the maximum and minimum values of the coordinate values of the personnel position on the x-axis, respectively.

[0097] When Dis m <Δ, the mth abnormal pixel point is added to the relevant point set; according to the number of relevant points in the relevant point set, the recognition result is determined.

[0098] By judging Dis m , the mth abnormal pixel point is added to the relevant point set; according to the number of relevant points in the relevant point set, the recognition result is determined. m <Δ, indicating that the mth abnormal pixel point is related to the personnel position, and the abnormal pixel point is added to the relevant point set, if Dis m ≥Δ, indicating that the mth abnormal pixel point is not related to the personnel position, and the abnormal pixel point is not added to the relevant point set. After all the abnormal pixel points are judged, the relevant point set is obtained, if the relevant set is not empty, it indicates that there is an abnormal pixel point related to the personnel position, if the relevant set is empty, it indicates that there is no abnormal pixel point related to the personnel position.

[0099] In the examples provided by the present application, the number of relevant points in the relevant point set is judged with zero to determine the recognition result, in other examples, the comparison value can be adjusted according to actual needs, for example, the number of relevant points in the relevant point set is judged with 1, if there is only one relevant point, it is considered that the relevant point may be misjudged, then it indicates that there is no abnormal pixel point related to the personnel position. The number of relevant points in the relevant point set and the number of abnormal pixel points in the abnormal pixel point set can also be judged by the ratio. For example, the ratio = the number of relevant points in the relevant point set / the number of abnormal pixel points in the abnormal pixel point set, when the ratio is less than the ratio preset value, it indicates that only a small part of the relevant points exists in the abnormal pixel points, then it indicates that there is no abnormal pixel point related to the personnel position; when the ratio is greater than or equal to the ratio preset value, it indicates that most of the points in the abnormal pixel points are relevant points, then it indicates that there is an abnormal pixel point related to the personnel position. For example, the ratio preset value is 50%, in other embodiments, the ratio preset value can be adjusted according to actual needs, which is not limited.

[0100] In a specific example, after obtaining the personnel position, the relative distance between the personnel position and the pixel abnormal point is calculated, and the pixel abnormal point is considered to be related to the personnel position if the distance is less than the set threshold, otherwise, the pixel abnormal point is considered to be not related to the personnel position. The set threshold is Δ, which is determined by the short side of the detection frame corresponding to the detected personnel position, so Δ is dynamically changed. In this example, x2 and x3 are the horizontal coordinates of the top left corner and the bottom right corner of the detection frame corresponding to the personnel position.

[0101] The above-mentioned personnel behavior recognition method based on thermal imaging further comprises:

[0102] Based on the recognition result, an image group corresponding to the to-be-recognized thermal image is acquired, wherein each image in the image group has a time sequence relationship; abnormal pixel points in each image in the image group are extracted respectively; a running track of the abnormal pixel points is obtained in combination with the time sequence relationship of each image in the image group; and the running track is analyzed to determine or adjust the recognition result.

[0103] In a specific embodiment, when the recognition result is that there is abnormal behavior, the recognition result needs to be reviewed, that is, an image group corresponding to the to-be-recognized thermal image is acquired, and a running track obtained based on abnormal pixel points in each image in the image group is analyzed. If the running track indicates that there is also abnormal behavior, it is determined that the recognition result is that there is abnormal behavior, and if the running result indicates that there is no abnormal behavior, the recognition result is adjusted to be that there is no abnormal behavior. If the recognition result is that there is no abnormal behavior, the image group corresponding to the to-be-recognized thermal image does not need to be acquired, that is, the recognition result does not need to be reviewed. In the examples provided in the present application, the above-mentioned abnormal behavior refers to the behavior of smoking in the power distribution station, and in other scenarios, it can be other non-compliant behaviors, which are not limited.

[0104] Further, according to the time sequence relationship of each image in the image group, the coordinates of the abnormal pixel points in the images are normalized in sequence to obtain standard coordinates; and the standard coordinates of each image in the image group are fused to obtain the running track of the abnormal pixel points.

[0105] In a specific example, according to the acquisition frequency of the to-be-recognized thermal image, all images within 10 seconds are continuously tracked and collected to obtain the image group, and the pixel abnormal points on each to-be-recognized thermal image in the image group are detected. The coordinates of the pixel abnormal points on each to-be-recognized thermal image are normalized to normalize the coordinates of the pixel abnormal points to 0-1 to obtain standard coordinates. The running track of the pixel abnormal points is constructed according to the time sequence relationship between each image in the image group.

[0106] Further, the running track is input into a preset track analysis model to obtain an analysis result; and based on the analysis result, the recognition result is determined or adjusted.

[0107] In a specific embodiment, the running track is input into a preset track analysis model for classification to obtain an analysis result, which is used to reflect whether the pixel abnormal points are caused by the smoking behavior of the staff. If the analysis result indicates that the pixel abnormal points are caused by the smoking behavior of the staff, it is determined that the recognition result is that there is abnormal behavior, and if the analysis result indicates that the pixel abnormal points are not caused by the smoking behavior of the staff, the recognition result is adjusted to be that there is no abnormal behavior.

[0108] The track analysis model is determined by the following way:

[0109] Obtain a training data set, wherein the training data set comprises a plurality of running trajectories of abnormal pixel points; and train a preset classification model based on the training data set to obtain a trajectory analysis model.

[0110] In one specific example, the trajectory analysis model is a machine learning classification model, which needs to be trained to obtain the trajectory analysis model. The training data set can be the running trajectory in the simulated smoking behavior, or the processed data collected in the power distribution station. The processed training data set is input into the machine learning classification model for training. For example, the coordinates of the pixel abnormal points in all the to-be-identified thermal imaging images within 10s are obtained in time sequence as the data in the training data set.

[0111] With reference to Figure 2 The present application extracts abnormal pixel points and personnel positions from the to-be-identified thermal imaging image based on the to-be-identified thermal imaging image, analyzes and judges the correlation between the personnel positions and the abnormal pixel points, and ends the judgment process if the personnel positions and the abnormal pixel points are not correlated, indicating that there is no abnormal behavior. If the personnel positions and the abnormal pixel points are correlated, indicating that there is an abnormal behavior, the result needs to be further reviewed, the image group corresponding to the to-be-identified thermal imaging image is obtained, and the running trajectory of the abnormal pixel points in the image group is analyzed. If the running trajectory and the abnormal behavior are matched, it is determined that the recognition result is that there is an abnormal behavior. If the running trajectory and the abnormal behavior are not matched, the recognition result is adjusted to be that there is no abnormal behavior, and the judgment process is ended.

[0112] With reference to Figure 3 The present application provides a personnel behavior recognition device based on thermal imaging, comprising:

[0113] An image acquisition module 201 is configured to acquire a to-be-identified thermal imaging image, wherein the to-be-identified thermal imaging image is a two-dimensional image.

[0114] An abnormality extraction module 202 is configured to extract an abnormal pixel point set and a personnel position according to the pixel values of each pixel point in the to-be-identified thermal imaging image.

[0115] An analysis and recognition module 203 is configured to analyze the abnormal pixel point set according to the positional relationship between the abnormal pixel point set and the personnel position, and obtain a recognition result.

[0116] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the described modules can refer to the corresponding process in the foregoing method embodiments, which will not be described herein.

[0117] While the preferred embodiments of the application have been described, additional variations and modifications can be made to these embodiments by those skilled in the art once they have the benefit of the present disclosure without departing from the spirit and scope of the application. Accordingly, it is intended that the appended claims include all such variations and modifications as fall within the scope of the present application. It is apparent that those skilled in the art can modify and adapt the present application in various ways without departing from the spirit and scope of the present application. It is therefore intended that the present application encompass all such modifications and variations as fall within the scope of the claims and their equivalents.

Claims

1. A method for recognizing human behavior based on thermal imaging, characterized in that, include: Acquire a thermal imaging image to be identified, wherein the thermal imaging image to be identified is a two-dimensional image acquired in the target space, which is captured by a fixed camera in the power distribution room, and the target space is the power distribution room; The pixel values ​​of each pixel in the thermal imaging image to be identified are obtained sequentially; when the pixel value exceeds the preset abnormal threshold, the pixel corresponding to the pixel value is an abnormal pixel in the abnormal pixel set; when the pixel value is within the preset body temperature pixel range, the pixel corresponding to the pixel value is a pixel in the personnel location. The horizontal distances between each abnormal pixel in the abnormal pixel set and the person's location are compared sequentially. When Dis m When the value is less than Δ, the m-th abnormal pixel is added to the relevant point set, where Δ is the width of the person's position in the horizontal direction, and Dis is the width of the person's position in the horizontal direction. m The horizontal distance between the m-th abnormal pixel in the abnormal pixel set and the person's location; the recognition result is determined based on the number of relevant points in the relevant point set; Based on the recognition results, the image group corresponding to the thermal image to be identified is obtained, wherein the images in the image group have a temporal relationship; the abnormal pixels in each image in the image group are extracted respectively; and the trajectory of the abnormal pixels is obtained by combining the temporal relationship of the images in the image group. When the identification result indicates the presence of abnormal behavior, the identification result is reviewed, the image group corresponding to the thermal image to be identified is obtained, and the running trajectory obtained based on the abnormal pixels of each image in the image group is analyzed. If the running trajectory also indicates the presence of abnormal behavior, the identification result is determined to be the presence of abnormal behavior. If the running trajectory indicates the absence of abnormal behavior, the identification result is adjusted to the absence of abnormal behavior.

2. The person behavior recognition method based on thermal imaging as described in claim 1, characterized in that, The thermal imaging image to be identified is determined in the following way: Acquire initial thermal imaging images; The initial thermal imaging image is compressed according to a preset compression factor to obtain the thermal imaging image to be identified, specifically as follows: ; Where O is the compressed thermal imaging image to be identified, O (i,j) Let be the pixel value of the i-th row and j-th column of the thermal image to be identified, N be the compression factor, w be the number of rows in the initial thermal image, h be the number of columns in the initial thermal image, and I be the pixel value of the i-th row and j-th column of the thermal image to be identified. (i,j) The pixel value of the pixel in the i-th row and j-th column of the initial thermal image.

3. The person behavior recognition method based on thermal imaging as described in claim 1, characterized in that, By combining the temporal relationships of the images in the image group, the trajectory of the abnormal pixels is obtained, specifically including: Based on the temporal relationship of each image in the image group, the coordinates of abnormal pixels in the images are normalized sequentially to obtain standard coordinates; By fusing the standard coordinates of each image in the image group, the trajectory of the abnormal pixel is obtained.

4. The person behavior recognition method based on thermal imaging as described in claim 1, characterized in that, Analyze the trajectory to determine or adjust the recognition results, specifically including: Input the running trajectory into the preset trajectory analysis model to obtain the analysis results; Based on the analysis results, determine or adjust the identification results.

5. The person behavior recognition method based on thermal imaging as described in claim 4, characterized in that, The trajectory analysis model is determined in the following way: Obtain the training dataset, which includes the running trajectories of multiple abnormal pixels; Based on the training dataset, a pre-defined classification model is trained to obtain a trajectory analysis model.

6. A human behavior recognition device based on thermal imaging, characterized in that, The method for human behavior recognition based on thermal imaging as described in any one of claims 1-5 includes: The image acquisition module is used to acquire a thermal imaging image to be identified, wherein the thermal imaging image to be identified is a two-dimensional image; The anomaly extraction module is used to extract the set of abnormal pixels and the location of personnel based on the pixel values ​​of each pixel in the thermal imaging image to be identified. The analysis and recognition module is used to analyze the abnormal pixel set based on the positional relationship between the abnormal pixel set and the personnel location, and obtain the recognition result.

Images