Personnel behavior identification method and device based on thermal imaging

Abstract

The invention discloses a personnel behavior recognition method and device based on thermal imaging, and the method comprises the steps: obtaining a to-be-recognized thermal imaging image which is a two-dimensional image; extracting an abnormal pixel point set and a personnel position according to the pixel value of each pixel point in the thermal imaging image to be identified; according to the position relation between the abnormal pixel point set and the personnel position, the abnormal pixel point set is analyzed, and an identification result is obtained. A thermal imaging technology is adopted to obtain a thermal imaging image to be identified, based on the characteristics of thermal imaging, the pixel value of each pixel point in the thermal imaging image to be identified is analyzed and judged to obtain an abnormal pixel point set and a personnel position, and then the personnel position is identified by combining the position relation between the abnormal pixel point set and the personnel position. And the abnormal pixel point set is further determined to obtain the recognition result, so that the staff behavior detection precision is improved, the erroneous judgment on the staff behavior is reduced, and the accuracy of the recognition result is ensured to a certain extent.

Description

Technical Field

[0001] The present invention belongs to the technical field of behavior detection, and in particular relates to a method and device for identifying human behavior based on thermal imaging. Background Art

[0002] The distribution station is an important part of the power system, located at the end of the power grid, connected to the substation above and various power-consuming equipment below. The distribution station needs to convert high-voltage electricity into low-voltage electricity and send it to various power users through distribution facilities, lines, and equipment. The distribution station also has the function of stabilizing power supply and ensuring power safety.

[0003] Various behaviors of staff in the distribution room may affect the equipment in the distribution room and cause unexpected situations. How to detect abnormal behaviors of staff in time is a current problem.

[0004] At present, the behavior detection of staff in the distribution room is mainly to detect smoking. The relevant technology detects the smoking of staff by using traditional cameras for sampling, using artificial intelligence technology for target detection, and detecting cigarettes. However, there are some problems with this detection method. First, due to its own hardware parameters and business data transmission requirements, the pixels of traditional cameras are not high, resulting in a small proportion of cigarette pixels in the imaging. Secondly, the lighting environment of the distribution station is unstable, resulting in the imaging effect of traditional cameras being affected by light. At the same time, the cigarettes of staff will be close to or similar to the background features, making feature extraction difficult and unable to separate the foreground and background.

[0005] Patent CN118865248A discloses a bank smoking detection method based on multi-visual data fusion, including: training a target detection algorithm model based on deep learning; using the target detection algorithm model to detect whether there is a cigarette in the real-time image containing the effective detection object, and if it exists, determine whether it crosses or overlaps with the human hand or mouth, and if there is a cross or overlap, mark the effective detection object as a suspicious detection object; obtain an abnormal area with a temperature higher than a preset temperature in the area containing the effective detection object, align the coordinates of the abnormal area with the real-time image where the suspicious detection object is determined, obtain a fused image, and then determine whether the suspicious detection object has smoking behavior. The detection results of the video surveillance camera and the infrared temperature camera are processed by coordinate alignment association, and then combined with the deep sorting tracking algorithm, the accuracy of identifying smoking behavior can be significantly improved and misjudgment can be reduced.

[0006] In the related technologies, how to improve the detection accuracy of staff smoking detection is a problem that needs to be solved at present. Summary of the invention

[0007] In view of the defects existing in the above-mentioned prior art, the present invention provides a method for identifying personnel behavior based on thermal imaging, comprising:

[0008] Acquire a thermal imaging image to be identified, wherein the thermal imaging image to be identified is a two-dimensional image;

[0009] According to the pixel value of each pixel in the thermal imaging image to be identified, the abnormal pixel set and the personnel position are extracted;

[0010] According to the positional relationship between the abnormal pixel point set and the personnel position, the abnormal pixel point set is analyzed to obtain the recognition result.

[0011] It can be seen from the above technical solution that by adopting thermal imaging technology to obtain the thermal imaging image to be identified, based on the characteristics of thermal imaging, the pixel value of each pixel point in the thermal imaging image to be identified is analyzed and judged to obtain the abnormal pixel point set and the personnel position, and then combined with the positional relationship between the abnormal pixel point set and the personnel position, the abnormal pixel point set is further determined to obtain the recognition result, which improves the accuracy of staff behavior detection while reducing the misjudgment of staff behavior, and ensures the accuracy of the recognition result to a certain extent.

[0012] In one possible implementation, acquiring an initial thermal imaging image;

[0013] According to the preset compression multiple, the initial thermal imaging image is compressed to obtain the thermal imaging image to be identified, which is specifically expressed as:

[0014]

[0015] Among them, O is the compressed thermal imaging image to be identified, O (i,j) is the pixel value of the pixel point in the i-th row and j-th column of the thermal imaging image to be identified, N is the compression factor, w is the number of rows in the initial thermal imaging image, h is the number of columns in the initial thermal imaging image, I (i,j) is the pixel value of the pixel at the i-th row and j-th column in the initial thermal imaging image.

[0016] In a possible implementation, extracting an abnormal pixel set and a person's position according to the pixel value of each pixel in the thermal imaging image to be identified specifically includes:

[0017] Sequentially obtain the pixel value of each pixel point in the thermal imaging image to be identified;

[0018] When the pixel value exceeds the preset abnormal threshold, the pixel point corresponding to the pixel value is an abnormal pixel point in the abnormal pixel point set;

[0019] When the pixel value is within the preset body temperature pixel range, the pixel point corresponding to the pixel value is the pixel point in the person's position.

[0020] In a possible implementation, the abnormal pixel set is analyzed according to the positional relationship between the abnormal pixel set and the position of the person to obtain the recognition result, which specifically includes:

[0021] The horizontal distances between each abnormal pixel point in the abnormal pixel point set and the personnel position are compared in turn, which can be specifically expressed as:

[0022] Δ=abs(x2-x3)

[0023] Dis m =min(abs(x 1,m -x2),abs(x1-x3))

[0024] Among them, Δ is the width of the personnel position in the horizontal position, Dis m is the horizontal distance between the mth abnormal pixel point in the abnormal pixel point set and the person’s position, x 1,m is the coordinate value of the mth abnormal pixel in the abnormal pixel set on the x-axis, and x2 and x3 are the maximum and minimum values ​​of the coordinate values ​​of the personnel position on the x-axis respectively;

[0025] When Dis m <Δ, add the mth abnormal pixel point to the relevant point set;

[0026] The recognition result is determined according to the number of related points in the related point set.

[0027] In a possible implementation, the method further includes:

[0028] Based on the recognition result, an image group corresponding to the thermal imaging to be recognized is obtained, wherein each image in the image group has a time sequence relationship;

[0029] Extract abnormal pixels of each image in the image group respectively;

[0030] Combining the temporal relationship of each image in the image group, the running trajectory of the abnormal pixel point is obtained;

[0031] Analyze the running trajectory and confirm or adjust the recognition results.

[0032] In a possible implementation, combining the temporal relationship of each image in the image group to obtain the running trajectory of the abnormal pixel point specifically includes:

[0033] According to the time sequence relationship of each image in the image group, the coordinates of abnormal pixel points in the image are normalized in turn to obtain standard coordinates;

[0034] The standard coordinates of each image in the image group are fused to obtain the running trajectory of the abnormal pixel points.

[0035] In a possible implementation, analyzing the running trajectory and determining or adjusting the recognition result specifically includes:

[0036] Input the running trajectory into the preset trajectory analysis model to obtain the analysis result;

[0037] Based on the analysis results, the recognition results are determined or adjusted.

[0038] In one possible implementation, the trajectory analysis model is determined in the following manner:

[0039] Obtaining a training data set, wherein the training data set includes running trajectories of a plurality of abnormal pixel points;

[0040] Based on the training data set, the preset classification model is trained to obtain the trajectory analysis model.

[0041] In a possible implementation, the personnel position is obtained in the following manner:

[0042] Convolution and pooling are performed on the thermal imaging image to be identified to obtain a first feature map;

[0043] Perform feature extraction on the first feature map to generate a candidate region;

[0044] Analyze the first feature map and the candidate region to obtain the region of interest;

[0045] Use the classifier to classify the area of ​​interest and obtain the location of the person.

[0046] The present invention also discloses a personnel behavior recognition device based on thermal imaging, comprising:

[0047] An image acquisition module is used to acquire a thermal imaging image to be identified, wherein the thermal imaging image to be identified is a two-dimensional image;

[0048] An abnormality extraction module, used to extract abnormal pixel points and personnel positions according to the pixel values ​​of each pixel point in the thermal imaging image to be identified;

[0049] The analysis and recognition module is used to analyze the abnormal pixel point set according to the positional relationship between the abnormal pixel point set and the personnel position to obtain the recognition result.

[0050] The present invention provides a method and device for identifying human behavior based on thermal imaging, which has at least the following beneficial effects:

[0051] By using thermal imaging technology to obtain the thermal imaging image to be identified, based on the characteristics of thermal imaging, the pixel value of each pixel in the thermal imaging image to be identified is analyzed and judged to obtain the abnormal pixel set and the personnel position. Then, combined with the positional relationship between the abnormal pixel set and the personnel position, the abnormal pixel set is further determined to obtain the identification result. This improves the accuracy of staff behavior detection while reducing the misjudgment of staff behavior, and ensures the accuracy of the identification result to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A flowchart of a method for identifying human behavior based on thermal imaging provided by an embodiment of the present invention;

[0053] Figure 2 A flow chart of a method for identifying personnel behavior based on thermal imaging provided by an embodiment of the present invention;

[0054] Figure 3 This is a structural block diagram of a method and device for identifying personnel behavior based on thermal imaging provided in an embodiment of the present invention.

[0055] Among them, 201 is an image acquisition module; 202 is an anomaly extraction module; 203 is an analysis and recognition module. DETAILED DESCRIPTION

[0056] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0057] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two.

[0058] It should also be noted that the term "includes", "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, so that a commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprising a ..." do not exclude the existence of other identical elements in the commodity or device including the elements.

[0059] Distribution stations play an extremely important role in the power system. Distribution stations convert high-voltage electricity into low-voltage electricity suitable for users through transformers to ensure that users can use electricity normally. In modern society, various facilities such as hospitals, schools, and transportation hubs require electricity supply. Distribution stations are an important part of urban infrastructure construction. In addition, distribution stations can also automatically adjust the voltage to ensure the stability and quality of power supply when the user's power consumption changes drastically. In addition, the protection devices in the distribution stations can detect power system faults and take timely measures to avoid power accidents, thereby ensuring the safety of electricity use.

[0060] The safety of the distribution station where the power distribution station is located not only directly affects the safety of the power distribution facilities, but also affects the user's power experience and safety. The security detection of the distribution station has always been the focus of security in the power system. The increasingly improved security detection system guarantees the safety of the equipment and personnel in the distribution room at all times.

[0061] With the development of monitoring, a variety of monitoring equipment is deployed in the power distribution room to monitor the equipment and staff in the room, and to monitor the abnormal behavior of the staff. Smoking in the power distribution room is a big safety hazard. How to improve the detection accuracy of smoking behavior of staff in the power distribution room based on the existing ordinary cameras is a problem that needs to be solved.

[0062] The problems that affect the accuracy of smoking behavior detection are mainly concentrated in the following three aspects:

[0063] First, since the detection target of smoking behavior accounts for a small proportion of the image, there will be a problem of low detection accuracy. In related technologies, the detection accuracy of the target cigarette is often improved by modifying the feature extraction network, increasing the network depth to improve the feature extraction effect, or modifying the input image size to increase the target pixel. However, this method has high limitations, increases the number of model parameters, and significantly reduces the detection speed.

[0064] Secondly, the detection image effect of people smoking is easily affected by visible light, resulting in inconsistent imaging effects in various distribution substations, which in turn makes the algorithm's recognition accuracy for detection images with different imaging effects inconsistent. Related technologies mainly increase pictures of different lighting conditions as training data, and learn target features under different lighting conditions during model training to improve the generalization ability of the model, or modify the model structure based on the model structure to enable the model to obtain better generalization ability. However, the approach to improving sample diversity has certain limitations. In distribution substations, various situations may occur, and data collection is difficult to cover all aspects. Therefore, the solution to improving sample diversity is too limited and data collection often requires a large labor cost.

[0065] Thirdly, due to the influence of visible light or other factors, the detection target of people smoking is easily integrated with the background, and there are few available features, resulting in low detection accuracy. This problem is often ignored in related technologies, resulting in extremely poor detection effects in some scenarios.

[0066] In order to solve the above problems, the embodiment of the present invention provides a method for identifying human behavior based on thermal imaging, which detects the characteristics of different colors reflected by different temperatures of the target in the thermal imaging image, identifies the temperature abnormality point, and thus finds the position of the cigarette lit in the image. Similarly, by using the different colors in the imaging due to different temperatures, the image feature differences between the foreground and the background can be ignored, thereby extracting the position of the cigarette lit.

[0067] In addition, in order to solve the problem of misdetection of abnormal temperature points in thermal imaging images that are not caused by smoking, the present invention determines whether the temperature anomaly is caused by smoking by detecting whether there are people in the vicinity of the temperature anomaly point. Since the temperature of the distribution station is relatively stable all year round and lower than the human body temperature, the shape of people will be reflected in the thermal imaging image. The presence of people in the vicinity of the temperature anomaly point can be used to preliminarily screen the misjudged points and reduce the false alarm rate of the detection system. At the same time, the present invention constructs the trajectory data of the temperature anomaly point through the position of the temperature anomaly point in continuous time periods, and determines whether the temperature anomaly point is caused by smoking through the trajectory data of the temperature anomaly point.

[0068] The present invention solves the problem of low accuracy in detecting people smoking in distribution substations caused by unstable imaging effects, small detection targets, unclear differences between detection target features and background features, and at the same time reduces the probability of false detection of people smoking in distribution substations by determining the correlation between abnormal temperature points and people and classifying the trajectories of abnormal temperature points.

[0069] like Figure 1 As shown, the embodiment of the present invention provides a method for identifying personnel behavior based on thermal imaging, and the specific steps are as follows:

[0070] S101: Acquire a thermal imaging image to be identified.

[0071] Specifically, the thermal imaging image to be identified is a two-dimensional image obtained in the target space. In the embodiment provided by the present invention, the above-mentioned target space is a power distribution station. In other embodiments, it can be other spaces or areas where human behavior needs to be identified, and there is no limitation on this. In this example, the thermal imaging image to be identified is collected by a fixed camera in the power distribution station. The thermal imaging image to be identified is determined by the following method:

[0072] Obtain an initial thermal imaging image; compress the initial thermal imaging image according to a preset compression multiple to obtain a thermal imaging image to be identified, which is specifically expressed as:

[0073]

[0074] Among them, O is the compressed thermal imaging image to be identified, O (i,j) is the pixel value of the pixel point in the i-th row and j-th column of the thermal imaging image to be identified, N is the compression factor, w is the number of rows in the initial thermal imaging image, h is the number of columns in the initial thermal imaging image, I (i,j) is the pixel value of the pixel at the i-th row and j-th column in the initial thermal imaging image.

[0075] It is understandable that if the pixels in the complete initial thermal imaging image are scanned and filtered, a large amount of computing resources will be occupied and the detection efficiency will be reduced. In order to improve the detection efficiency, the initial thermal imaging image is compressed. In a specific example, the compression factor is set to 3, that is, the 9 pixels in the 3×3 matrix in the initial thermal imaging image are compressed into 1 pixel. In theory, the detection time can be shortened to the original time. The specific compression process is as follows:

[0076]

[0077] Among them, O is the compressed thermal imaging image to be identified, O (i,j is the pixel value of the pixel point in the i-th row and j-th column in the thermal imaging image to be identified, w is the number of rows in the initial thermal imaging image, h is the number of columns in the initial thermal imaging image, and I (i,j is the pixel value of the pixel at the i-th row and j-th column in the initial thermal imaging image.

[0078] In the above compression process, the maximum pixel value among the 9 pixels in the 3×3 matrix in the initial thermal imaging image is used as the pixel value of one pixel after compression.

[0079] S102: Extracting an abnormal pixel set and a person's position according to the pixel value of each pixel in the thermal imaging image to be identified.

[0080] Specifically, the pixel value of each pixel in the thermal imaging image to be identified is obtained in turn. When the pixel value exceeds the preset abnormal threshold, the pixel corresponding to the pixel value is an abnormal pixel in the abnormal pixel set. When the pixel value is within the preset body temperature pixel range, the pixel corresponding to the pixel value is a pixel in the personnel position.

[0081] In a specific embodiment, due to the characteristics of the image formed by thermal imaging, the higher the temperature, the higher the brightness of the area where the pixel point appears on the thermal imaging image to be identified, and the closer the pixel value of the pixel point is to 255. The surface temperature of a lit cigarette butt is between 200° and 300°, and the center temperature can reach 700° to 800°. The temperature difference makes the pixel value of the pixel point corresponding to the burning cigarette in the thermal imaging image to be identified much greater than the pixel value of the pixel point in other areas. Therefore, an abnormal threshold can be set. When the pixel value of a pixel point on the thermal imaging image to be identified is greater than the set abnormal threshold, the pixel point is considered to be a temperature abnormal point, that is, an abnormal pixel point.

[0082] It is understandable that the detection of abnormal pixels can be completed on the thermal imaging image to be identified or on the initial thermal imaging image. If it is completed on the thermal imaging image to be identified, the position of the abnormal pixel on the initial thermal imaging image needs to be determined based on the correspondence between the pixels in the initial thermal imaging image and the thermal imaging image to be identified.

[0083] The correspondence between the pixels in the initial thermal imaging image and the thermal imaging image to be identified is specifically expressed as:

[0084] I (i,j) =O (i′ , j′)

[0085] i∈(i′*3,i′*3+1,i′*3+2),j∈(j′*3,j′*3+1,j′*3+2)

[0086] i′∈(0,1,…w),j′∈(0,1,…,h)

[0087] Among them, I (i,j) is the pixel point at the i-th row and j-th column in the initial thermal imaging image, O (i′,j′) is the pixel at the i′th row and j′th column in the thermal imaging image to be identified, w is the number of rows in the initial thermal imaging image, and h is the number of columns in the initial thermal imaging image.

[0088] In a specific implementation, because the temperature of the distribution station room is kept below 35 degrees Celsius all year round, which is somewhat different from the human body temperature, under normal circumstances, the shape of the person can be clearly seen in the thermal imaging image to be identified by judging the pixel value at the pixel point.

[0089] In another specific implementation, the position of the person can be determined by performing target detection on the thermal imaging image to be identified using a target detection model. The specific detection process of the target detection model is as follows: performing convolution and pooling on the thermal imaging image to be identified to obtain a first feature map; performing feature extraction on the first feature map to generate a candidate region; analyzing the first feature map and the candidate region to obtain an area of ​​interest; and using a classifier to classify the area of ​​interest to obtain the position of the person.

[0090] In a specific example, the target detection model is a regional convolutional neural network (regions with CNN features, R-CNN), and the target detection model includes a feature extraction network, a recommended candidate region, a region of interest pooling layer, and a classification regression layer. The feature extraction network performs convolution and pooling on the thermal imaging image to be identified to obtain a first feature map, and transmits the first feature map to the recommended candidate region and the region of interest pooling layer. The recommended candidate region performs feature extraction on the received first feature map, generates a candidate region, and transmits the generated candidate region to the region of interest pooling layer. The region of interest pooling layer is used to receive the first feature map and the candidate region, obtain a region of interest with a fixed size, and transmit the obtained region of interest with a fixed size to the classification regression layer. The classification regression layer is used to classify the received region of interest to obtain the location of the person. The classification regression layer is a type of classifier.

[0091] The above target detection model can be obtained by collecting data including thermal imaging images of workers to be identified from the power distribution station, manually annotating the data, and then iterating the training of the target detection model.

[0092] S103: Analyze the abnormal pixel point set according to the positional relationship between the abnormal pixel point set and the position of the person, and obtain a recognition result.

[0093] The horizontal distances between each abnormal pixel point in the abnormal pixel point set and the personnel position are compared in turn, which can be specifically expressed as:

[0094] Δ=abs(x2-x3)

[0095] Dis m =min(abs(x 1,m -x2),abs(x1-x3))

[0096] Among them, Δ is the width of the personnel position in the horizontal position, Dis m is the horizontal distance between the mth abnormal pixel point in the abnormal pixel point set and the person’s position, x 1,m is the coordinate value of the mth abnormal pixel in the abnormal pixel set on the x-axis, and x2 and x3 are the maximum and minimum values ​​of the coordinate values ​​of the personnel position on the x-axis respectively;

[0097] When Dis m <Δ, the mth abnormal pixel point is added to the relevant point set; and the recognition result is determined according to the number of relevant points in the relevant point set.

[0098] Through Dis m Make a judgment on the abnormal pixels in the abnormal pixel set. If Dis m <Δ, indicating that the mth abnormal pixel is related to the personnel position, and the abnormal pixel is added to the related point set. If Dis m ≥Δ, indicating that the mth abnormal pixel is not related to the position of the person, and the abnormal pixel is not added to the related point set. After all abnormal pixels are judged, a related point set is obtained. If the related set is not empty, it means that there are abnormal pixels related to the position of the person. If the related set is empty, it means that there are no abnormal pixels related to the position of the person.

[0099] In the example provided by the present invention, the number of related points in the related point set and zero are judged to determine the recognition result. In other examples, the comparison value can be adjusted according to actual needs. For example, the number of related points in the related point set and 1 are judged. If there is only one related point, it is considered that the related point may be misjudged, which means that there are no abnormal pixels related to the position of the person. It can also be judged by the ratio of the number of related points in the related point set to the number of abnormal pixels in the abnormal pixel set. For example, the ratio = the number of related points in the related point set / the number of abnormal pixels in the abnormal pixel set. When the ratio is less than the preset ratio value, it means that there are only a few related points in the abnormal pixels, which means that there are no abnormal pixels related to the position of the person; when the ratio is greater than or equal to the preset ratio value, it means that most of the abnormal pixels are related points, which means that there are abnormal pixels related to the position of the person. For example, the preset ratio value is 50%. In other embodiments, the preset ratio value can be adjusted according to actual needs, and this is not limited.

[0100] In a specific example, after obtaining the position of the person, the relative distance between the person's position and the pixel outlier is calculated. If the distance is less than the set threshold, the pixel outlier is considered to be related to the person's position. Otherwise, the pixel outlier is considered to be irrelevant to the person's position. The threshold is set to Δ, which is determined by the short side of the detection frame corresponding to the detected person's position, so Δ is dynamically changing. In this example, x2 and x3 are the horizontal coordinates of the upper left corner and lower right corner of the detection frame corresponding to the person's position.

[0101] The above-mentioned personnel behavior recognition method based on thermal imaging also includes:

[0102] Based on the recognition result, an image group corresponding to the thermal imaging to be recognized is obtained, wherein each image in the image group has a time sequence relationship; abnormal pixel points of each image in the image group are extracted respectively; the running trajectory of the abnormal pixel points is obtained by combining the time sequence relationship of each image in the image group; the running trajectory is analyzed to determine or adjust the recognition result.

[0103] In a specific embodiment, when the recognition result is that abnormal behavior exists, it is necessary to review the recognition result, that is, obtain the image group corresponding to the thermal imaging to be identified, and analyze the operation trajectory obtained based on the abnormal pixels of each image in the image group. If the operation trajectory indicates that abnormal behavior also exists, the recognition result is determined to be that abnormal behavior exists. If the operation result indicates that there is no abnormal behavior, the recognition result is adjusted to that there is no abnormal behavior. If the recognition result is that there is no abnormal behavior, it is not necessary to obtain the image group corresponding to the thermal imaging to be identified, that is, there is no need to review the recognition result. In the example provided by the present invention, the above-mentioned abnormal behavior refers to smoking in a power distribution station room. In other scenarios, it can be other non-compliant behaviors, which are not limited to this.

[0104] Furthermore, according to the temporal relationship of each image in the image group, the coordinates of the abnormal pixel points in the image are normalized in turn to obtain the standard coordinates; the standard coordinates of each image in the image group are fused to obtain the running trajectory of the abnormal pixel points.

[0105] In a specific example, according to the acquisition frequency of the thermal imaging images to be identified, all images within 10 seconds are continuously tracked and collected to obtain an image group, and the pixel abnormal points on each thermal imaging image to be identified in the image group are detected. The coordinates of the pixel abnormal points on each thermal imaging image to be identified are normalized to between 0 and 1 to obtain standard coordinates. The running trajectory of the pixel abnormal points is constructed according to the time series relationship between each image in the image group.

[0106] Furthermore, the running trajectory is input into a preset trajectory analysis model to obtain an analysis result; based on the analysis result, the recognition result is determined or adjusted.

[0107] In a specific implementation, the running trajectory is input into a preset trajectory analysis model for classification to obtain an analysis result, which is used to reflect whether the pixel abnormality point is caused by the smoking behavior of the staff. If the analysis result indicates that the pixel abnormality point is caused by the smoking behavior of the staff, the recognition result is determined to be the presence of abnormal behavior. If the analysis result shows that the pixel abnormality point is not caused by the smoking behavior of the staff, the recognition result is adjusted to the absence of abnormal behavior.

[0108] The trajectory analysis model is determined in the following way:

[0109] A training data set is obtained, wherein the training data set includes running trajectories of multiple abnormal pixel points; based on the training data set, a preset classification model is trained to obtain a trajectory analysis model.

[0110] In a specific example, the trajectory analysis model is a machine learning classification model, and the machine learning classification model needs to be trained to obtain the trajectory analysis model. The above training data set can be the running trajectory in artificially simulated smoking behavior, or it can be the data actually collected and processed in the power distribution station. The processed training data set is input into the machine learning classification model for training. For example, the coordinates of all pixel abnormal points in the thermal imaging images to be identified within 10 consecutive seconds are obtained in chronological order as the data in the training data set.

[0111] Reference Figure 2 The present invention collects the thermal imaging image to be identified and extracts abnormal pixels and personnel positions from the thermal imaging image to be identified, and then analyzes and determines the correlation between the personnel position and the abnormal pixels. If the personnel position and the abnormal pixels are not correlated, it means that there is no abnormal behavior, and the judgment process ends. If the personnel position and the abnormal pixels are correlated, it means that there is abnormal behavior, and the result needs to be further reviewed. The image group corresponding to the collected thermal imaging image to be identified is obtained and the running track of the abnormal pixels in the image group is analyzed. If the running track and the abnormal behavior match, the recognition result is determined to be the presence of abnormal behavior. If the running track and the abnormal behavior do not match, the recognition result is adjusted to the absence of abnormal behavior, and the judgment process ends.

[0112] Reference Figure 3 , an embodiment of the present invention provides a personnel behavior recognition device based on thermal imaging, comprising:

[0113] An image acquisition module 201 is used to acquire a thermal imaging image to be identified, wherein the thermal imaging image to be identified is a two-dimensional image;

[0114] The abnormality extraction module 202 is used to extract the abnormal pixel point set and the personnel position according to the pixel value of each pixel point in the thermal imaging image to be identified;

[0115] The analysis and identification module 203 is used to analyze the abnormal pixel point set according to the positional relationship between the abnormal pixel point set and the position of the person to obtain the identification result.

[0116] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the described module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0117] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for identifying personnel behavior based on thermal imaging, characterized in that: include: Acquire a thermal imaging image to be identified, wherein the thermal imaging image to be identified is a two-dimensional image; According to the pixel value of each pixel in the thermal imaging image to be identified, the abnormal pixel set and the personnel position are extracted; According to the positional relationship between the abnormal pixel point set and the personnel position, the abnormal pixel point set is analyzed to obtain the recognition result.

2. The method for identifying personnel behavior based on thermal imaging according to claim 1, characterized in that: The thermal imaging image to be identified is determined in the following way: Acquiring an initial thermal imaging image; According to the preset compression multiple, the initial thermal imaging image is compressed to obtain the thermal imaging image to be identified, which is specifically expressed as: Among them, O is the compressed thermal imaging image to be identified, O (i,j) is the pixel value of the pixel point in the i-th row and j-th column of the thermal imaging image to be identified, N is the compression factor, w is the number of rows in the initial thermal imaging image, h is the number of columns in the initial thermal imaging image, I (i,j) is the pixel value of the pixel at the i-th row and j-th column in the initial thermal imaging image.

3. The method for identifying personnel behavior based on thermal imaging according to claim 1, characterized in that: According to the pixel value of each pixel in the thermal imaging image to be identified, the abnormal pixel set and the personnel position are extracted, including: Sequentially obtain the pixel value of each pixel point in the thermal imaging image to be identified; When the pixel value exceeds the preset abnormal threshold, the pixel point corresponding to the pixel value is an abnormal pixel point in the abnormal pixel point set; When the pixel value is within the preset body temperature pixel range, the pixel point corresponding to the pixel value is the pixel point in the person's position.

4. The method for identifying personnel behavior based on thermal imaging according to claim 1, characterized in that: According to the positional relationship between the abnormal pixel set and the personnel position, the abnormal pixel set is analyzed to obtain the recognition result, which specifically includes: The horizontal distances between each abnormal pixel point in the abnormal pixel point set and the personnel position are compared in turn, which can be specifically expressed as: Δ=abs(x2-x3 ) Dis m =min(abs(x 1,m -x2),abs(x1-x3)) Among them, Δ is the width of the personnel position in the horizontal position, Dis m is the horizontal distance between the mth abnormal pixel point in the abnormal pixel point set and the person’s position, x 1,m is the coordinate value of the mth abnormal pixel in the abnormal pixel set on the x-axis, and x2 and x3 are the maximum and minimum values ​​of the coordinate values ​​of the personnel position on the x-axis respectively; When Dis m <Δ, add the mth abnormal pixel point to the relevant point set; The recognition result is determined according to the number of related points in the related point set.

5. The method for identifying personnel behavior based on thermal imaging according to claim 1, characterized in that: The method also includes: Based on the recognition result, an image group corresponding to the thermal imaging to be recognized is obtained, wherein each image in the image group has a time sequence relationship; Extract abnormal pixels of each image in the image group respectively; Combining the temporal relationship of each image in the image group, the running trajectory of the abnormal pixel point is obtained; Analyze the running trajectory and confirm or adjust the recognition results.

6. The method for identifying personnel behavior based on thermal imaging according to claim 5, characterized in that: Combined with the temporal relationship of each image in the image group, the running trajectory of the abnormal pixel point is obtained, including: According to the time sequence relationship of each image in the image group, the coordinates of abnormal pixel points in the image are normalized in turn to obtain standard coordinates; The standard coordinates of each image in the image group are fused to obtain the running trajectory of the abnormal pixel points.

7. The method for identifying personnel behavior based on thermal imaging according to claim 5, characterized in that: Analyze the running trajectory and determine or adjust the recognition results, including: Input the running trajectory into the preset trajectory analysis model to obtain the analysis result; Based on the analysis results, the recognition results are determined or adjusted.

8. The method for identifying personnel behavior based on thermal imaging according to claim 7, characterized in that: The trajectory analysis model is determined in the following way: Obtaining a training data set, wherein the training data set includes running trajectories of a plurality of abnormal pixel points; Based on the training data set, the preset classification model is trained to obtain the trajectory analysis model.

9. The method for identifying human behavior based on thermal imaging according to claim 1, characterized in that: The personnel location is obtained by: Convolution and pooling are performed on the thermal imaging image to be identified to obtain a first feature map; Perform feature extraction on the first feature map to generate a candidate region; Analyze the first feature map and the candidate region to obtain the region of interest; Use the classifier to classify the area of ​​interest and obtain the location of the person.

10. A device for identifying human behavior based on thermal imaging, characterized in that: The method for identifying personnel behavior based on thermal imaging as claimed in any one of claims 1 to 9 comprises: An image acquisition module is used to acquire a thermal imaging image to be identified, wherein the thermal imaging image to be identified is a two-dimensional image; An abnormality extraction module, used to extract abnormal pixel points and personnel positions according to the pixel values ​​of each pixel point in the thermal imaging image to be identified; The analysis and recognition module is used to analyze the abnormal pixel point set according to the positional relationship between the abnormal pixel point set and the personnel position to obtain the recognition result.

Images