Abnormal behavior detection method and device, computer device and storage medium

Abstract

The application relates to an abnormal behavior detection method and device, computer equipment, a storage medium and a computer program product, wherein visible light training image information and infrared training image information are simultaneously used for model training to obtain a preset double-flow fusion target detection model. In actual detection, visible light image information and infrared image information can be simultaneously acquired, the trained preset double-flow fusion target detection model is combined for analysis, whether the current target person has abnormal behaviors such as smoking is judged, and finally, in the case that abnormal behaviors are detected, alarm prompt information is output. The above scheme combines the advantages of visible light image prediction of abnormal behaviors and infrared light prediction of abnormal behaviors, and in the case that partial shielding or a long distance occurs, the abnormal behaviors of the target person can still be effectively detected, and the detection precision is high.

Description

Technical Field

[0001] The present application relates to the field of detection technology, and in particular to an abnormal behavior detection method, apparatus, computer equipment, storage medium, and computer program product. Background Art

[0002] With the continuous development of power grid technology, the types and number of loads within the grid are increasing, and grid operations are becoming increasingly complex and diverse. To ensure power supply reliability, the safe operation of transmission lines is crucial. The equipment and instruments within most transmission lines require the use of various lubricants for routine maintenance. These lubricants are highly susceptible to fires when exposed to open flames, leading to serious safety accidents.

[0003] Currently, infrared recognition technology is commonly used to detect whether workers are engaging in unusual behavior, such as smoking, to prevent fires. However, in situations such as occlusion, such unusual behavior is often difficult to detect, and existing methods for detecting unusual behavior suffer from low detection accuracy. Summary of the Invention

[0004] Based on this, it is necessary to provide an abnormal behavior detection method, device, computer equipment, storage medium and computer program product with high detection accuracy to address the above technical problems.

[0005] A method for detecting abnormal behavior includes: obtaining visible light image information and infrared image information corresponding to a target person; obtaining abnormal behavior location information and temperature information based on the visible light image information, the infrared image information, and a preset dual-stream fusion target detection model; training the preset dual-stream fusion target detection model based on visible light training image information and infrared training image information; judging whether the target person has abnormal behavior based on the abnormal behavior location information and the temperature information; and outputting an alarm prompt message if the target person has abnormal behavior.

[0006] The above-mentioned abnormal behavior detection method simultaneously trains models based on visible light training image information and infrared training image information to obtain a preset dual-stream fusion target detection model. During actual detection, visible light and infrared image information can be simultaneously acquired and analyzed in conjunction with the trained preset dual-stream fusion target detection model to determine whether the current target person is engaging in abnormal behavior, such as smoking. Ultimately, an alarm is output if abnormal behavior is detected. This solution, combining the advantages of abnormal behavior prediction using visible light images and infrared light, can effectively detect abnormal behavior of the target person even in the presence of partial occlusion or at long distances, achieving high detection accuracy.

[0007] In one embodiment, the method for determining the preset dual-stream fusion target detection model includes: obtaining visible light training image information and infrared training image information; preprocessing, splicing and feature extraction of the visible light training image information and the infrared training image information respectively to obtain corresponding feature images and position codes of the feature images; performing encoder encoding according to the feature images and the position codes to obtain a first output value; performing feature restoration, head network prediction and proportional network matching according to the first output value to obtain a second output value; performing gradient return loss calculation according to the second output value to obtain trained network parameters; updating the dual-stream target detection model according to the trained network parameters to obtain a preset dual-stream fusion target detection model.

[0008] In one embodiment, the encoder includes at least one of a multi-head attention mechanism network, a residual network, a layer normalization network, and a feedforward network.

[0009] In one embodiment, feature extraction is performed on the visible light training image information and the infrared training image information respectively, including: feature extraction is performed on the visible light training image information and the infrared training image respectively through a lightweight neural network.

[0010] In one embodiment, the abnormal behavior location information and temperature information are obtained based on the visible light image information, the infrared image information and the preset dual-stream fusion target detection model, including: performing model analysis based on the visible light image information, the infrared image information and the preset dual-stream fusion target detection model to obtain the abnormal behavior location information; performing temperature inversion based on the infrared image information to obtain temperature information.

[0011] In one embodiment, judging whether the target person has abnormal behavior based on the abnormal behavior location information and the temperature information includes: if the abnormal behavior location information matches the preset abnormal location information, determining that the target person has abnormal behavior; or if there is temperature information greater than or equal to a preset temperature threshold, determining that the target person has abnormal behavior.

[0012] An abnormal behavior detection device includes: an image acquisition module for acquiring visible light image information and infrared image information corresponding to a target person; a model matching module for obtaining abnormal behavior location information and temperature information based on the visible light image information, the infrared image information and a preset dual-stream fusion target detection model; the preset dual-stream fusion target detection model is obtained by model training based on visible light training image information and infrared training image information; an abnormality judgment module for judging whether the target person has abnormal behavior based on the abnormal behavior location information and the temperature information; and an alarm prompt module for outputting an alarm prompt message if the target person has abnormal behavior.

[0013] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the abnormal behavior detection method when executing the computer program.

[0014] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned abnormal behavior detection method.

[0015] A computer program product includes a computer program, which implements the steps of the abnormal behavior detection method when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a diagram of the application environment of the abnormal behavior detection method in one embodiment of the present application;

[0017] Figure 2 This is a flowchart of an abnormal behavior detection method in one embodiment of the present application;

[0018] Figure 3 This is a flowchart of abnormal behavior detection in one embodiment of the present application;

[0019] Figure 4 This is a flowchart of an abnormal behavior detection method in another embodiment of the present application;

[0020] Figure 5 This is a structural block diagram of an abnormal behavior detection device in one embodiment of the present application;

[0021] Figure 6 This is a diagram of the internal structure of a computer device in one embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0023] The abnormal behavior detection method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the image acquisition device 102 is connected to the processing device 104 via wired or wireless communication. The processing device 104 can be a terminal, a server, or a server cluster, without specific limitation, as long as it has the corresponding image data processing function. Specifically, the terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablet computers, and portable wearable devices, without specific limitation.

[0024] In more detail, in one embodiment, the abnormal behavior detection method is applied to a power grid. An image acquisition device 102 is disposed on the power grid to collect visible light and infrared image information of power grid workers and transmit it to a processing device 104. Processing device 104 can be deployed on the power grid or remotely. As long as it can promptly receive the visible light and infrared image information transmitted by image acquisition device 102, it can detect whether power grid workers engage in abnormal behavior (e.g., smoking) during their work. In other embodiments, the abnormal behavior detection method can also be applied to other scenarios, such as those requiring open flame detection or smoke detection in warehouses, without limitation.

[0025] In one embodiment, Figure 2 As shown, a method for detecting abnormal behavior is provided, which is applied to the power grid and Figure 1 The processing device 104 in the embodiment of the present invention is described as follows, which includes the following steps:

[0026] Step 202: Obtain visible light image information and infrared image information corresponding to the target person.

[0027] Specifically, the target person is the person whose unusual behavior needs to be detected. Visible light image information is image information obtained when capturing images based on visible light, and infrared image information is image information obtained when capturing images based on infrared light, i.e., image information corresponding to the infrared image. An image acquisition device is installed in the power grid. This image acquisition device can capture images in real time, obtain visible light image information and infrared image information, and upload the information to a processing device via wired or wireless communication.

[0028] It should be pointed out that the type of image acquisition device is not unique. In one embodiment, it can be an image acquisition device that integrates visible light image acquisition and infrared image acquisition. In another embodiment, it can also include two image acquisition devices, one for visible light image acquisition and the other for infrared image acquisition. There is no specific limitation.

[0029] It is understood that the specific types of abnormal behavior are not limited to a single type, and any abnormal behavior that causes a change in the target person's behavior and a change in the temperature of the area where the target person is located is acceptable. For example, in a more detailed embodiment, the abnormal behavior includes smoking (including but not limited to e-cigarettes, cigarettes, etc.).

[0030] In one embodiment, the visible light image information is image information obtained by marking each pixel point in the visible light image according to a coordinate system, and the infrared image information is image information obtained by marking each pixel point in the infrared image according to a coordinate system.

[0031] It should be noted that the visible light image information and infrared image information can be directly acquired by the image acquisition device. That is, after acquiring the infrared image and visible light image, the image acquisition device annotates them to obtain the visible light image information and infrared image information, and transmits them to the processing device. In another embodiment, the image acquisition device may acquire the infrared image and visible light image and directly transmit them to the processing device. The processing device then performs further annotation to obtain the visible light image information and infrared image information. This is not limited to the specific method.

[0032] Step 204 : Obtain abnormal behavior location information and temperature information based on the visible light image information, the infrared image information, and a preset dual-stream fusion target detection model.

[0033] Specifically, the preset dual-stream fusion target detection model is trained based on visible light training image information and infrared training image information. The preset dual-stream fusion target detection model is based on a preset detection model capable of predicting behavior using dual-stream data information (visible light image information and infrared image information). It characterizes the correspondence between visible light image information, infrared image information, and abnormal behavior location information. Therefore, in actual applications, after inputting visible light image information and infrared image information into the preset dual-stream fusion target detection model for prediction, abnormal behavior location information can be output. Further analysis combined with infrared image information can determine the actual temperature corresponding to each pixel in the infrared image.

[0034] It can be understood that in another embodiment, after obtaining the visible light image information and the infrared image information, the processing device may pre-process the visible light image information and the infrared image information (including denoising, image enhancement processing, etc.) to ensure the subsequent detection accuracy and detection reliability, and then perform matching analysis on the pre-processed visible light image information and the infrared image information, and finally issue an alarm prompt when abnormal behavior is detected.

[0035] Step 206: Determine whether the target person has abnormal behavior based on the abnormal behavior location information and temperature information.

[0036] Specifically, after obtaining the abnormal behavior location information and temperature information of the target person, the processing device will further analyze and judge in combination with the abnormal behavior location information and temperature information to determine whether the target user currently has abnormal behavior, for example, whether the target user currently has smoking behavior, etc.

[0037] Step 208: If the target person has abnormal behavior, output an alarm prompt message.

[0038] Specifically, after the processing device analyzes the abnormal behavior location information and temperature information and determines that the target person has abnormal behavior, it will output an alarm prompt message to remind the target person to stop the abnormal behavior, or remind the monitoring personnel that the current target person has abnormal behavior and needs to be stopped.

[0039] It should be noted that the processing device is not the only way to output the alarm prompt information. In one embodiment, the alarm prompt information can be output to the alarm device, and the alarm device can output it in the form of sound, light, text or vibration to inform the target person or monitoring person.

[0040] It can be understood that if the processing device determines that the target person has no abnormal behavior at this time, it will return to execute the operation of obtaining the visible light image information and infrared image information corresponding to the target person, and re-perform the next round of analysis and judgment, so that when the target person has abnormal behavior, it can output alarm prompt information in time to remind.

[0041] The above-mentioned abnormal behavior detection method simultaneously trains models based on visible light training image information and infrared training image information to obtain a preset dual-stream fusion target detection model. During actual detection, visible light and infrared image information can be simultaneously acquired and analyzed in conjunction with the trained preset dual-stream fusion target detection model to determine whether the current target person is engaging in abnormal behavior, such as smoking. Ultimately, an alarm is output if abnormal behavior is detected. This solution, combining the advantages of abnormal behavior prediction using visible light images and infrared light, can effectively detect abnormal behavior of the target person even in the presence of partial occlusion or at long distances, achieving high detection accuracy.

[0042] Please refer to Figure 3In one embodiment, a method for determining a preset dual-stream fusion target detection model includes: obtaining visible light training image information and infrared training image information; preprocessing, splicing, and feature extraction of the visible light training image information and the infrared training image information respectively to obtain corresponding feature images and position codes of the feature images; performing encoder encoding according to the feature images and position codes to obtain a first output value; performing feature restoration, head network prediction, and proportional network matching according to the first output value to obtain a second output value; performing gradient return loss calculation according to the second output value to obtain trained network parameters; and updating the dual-stream target detection model (i.e., the dual-stream Yolov5 model) according to the trained network parameters to obtain a preset dual-stream fusion target detection model.

[0043] Specifically, the visible light training image information is image information obtained by annotating each pixel point in the visible light image used for training according to the coordinate system, and the infrared training image information is image information obtained by annotating each pixel point in the infrared image used for training according to the coordinate system.

[0044] For example, in a more detailed embodiment, the visible light training image information or infrared training image information corresponding to the target person can be marked as (x, y, w, h), where (x, y) represents the center point coordinates of the target person area, w represents the width of the target person area, and h represents the height of the target person area. The category is marked as L, indicating the target person.

[0045] The obtained visible light training image information and infrared training image information are then preprocessed. The specific processing method is not unique and can be selected based on actual needs. For example, it can be data denoising, image enhancement or amplification processing, etc., and there is no specific limitation.

[0046] The obtained visible light training image information and infrared training image information are then combined to perform feature extraction. Specifically, the coordinate position parameters corresponding to the feature points in the visible light training image information and the infrared training image information are extracted, and the coordinate position parameters corresponding to the two image information are spliced ​​together to obtain the final feature image and the position code of each pixel in the feature image. In more detail, in one embodiment, the position code (PE) is calculated as follows:

[0047]

[0048]

[0049] Among them, pos represents the position of the pixel in the feature image, i represents the dimension of the feature map, d model The dimension of the vector representing the two-stream object detection model settings.

[0050] After that, the obtained feature image and position code are input into the encoder for encoding processing to obtain the first output value. Specifically, the encoder includes six encoders of the same type connected in sequence in the transformer network. In the first encoder, the feature image X is input pos and position encoding (PE) to obtain an output value, which is then input to the next encoder, which then outputs an output value, and so on, until the last encoder outputs the first output value. It is understood that in other embodiments, the number of encoders can also be set to other numbers, which can be selected based on actual needs.

[0051] Since the data originally in matrix form is processed into column vector form during the position encoding and encoder processing, it is necessary to perform feature restoration on the first output value to restore it to the same matrix size as the feature layer after feature extraction to obtain the output value. The output value is then input into the prediction head network prediction and the proportional network matching in sequence to obtain the second output value. Finally, the second output value and the gradient return loss function are combined for calculation to obtain the corresponding network parameters at this time, that is, the trained network parameters. The trained network parameters are used to update the dual-stream target detection model to obtain the trained dual-stream target detection model (that is, the dual-stream fusion Yolov5 model) and store it as the preset dual-stream fusion target detection model.

[0052] In one embodiment, the encoder includes at least one of a multi-head attention mechanism network, a residual network, a layer normalization network, and a feedforward network.

[0053] Specifically, for the multi-head attention mechanism network, the concepts of Query, Key and Value are introduced. Query (Q) means query, and Key (K) is the key used to compare with the query to be queried. The comparison results in a score (relevance or similarity) which is then multiplied by the Value (V) to get the final result. The input parameter Q is the output of the previous encoder (if the current encoder is the first encoder, it is the feature image X pos ) plus position encoding (PE), and matrix W Q The result of the multiplication, K is the output of the previous encoder plus the position code (PE), and the matrix W K The result of multiplication, V is the output of the previous encoder plus the position code (PE), and the matrix W V The result of multiplication.

[0054] The multi-head attention mechanism network is calculated through the multi-head attention mechanism, and the calculation formula is as follows:

[0055]

[0056] wherein the multi-head attention mechanism network is used to calculate the correlation, is to change the attention matrix into a standard normal distribution, the above formula indicates that the multi-head attention mechanism uses multiple different parameters Q, K, V to calculate the result of Attention, performs concatenation, and then performs dimension adjustment through a matrix multiplication to adjust the dimension to be consistent with the dimension of the input value of the encoder.

[0057] The feedforward network is a two-layer fully connected layer, the activation function of the first layer is Relu, and the second layer does not use the activation function. The output matrix obtained by the feedforward network finally has the same dimension as the input matrix. The calculation formula is as follows:

[0058] output = max(0, XW1 + 1)W2 + 2

[0059] wherein X represents the input, W1 represents the matrix in the full connection, b1 represents the bias term, max() represents the maximum value function, W2 represents the matrix in the full connection, and b2 represents the bias term.

[0060] In more detail, in one embodiment, for the above-mentioned encoder for encoding, each encoder includes, in sequence, a multi-head attention mechanism network, a residual network, a layer normalization network, a feedforward network, a residual network, and a layer normalization network.

[0061] In one embodiment, feature extraction is performed on the visible light training image information and the infrared training image information respectively, including: feature extraction is performed on the visible light training image information and the infrared training image information respectively through a lightweight neural network.

[0062] Specifically, in the scheme of the embodiment, in the dual-flow target detection model, two lightweight neural networks are arranged, and when performing feature extraction, one lightweight neural network is used for feature extraction for the visible light training image information and the infrared training image information respectively to obtain the visible light feature image and the infrared feature image after feature extraction, and then the two images are spliced to perform subsequent operations. In more detail, in one embodiment, the lightweight neural network is specifically a MobileNet V2 type network, and each MobileNet V2 network is responsible for feature extraction of one type of data. The MobileNet V2 network is more lightweight, and in the actual application process, the test speed can be guaranteed.

[0063] Please refer to Figure 4 In one embodiment, step 204 includes step 402 and step 404.

[0064] Step 402 : Perform model analysis based on the visible light image information, the infrared image information, and a preset dual-stream fusion target detection model to obtain abnormal behavior location information.

[0065] Step 404: Perform temperature inversion based on the infrared image information to obtain temperature information.

[0066] Specifically, a pre-set dual-stream fusion target detection model characterizes the correspondence between visible light image information, infrared image information, and abnormal behavior location information. Therefore, analysis based on the pre-set dual-stream fusion target detection model and the collected visible light and infrared image information can ultimately confirm the abnormal behavior location of the current target person. Temperature information can be obtained by simply performing temperature inversion on the infrared images to obtain temperature information corresponding to different pixels in different infrared images. Finally, the processing device combines the abnormal behavior location and temperature information to perform abnormal behavior analysis.

[0067] It should be pointed out that in one embodiment, after the visible light image information and the infrared image information are input into the preset dual-stream fusion target detection model for prediction, the predicted bounding box is output. After that, the maximum suppression method is needed to suppress the duplicate boxes, and the output result is used as the final predicted box to obtain the abnormal behavior location information of the target person.

[0068] In one embodiment, based on the abnormal behavior location information and temperature information, it is determined whether the target person has abnormal behavior, including: if the abnormal behavior location information matches the preset abnormal location information, it is determined that the target person has abnormal behavior; or if there is temperature information greater than or equal to a preset temperature threshold, it is determined that the target person has abnormal behavior.

[0069] Specifically, after the processing device obtains the abnormal behavior location information and temperature information, it matches the abnormal behavior location information with the preset abnormal location information, and compares and analyzes the temperature information of each pixel point with the preset temperature threshold. If the abnormal behavior location information matches the preset abnormal location information, or there is temperature information greater than or equal to the preset temperature threshold, it is considered that the target person has abnormal behavior at this time, and an alarm prompt information is output.

[0070] In order to facilitate understanding of the technical solution of the present application, the present application is explained below in conjunction with more detailed embodiments.

[0071] First, an image capture device (which can be a camera integrating visible light and infrared capture) is used to capture images of smokers. The captured visible light and infrared images are annotated to obtain visible light and infrared training image information. The visible light and infrared training image information are then subjected to denoising, image enhancement, and image amplification to obtain processed visible light and infrared training image information. The processed visible light and infrared training image information are respectively input into the corresponding MobileNet V2 networks for feature extraction. The output data from the feature extraction of the two MobileNet V2 networks is concatenated and encoded to obtain a feature image and corresponding position code. The feature image and position code are then input into an encoder for sequential encoding, ultimately outputting a first output value.

[0072] The first output is then restored to the same size as the feature layer after the feature extraction module and input into the prediction header module for prediction. It is then transferred to the positive example matching module for matching, resulting in the second output value. Finally, the classification regression loss is calculated based on the second output value, and the network parameters are updated through gradient backpropagation to obtain the trained two-stream object detection model, which is also the preset two-stream fusion object detection model.

[0073] In actual application, the camera collects visible light image information and infrared image information in real time, and inputs it into the preset dual-stream fusion target detection model for prediction. It outputs the predicted bounding box, uses non-maximum suppression to suppress duplicate boxes, and uses the output result as the final predicted box to obtain abnormal behavior location information. At the same time, temperature inversion can be performed based on the infrared image information to determine the temperature information of each pixel in the infrared image. Finally, when the abnormal behavior location information is detected to match the preset abnormal location information, or when there is temperature information greater than or equal to the preset temperature threshold, an alarm prompt information is output to indicate the presence of smokers or to warn smokers to stop smoking.

[0074] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0075] Based on the same inventive concept, embodiments of the present application also provide an abnormal behavior detection device for implementing the aforementioned abnormal behavior detection method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations in one or more of the abnormal behavior detection device embodiments provided below can be found in the above-described limitations on the abnormal behavior detection method and will not be further elaborated here.

[0076] In one embodiment, Figure 5 As shown, an abnormal behavior detection device is provided, including: an image acquisition module 502, a model matching module 504, an abnormality judgment module 506 and an alarm prompt module 508, wherein:

[0077] The image acquisition module 502 is used to obtain visible light image information and infrared image information corresponding to the target person; the model matching module 504 is used to obtain abnormal behavior location information and temperature information based on the visible light image information, infrared image information and the preset dual-stream fusion target detection model; the preset dual-stream fusion target detection model is obtained by model training based on visible light training image information and infrared training image information; the abnormality judgment module 506 is used to judge whether the target person has abnormal behavior based on the abnormal behavior location information and temperature information; the alarm prompt module 508 is used to output alarm prompt information if the target person has abnormal behavior.

[0078] In one embodiment, the model matching module 504 is also used to perform model analysis based on visible light image information, infrared image information and a preset dual-stream fusion target detection model to obtain abnormal behavior location information; and perform temperature inversion based on the infrared image information to obtain temperature information.

[0079] In one embodiment, the alarm prompt module 508 is also used to determine that the target person has abnormal behavior if the abnormal behavior location information matches the preset abnormal location information; or, if there is temperature information greater than or equal to the preset temperature threshold, determine that the target person has abnormal behavior.

[0080] Each module in the abnormal behavior detection device described above may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in hardware form, or may be stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0081] The abnormal behavior detection device performs model training based on both visible light and infrared training image information, generating a preset dual-stream fusion target detection model. During actual detection, both visible light and infrared image information are simultaneously acquired and analyzed using the trained, preset dual-stream fusion target detection model to determine whether the target person is engaging in abnormal behavior, such as smoking. Ultimately, if abnormal behavior is detected, an alarm is output. This solution, combining the advantages of predicting abnormal behavior using visible light images and infrared light, can effectively detect abnormal behavior in the target person even in the presence of partial occlusion or at long distances, achieving high detection accuracy.

[0082] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, an abnormal behavior detection method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0083] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0084] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0085] Visible light image information and infrared image information corresponding to the target person are acquired; abnormal behavior position information and temperature information are obtained according to the visible light image information, the infrared image information and a preset double-flow fusion target detection model; whether the target person has an abnormal behavior is judged according to the abnormal behavior position information and the temperature information; and if the target person has an abnormal behavior, an alarm prompt information is output.

[0086] In one embodiment, a computer readable storage medium is provided, having stored thereon a computer program, the computer program being executed by a processor to implement the following steps:

[0087] Visible light image information and infrared image information corresponding to the target person are acquired; abnormal behavior position information and temperature information are obtained according to the visible light image information, the infrared image information and a preset double-flow fusion target detection model; whether the target person has an abnormal behavior is judged according to the abnormal behavior position information and the temperature information; and if the target person has an abnormal behavior, an alarm prompt information is output.

[0088] In one embodiment, a computer program product is provided, comprising a computer program, the computer program being executed by a processor to implement the following steps:

[0089] Visible light image information and infrared image information corresponding to the target person are acquired; abnormal behavior position information and temperature information are obtained according to the visible light image information, the infrared image information and a preset double-flow fusion target detection model; whether the target person has an abnormal behavior is judged according to the abnormal behavior position information and the temperature information; and if the target person has an abnormal behavior, an alarm prompt information is output.

[0090] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0091] The aforementioned computer device, storage medium, and computer program product simultaneously train models based on visible light training image information and infrared training image information to obtain a preset dual-stream fusion target detection model. During actual detection, visible light image information and infrared image information can be simultaneously acquired and analyzed in conjunction with the trained preset dual-stream fusion target detection model to determine whether the current target person is engaging in abnormal behavior, such as smoking. Ultimately, if abnormal behavior is detected, an alarm prompt is output. This solution, combining the advantages of predicting abnormal behavior using visible light images and infrared light, can effectively detect abnormal behavior of the target person even in the presence of partial occlusion or at long distances, achieving high detection accuracy.

[0092] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0093] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for detecting abnormal behavior, characterized in that: include: Obtain visible light image information and infrared image information corresponding to the target person; Performing model analysis based on the visible light image information, the infrared image information, and a preset dual-stream fusion target detection model to obtain abnormal behavior location information, and performing temperature inversion based on the infrared image information to obtain temperature information; The preset dual-stream fusion target detection model is obtained by model training based on visible light training image information and infrared training image information; If the abnormal behavior location information matches the preset abnormal location information, it is determined that the target person has abnormal behavior; Or, if there is temperature information greater than or equal to a preset temperature threshold, it is determined that the target person has abnormal behavior; If the target person has abnormal behavior, an alarm prompt message is output.

2. The abnormal behavior detection method according to claim 1, characterized in that: The method for determining the preset dual-stream fusion target detection model includes: Obtaining visible light training image information and infrared training image information; Preprocessing, splicing, and feature extraction are performed on the visible light training image information and the infrared training image information respectively to obtain corresponding feature images and position codes of the feature images; Performing encoder encoding according to the feature image and the position code to obtain a first output value; Perform feature restoration, head network prediction, and proportional network matching based on the first output value to obtain a second output value; Perform gradient backpropagation loss calculation based on the second output value to obtain trained network parameters; The dual-stream target detection model is updated according to the trained network parameters to obtain a preset dual-stream fusion target detection model.

3. The abnormal behavior detection method according to claim 2, characterized in that: The encoder includes at least one of a multi-head attention mechanism network, a residual network, a layer normalization network and a feedforward network.

4. The abnormal behavior detection method according to claim 2, characterized in that: Feature extraction is performed on the visible light training image information and the infrared training image information respectively, including: Feature extraction is performed on the visible light training image information and the infrared training image respectively through a lightweight neural network.

5. An abnormal behavior detection device, characterized in that: include: An image acquisition module is used to acquire visible light image information and infrared image information corresponding to the target person; a model matching module, configured to perform model analysis based on the visible light image information, the infrared image information, and a preset dual-stream fusion target detection model to obtain abnormal behavior location information, and perform temperature inversion based on the infrared image information to obtain temperature information; The preset dual-stream fusion target detection model is obtained by model training based on visible light training image information and infrared training image information; An abnormality judgment module, configured to determine that the target person has abnormal behavior if the abnormal behavior location information matches the preset abnormal location information; Or, if there is temperature information greater than or equal to a preset temperature threshold, it is determined that the target person has abnormal behavior; The alarm prompt module is used to output an alarm prompt message if the target person has abnormal behavior.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the abnormal behavior detection method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the abnormal behavior detection method according to any one of claims 1 to 4 are implemented.

8. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the abnormal behavior detection method according to any one of claims 1 to 4 are implemented.

Images