Dangerous case identification method and device, medium and program product

Abstract

The invention relates to the technical field of computers, and particularly provides a dangerous case recognition method and device, a medium and a program product, and the dangerous case recognition method comprises the steps: receiving a current video stream sent by an image collection device, inputting a to-be-processed image frame in the current video stream into a first dangerous case recognition network group in a pre-trained dangerous case recognition model, obtaining a feature image and a recognition probability value; when the identification probability value is smaller than or equal to a preset threshold value, determining a second dangerous case identification network group adjacent to the first dangerous case identification network group as the updated first dangerous case identification network group; and repeating the above process until each dangerous case identification network group in the dangerous case identification model is traversed, and obtaining a current dangerous case identification result. According to the invention, the dangerous case identification efficiency and real-time performance can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a method, device, medium and program product for identifying a dangerous situation. Background Art

[0002] In daily life, dangerous situations such as fires, floods or traffic accidents often occur. In order to prevent damage to life and property caused by dangerous situations, image acquisition equipment is usually installed in areas where dangerous situations frequently occur. The video or image pictures collected by the image acquisition equipment are used to determine whether a dangerous situation has occurred.

[0003] In the related technology, a neural network model can be pre-trained based on danger identification technology and deep learning technology, and the real-time collected scene images can be provided to the neural network model to determine whether a danger occurs at present.

[0004] However, since deep learning models usually have complex model structures, identifying dangerous situations based on deep learning models not only requires a large amount of resource consumption, but also leads to low recognition efficiency, affecting the real-time and reliability of dangerous situation identification. Summary of the invention

[0005] The present disclosure is proposed in view of the above problems. The present disclosure provides a method, device, medium and program product for identifying dangerous situations, which can improve the efficiency of identifying dangerous situations.

[0006] According to one aspect of the present disclosure, a method for identifying a dangerous situation is provided, and the method is applied to a dangerous situation identification device, comprising:

[0007] Receiving a current video stream sent by an image acquisition device, wherein the current video stream includes a plurality of image frames to be processed;

[0008] Inputting the image frame to be processed into the first danger recognition network group in the pre-trained danger recognition model to obtain a characteristic image and a recognition probability value, wherein the first danger recognition network group includes a danger recognition network and a threshold network, the danger recognition network is used to determine the characteristic image of the image frame to be processed, the threshold network is used to determine the recognition probability value according to the characteristic image, and the recognition probability value is used to represent the probability that the first danger recognition network group adjacent to the first danger recognition network group participates in danger recognition;

[0009] When the recognition probability value is less than or equal to a preset threshold, determining a second danger recognition network group adjacent to the first danger recognition network group as the updated first danger recognition network group, and determining the feature image as the updated image frame to be processed;

[0010] Starting from inputting the image frame to be processed into the first hazard recognition network group in the pre-trained hazard recognition model, the above process is repeated until each hazard recognition network group in the hazard recognition model is traversed to obtain the current hazard recognition result.

[0011] According to another aspect of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method.

[0012] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0013] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein the computer program / instructions implement the steps of the above method when executed by a processor.

[0014] The hazard identification method, device, medium and program product provided by the embodiments of the present disclosure configure a threshold network for each hazard identification network in the hazard identification model, and use the threshold network to process the feature image output by the hazard identification network to determine whether the next hazard identification network needs to participate in hazard identification. When the identification probability value is less than or equal to a preset threshold, it can be determined that the next hazard identification network does not participate in the processing flow, thereby reducing resource consumption in the hazard identification process, reducing the hazard identification process in the hazard identification process, improving the efficiency of hazard identification based on the neural network model, and ensuring the real-time nature of hazard identification.

[0015] It is to be understood that both the foregoing general description and the following detailed description are exemplary, and are intended to provide further explanation of the technology as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above and other purposes, features and advantages of the present disclosure will become more apparent by describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0017] Figure 1 It is a schematic diagram illustrating an implementation scenario of the dangerous situation identification solution according to an embodiment of the present disclosure.

[0018] Figure 2 is a flowchart illustrating a method for identifying a dangerous situation according to an embodiment of the present disclosure.

[0019] Figure 3 It is a structural diagram illustrating a danger identification model according to an embodiment of the present disclosure.

[0020] Figure 4 is a flow chart illustrating another method for identifying a dangerous situation according to an embodiment of the present disclosure.

[0021] Figure 5 is a block diagram illustrating a danger identification device according to an embodiment of the present disclosure.

[0022] Figure 6 is a schematic diagram illustrating a computer program product according to an embodiment of the present disclosure.

[0023] Figure 7 is a hardware block diagram illustrating an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present disclosure more obvious, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described here.

[0025] In the related art, during the process of hazard identification, the collected video stream can be processed by a pre-trained deep learning model to perform hazard identification.

[0026] However, the deep learning models used in related technologies usually include a large number of neural network layers. The process of processing the video frames in the video stream through each neural network layer requires a lot of time and computing resources; especially when the resolution of the video frames is large, processing high-resolution video frames requires more resources and time, resulting in further deterioration of the real-time performance of danger identification.

[0027] In order to solve the above problems, the present disclosure provides a dangerous situation identification solution, such as Figure 1 As shown, Figure 1 FIG. 1 is a schematic diagram showing an implementation scenario of a dangerous situation identification solution provided by an exemplary embodiment of the present disclosure. Figure 1 As shown, the implementation scenario 100 includes an image acquisition device 101 and a danger identification device 102, wherein the image acquisition device 101 may be a camera or an electronic device equipped with a camera, and the danger identification device 102 may be a computer, a notebook, a tablet computer or a server, etc.

[0028] Among them, a communication link can be established between the image acquisition device 101 and the danger identification device 102 to implement the danger identification solution provided by the embodiment of the present disclosure.

[0029] Figure 2A flowchart of a method for identifying a dangerous situation according to an exemplary embodiment of the present disclosure is shown. The method can be applied to a dangerous situation identification device, such as Figure 2 As shown, the method of the embodiment of the present disclosure may include:

[0030] Step S201, receiving a current video stream sent by an image acquisition device;

[0031] Wherein, the current video stream includes a plurality of image frames to be processed;

[0032] Step S202, inputting the image frame to be processed into the first hazard recognition network group in the pre-trained hazard recognition model to obtain a feature image and a recognition probability value;

[0033] The first danger recognition network group includes a danger recognition network and a threshold network. The danger recognition network is used to determine a feature image of an image frame to be processed. The threshold network is used to determine a recognition probability value according to the feature image. The recognition probability value is used to represent the probability that the first danger recognition network group adjacent to the first danger recognition network group participates in danger recognition.

[0034] Step S203, when the recognition probability value is less than or equal to a preset threshold, determining a second danger recognition network group adjacent to the first danger recognition network group as the updated first danger recognition network group, and determining the feature image as the updated image frame to be processed;

[0035] Step S204, starting from inputting the image frame to be processed into the first hazard recognition network group in the pre-trained hazard recognition model, repeating the above process until each hazard recognition network group in the hazard recognition model is traversed to obtain the current hazard recognition result.

[0036] To summarize, the danger identification method provided by the embodiment of the present disclosure configures a threshold network for each danger identification network in the danger identification model, and uses the threshold network to process the feature image output by the danger identification network to determine whether the next danger identification network needs to participate in the danger identification. When the recognition probability value is less than or equal to a preset threshold, it can be determined that the next danger identification network does not participate in the processing flow, thereby reducing resource consumption in the danger identification process, reducing the danger identification process in the danger identification process, improving the efficiency of danger identification based on the neural network model, and ensuring the real-time nature of danger identification.

[0037] The following Figure 2 The specific implementation methods of each step in the embodiment shown are described in detail:

[0038] In step S201, the danger identification device may receive the current video stream sent by the image acquisition device.

[0039] In the disclosed embodiment, the image acquisition device can acquire images of danger-prone areas in real time, generate a current video stream, and send the current video stream to the danger identification device; wherein the current video stream includes a plurality of image frames to be processed.

[0040] It should be noted that in the embodiments of the present disclosure, the format and resolution of the image frames to be processed in the video stream generated by the image acquisition device can be determined based on the hardware capabilities of the image acquisition device, and the embodiments of the present disclosure are not limited to this; for example, the format of the image frames to be processed can be H264 format, and the resolution can be 720p.

[0041] In an optional implementation, after the image acquisition device sends the current video stream to the danger recognition device, the danger recognition device may receive the current video stream sent by the image acquisition device.

[0042] It can be understood that in the embodiments of the present disclosure, the current video stream may include multiple frames of images, wherein the danger identification device may determine each frame of the image in the video stream as an image to be processed for danger identification, so as to ensure the accuracy of danger identification; or, since the picture content of adjacent frames is not much different, the danger identification device may also determine some images in the video stream as images to be processed for danger identification, so as to further improve the efficiency of danger identification.

[0043] Among them, the mode in which the danger recognition device selects some images from the video stream as images to be processed can be determined based on actual needs, and the embodiments of the present disclosure are not limited to this; for example, the danger recognition device can collect one frame of image from the current video stream as the image to be processed every three frames, or the danger recognition device can collect one frame of image from the current video stream as the image to be processed every 10ms.

[0044] In step S202, the danger recognition device may input the image frame to be processed into the first danger recognition network group in the pre-trained danger recognition model to obtain a feature image and a recognition probability value.

[0045] In the disclosed embodiment, the hazard recognition model may be composed of a plurality of hazard recognition network groups connected in sequence, and a fully connected layer connected to the last hazard recognition network group; wherein each hazard recognition network group includes a hazard recognition network and a threshold network, the hazard recognition network group is used to extract features of the image to be processed, the threshold network is used to determine the recognition probability value of the next hazard recognition network group adjacent to the hazard recognition network group, and the fully connected layer is used to determine the hazard recognition result based on the feature vector of the image to be processed finally obtained.

[0046] It can be understood that the first hazard identification network group in the hazard identification model includes a hazard identification network and a threshold network. The hazard identification network is used to determine the feature image of the image frame to be processed, and the threshold network is used to determine the recognition probability value based on the feature image. The recognition probability value is used to characterize the probability that the first hazard identification network group adjacent to the first hazard identification network group participates in hazard identification, wherein the higher the recognition probability value, the higher the possibility that the first hazard identification network group participates in hazard identification.

[0047] For example, Figure 3 As shown, Figure 3 A structural schematic diagram of a danger identification model provided by an embodiment of the present disclosure is shown, wherein the danger identification model includes n danger identification network groups 301, and a fully connected layer 302 connected to the last danger identification network group, wherein the danger identification network group includes a danger identification network 3011 and a threshold network 3012.

[0048] In an optional embodiment, the process of training a danger recognition model by a danger recognition device may include: obtaining a sample video data set; then, inputting a sample image frame in a sample video stream into a first danger recognition network group in a danger recognition model to be trained, and obtaining a sample feature map and a sample recognition probability value of the sample image, wherein the sample recognition probability value is used to characterize the sample probability that a first danger recognition network group adjacent to the first danger recognition network group participates in danger recognition; further, when the sample recognition probability value is less than a preset threshold, determining a second danger recognition network group adjacent to the first danger recognition network group as an updated first danger recognition network group, and determining the sample feature map as an updated sample. This image frame, and start from inputting the sample image frame in the sample video stream into the first hazard recognition network group in the hazard recognition model to be trained, repeat the above process until each hazard recognition network group in the hazard recognition model is traversed to obtain the sample hazard recognition result of the sample video stream; finally, based on the video label, the sample hazard recognition result and the loss function, determine the loss function value of the current iterative process, and when the loss function value is less than or equal to the loss function threshold, determine that the hazard recognition model converges; wherein, the sample video data set includes multiple sample video streams, and a video label corresponding to each sample video stream, and the loss function threshold can be determined based on actual needs, and the embodiments of the present disclosure are not limited to this. A hazard recognition model including a plurality of hazard recognition network groups connected in sequence can be trained so that the hazard recognition networks in the hazard recognition network group can perform image feature extraction, and the threshold network in the hazard recognition network group can determine whether the next hazard recognition network adjacent to the hazard recognition network participates in the recognition process, so that during the hazard recognition process, some hazard recognition networks can skip the feature extraction process in the hazard recognition process, thereby improving the recognition efficiency of the trained hazard recognition model in the hazard recognition process.

[0049] It should be noted that in the embodiments of the present disclosure, after obtaining the sample video data set, the hazard recognition device can pre-process the sample image frames to obtain pre-processed sample image frames, and perform hazard recognition model training based on the pre-processed sample image frames. The pre-processing process may include, but is not limited to, normalization, denoising, resolution adjustment, cropping and / or flipping, etc., which can be determined based on actual needs, and the embodiments of the present disclosure do not limit this. For example, the hazard recognition device can reduce the resolution of the sample image frames to reduce resource consumption during the hazard recognition process.

[0050] The danger identification device may perform normalization processing on the sample image frame based on a first formula, and the first formula is:

[0051]

[0052] In Formula 1, normalized_image is the normalized image value, image is the original pixel value of the image frame, μ is the average pixel value of the image frame, and σ is the standard deviation of the pixel value of the image frame.

[0053] In an optional embodiment, the process of determining the loss function value of the current iteration process by the danger identification device based on the video label, the sample danger identification result and the loss function may include: determining the data loss value of the current iteration process based on the video label, the sample danger identification result and the data loss term in the loss function; then, determining the regular loss value of the current iteration process based on the sample identification probability value of each danger identification network group and the regular loss term in the loss function; further, determining the sum of the data loss value and the regular loss value to obtain the loss function value of the current iteration process. The loss function including the regular loss term can be used to adjust the parameters of the danger identification model to prevent the trained danger identification model from overfitting and ensure the accuracy and reliability of the danger identification results determined by the danger identification model.

[0054] The loss function may be a cross entropy loss function, which may be:

[0055] [L=L cls +L reg ]; (Formula 2)

[0056] In formula 2, L cls is the data loss term in the loss function, L reg is the regular loss term in the loss function, where the data loss term is:

[0057]

[0058] In Formula 3, C is the number of categories of the recognition result, y c is the one-hot encoding of the video tag, It is the sample hazard identification result.

[0059] The regularized loss term is:

[0060] [L reg =λ∑p i ]; (Formula 4)

[0061] In formula 4, p i is the sample recognition probability value of each i-th hazard recognition network group, and λ is the regularization coefficient.

[0062] In an optional implementation, the process of determining the loss function value of the current iteration process based on the video label, the sample danger recognition result and the loss function may include: determining the data loss value of the current iteration process based on the video label, the sample danger recognition result and the data loss term in the loss function; and then determining the loss function value of the current iteration process based on the data loss value. The convergence speed of the model can be improved during the training of the danger recognition model.

[0063] It can be understood that in the embodiments of the present disclosure, when the loss function value is greater than the loss function threshold, the parameters of the hazard recognition model can be adjusted, and the hazard recognition model can be trained for the next iteration using the currently selected sample video stream, or the sample video stream can be reselected to train the hazard recognition model until the model converges.

[0064] Among them, the process of adjusting the parameters of the hazard identification model by the hazard identification device can be implemented based on the model parameter optimization method. The model parameter optimization method can be selected based on actual conditions, and the embodiments of the present disclosure are not limited to this. By way of example, the model parameter optimization method can be a gradient descent method, an Adam optimization algorithm, or a stochastic gradient descent (SGD) algorithm.

[0065] For example, when the model parameter optimization method is the SGD algorithm, the process of adjusting the parameters of the danger identification model by the danger identification device can be implemented by the second formula, wherein the second formula is:

[0066]

[0067] In Formula 5, θ t are the model parameters before adjustment, η is the learning rate, α is the momentum factor, λ is the weight decay coefficient, L(θ) is the loss function, is the gradient of the loss function with respect to the model parameters.

[0068] It should be noted that in the embodiments of the present disclosure, the danger identification network in the danger identification model can be a convolutional neural network of any architecture. The specific architecture can be determined based on actual needs, and the embodiments of the present disclosure do not limit this.

[0069] In an optional implementation, in order to further improve the efficiency of danger identification, the danger identification network in the danger identification model can be a lightweight network model. For example, the danger identification model is constructed based on the SqueezeNet network, and the danger identification network is a Fire Module network; then the danger identification device inputs the image frame to be processed into the first danger identification network group in the pre-trained danger identification model, and the process of obtaining the feature image and the recognition probability value can include: inputting the image frame to be processed into the Fire Module network in the first danger identification network group to obtain the feature image; then, inputting the feature image into the threshold network adjacent to the Fire Module network in the first danger identification network group to obtain the recognition probability value. The image feature extraction can be performed through the lightweight Fire Module network, and the danger identification network group participating in the danger identification can be screened in cooperation with the threshold network that can determine whether the next Fire Module network participates in the image feature extraction, so as to further improve the danger identification efficiency of the danger identification model.

[0070] In an optional implementation, the process of the danger identification device inputting the feature image into the threshold network adjacent to the Fire Module network to obtain the recognition probability value may include: inputting the feature image into the global average pooling layer in the threshold network adjacent to the Fire Module network to obtain a feature vector; then, processing the feature vector based on the activation function in the threshold network to obtain the recognition probability value; wherein the activation function may be a Sigmoid function. The features extracted by the danger identification network may be reduced in dimension through the global average pooling layer, thereby reducing the parameters in the process of determining the recognition probability value, improving the efficiency of determining the recognition probability value, and further improving the efficiency of danger identification.

[0071] The process in which the danger identification device inputs the feature image into the global average pooling layer in the threshold network adjacent to the Fire Module network to obtain the feature vector can be implemented based on the third formula, where the third formula is:

[0072]

[0073] In Formula 6, x is the feature image, H is the height of the feature image, W is the width of the feature image, i is the i-th pixel of the feature image in the height direction, and j is the j-th pixel of the feature image in the width direction; GAP represents the global average pooling layer, and GAP(x) is the feature vector.

[0074] The process of the danger identification device processing the feature vector based on the activation function in the threshold network to obtain the identification probability value can be implemented based on the fourth formula, which is:

[0075] [p =σ(W·GAP(x)+b)]; (Formula 7)

[0076] In Formula 7, σ is the Sigmoid function, W is the weight of the global average pooling layer, b is the bias term, and p is the recognition probability value.

[0077] In an optional implementation, the danger identification device inputs the feature image into a threshold network adjacent to the Fire Module network, and the process of obtaining the recognition probability value may include: inputting the feature image into a fully connected layer in the threshold network adjacent to the Fire Module network to obtain a feature vector; then, processing the feature vector based on the activation function in the threshold network to obtain the recognition probability value; the integrity of the feature information retained in the feature vector can be ensured, thereby improving the accuracy of determining the recognition probability value.

[0078] It should be noted that in the embodiments of the present disclosure, in order to further improve the accuracy and reliability of the determined hazard identification results, after determining the image frame to be processed, the image frame to be processed can be preprocessed, wherein the process of preprocessing the image to be processed by the hazard identification device can refer to the process of preprocessing the sample image frame in the above embodiment, and the embodiments of the present disclosure will not elaborate on this.

[0079] In step S202, the danger identification device may determine the second danger identification network group adjacent to the first danger identification network group as the updated first danger identification network group when the identification probability value is less than or equal to a preset threshold, and determine the feature image as the updated image frame to be processed.

[0080] In the embodiment of the present disclosure, the preset threshold is a reference value for determining whether any danger identification network in the danger identification model participates in the danger identification process. The preset threshold can be determined based on actual needs, and the embodiment of the present disclosure does not limit this. For example, the preset threshold can be 0.5.

[0081] In an optional implementation, the danger identification device may determine the first danger identification network group as the first danger identification network group when the recognition probability value is greater than a preset threshold, and determine the characteristic image as the updated image frame to be processed. When the recognition probability value is greater than a preset threshold, the next danger identification network may be determined to participate in the processing flow to ensure the reliability of the danger identification process and to improve the accuracy of the determined danger identification result.

[0082] It can be understood that in the embodiment of the present disclosure, when the recognition probability value is greater than a preset threshold, the danger identification device can determine that the first danger identification network group needs to participate in danger identification, and determine the first danger identification network group as the first danger identification network group after update, wherein some danger identification networks in the danger identification model that do not participate in the danger identification process are determined based on the image features of the image frames to be processed in the current video stream collected in real time, which can improve the reliability of the determined danger identification results.

[0083] In step S204, the danger recognition device may start by inputting the image frame to be processed into the first danger recognition network group in the pre-trained danger recognition model, and repeat the above process until each danger recognition network group in the danger recognition model is traversed to obtain the current danger recognition result.

[0084] In the embodiment of the present disclosure, the danger identification result output by the danger identification model may be a probability value of danger occurrence, wherein when the probability value of danger occurrence is greater than a probability threshold, it is determined that a danger has occurred; the probability threshold may be determined based on actual needs, and the embodiment of the present disclosure does not limit this.

[0085] It should be noted that in the embodiment of the present disclosure, the danger identification device can process each image frame to be processed in chronological order based on the process in the above steps S202 to S204, so as to timely and accurately determine whether a danger occurs at present.

[0086] In an optional implementation, the danger identification device may also determine the current video stream as a sample video stream, and determine the current danger identification result as a video tag, to obtain an updated sample video data set; then, after a preset time, the danger identification model is trained based on the updated sample video data set to obtain an updated danger identification model. The danger identification model may be updated periodically to improve the matching degree between the recognition capability of the danger identification model and the current actual environmental conditions, and to ensure the accuracy and reliability of the danger identification result determined by the danger identification model.

[0087] It can be understood that in the embodiments of the present disclosure, the danger recognition device can also provide a danger warning when determining that a danger has occurred, wherein the danger recognition device can send a danger signal to the alarm device so that the alarm device issues an alarm after receiving the danger signal; or, the danger recognition device can send a danger warning message to a terminal device used by maintenance personnel (such as a mobile phone or a wearable device, etc.) to remind the maintenance personnel that a danger has occurred.

[0088] Figure 4 A flowchart of another method for identifying a dangerous situation according to an exemplary embodiment of the present disclosure is shown. Figure 4 As shown, the method of the embodiment of the present disclosure may include:

[0089] Step S401, receiving a current video stream sent by an image acquisition device;

[0090] Step S402, inputting the image frame to be processed into the first hazard recognition network group in the pre-trained hazard recognition model to obtain a feature image and a recognition probability value;

[0091] Step S403, determining whether the recognition probability value is less than or equal to a preset threshold;

[0092] Step S404, when the recognition probability value is less than or equal to the preset threshold, determining the second danger recognition network group adjacent to the first danger recognition network group as the updated first danger recognition network group, and determining the feature image as the updated image frame to be processed;

[0093] Step S405, when the recognition probability value is greater than a preset threshold, determining the first danger recognition network group as the first danger recognition network group, and determining the feature image as the updated image frame to be processed;

[0094] Step S406, starting from inputting the image frame to be processed into the first hazard recognition network group in the pre-trained hazard recognition model, repeating the above process until each hazard recognition network group in the hazard recognition model is traversed to obtain the current hazard recognition result;

[0095] Step S407: When it is determined that a dangerous situation has occurred, a dangerous situation warning is issued.

[0096] An exemplary embodiment of the present disclosure provides a danger identification apparatus, which may be a danger identification device or a chip applied to a danger identification device. Figure 5 FIG. 1 shows a schematic block diagram of functional modules of a danger identification device according to an exemplary embodiment of the present disclosure. Figure 5 As shown, the danger identification device 500 includes:

[0097] The receiving module 501 is configured to receive a current video stream sent by an image acquisition device, wherein the current video stream includes a plurality of image frames to be processed;

[0098] The first acquisition module 502 is configured to input the image frame to be processed into the first danger recognition network group in the pre-trained danger recognition model to obtain a feature image and a recognition probability value, wherein the first danger recognition network group includes a danger recognition network and a threshold network, the danger recognition network is used to determine the feature image of the image frame to be processed, the threshold network is used to determine the recognition probability value according to the feature image, and the recognition probability value is used to represent the probability that the first danger recognition network group adjacent to the first danger recognition network group participates in danger recognition;

[0099] The determination module 503 is configured to determine, when the recognition probability value is less than or equal to a preset threshold, a second danger identification network group adjacent to the first danger identification network group as the updated first danger identification network group, and determine the feature image as the updated image frame to be processed;

[0100] The second acquisition module 504 is configured to start from inputting the image frame to be processed into the first hazard recognition network group in the pre-trained hazard recognition model, and repeat the above process until each hazard recognition network group in the hazard recognition model is traversed to obtain the current hazard recognition result.

[0101] Optionally, the danger identification model is constructed based on a SqueezeNet network, and the danger identification network is a Fire Module network.

[0102] The first acquisition module 502 is configured to:

[0103] Inputting the image frame to be processed into the Fire Module network in the first danger identification network group to obtain the feature image;

[0104] The feature image is input into a threshold network adjacent to the Fire Module network in the first danger identification network group to obtain the identification probability value.

[0105] Optionally, the first acquisition module 502 is configured to:

[0106] Inputting the feature image into a global average pooling layer in a threshold network adjacent to the Fire Module network to obtain a feature vector;

[0107] The feature vector is processed based on the activation function in the threshold network to obtain the recognition probability value.

[0108] Optionally, the determining module 503 is further configured to:

[0109] When the recognition probability value is greater than a preset threshold, the first danger recognition network group is determined as the first danger recognition network group, and the feature image is determined as an updated image frame to be processed.

[0110] Optionally, the device further includes an updating module 505 configured to:

[0111] Determine the current video stream as a sample video stream, and determine the current danger identification result as a video label, to obtain an updated sample video data set;

[0112] After a preset time period, the hazard recognition model is trained based on the updated sample video data set to obtain an updated hazard recognition model.

[0113] Optionally, the apparatus further includes a training module 506 configured to:

[0114] Acquire a sample video data set, wherein the sample video data set includes multiple sample video streams and a video label corresponding to each sample video stream;

[0115] Inputting the sample image frame in the sample video stream into the first danger recognition network group in the danger recognition model to be trained, obtaining a sample feature map of the sample image and a sample recognition probability value of a first danger recognition network group adjacent to the first danger recognition network group;

[0116] In the case where the sample recognition probability value is less than the preset threshold, a second danger recognition network group adjacent to the first danger recognition network group is determined as the updated first danger recognition network group, and the sample feature map is determined as the updated sample image frame, and the sample image frame in the sample video stream is input into the first danger recognition network group in the danger recognition model to be trained, and the above process is repeated until each danger recognition network group in the danger recognition model is traversed to obtain a sample danger recognition result of the sample video stream;

[0117] Based on the video label, the sample hazard identification result and the loss function, a loss function value of the current iteration process is determined, and when the loss function value is less than or equal to a loss function threshold, it is determined that the hazard identification model has converged.

[0118] Optionally, the training module 506 is configured to:

[0119] Determine a data loss value of a current iteration process based on the video label, the sample danger identification result, and the data loss term in the loss function;

[0120] Determining the regularized loss value of the current iteration process based on the sample recognition probability value of each hazard recognition network group and the regularized loss term in the loss function;

[0121] Determine the sum of the data loss value and the regularization loss value to obtain the loss function value of the current iteration process.

[0122] The exemplary embodiment of the present disclosure also provides an electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication. The memory stores a computer program that can be executed by the at least one processor, and the computer program is used to cause the electronic device to perform the method according to the embodiment of the present disclosure when executed by the at least one processor.

[0123] The exemplary embodiments of the present disclosure also provide a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to perform the method according to the embodiments of the present disclosure.

[0124] like Figure 6 As shown, the exemplary embodiment of the present disclosure further provides a computer program product 600, including a computer program 601, wherein the computer program 601 is used to enable the computer to perform the method according to the embodiment of the present disclosure when being executed by a processor of a computer.

[0125] refer to Figure 7 , a block diagram of an electronic device 700 that can be used as a server or client of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0126] like Figure 7 As shown, the electronic device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0127] A plurality of components in the electronic device 700 are connected to the I / O interface 705, including: an input unit 706, an output unit 707, a storage unit 708, and a communication unit 709. The input unit 706 may be any type of device capable of inputting information to the electronic device 700, and the input unit 706 may receive input digital or character information, and generate key signal inputs related to user settings and / or function control of the electronic device. The output unit 707 may be any type of device capable of presenting information, and may include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 708 may include but is not limited to a disk, an optical disk. The communication unit 709 allows the electronic device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth™ device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0128] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above. For example, in some embodiments, the method of the embodiment of the present disclosure may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 700 via the ROM 702 and / or the communication unit 709. In some embodiments, the computing unit 701 may be configured to perform the method of the embodiment of the present disclosure by any other appropriate means (e.g., by means of firmware).

[0129] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0130] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0131] As used in this disclosure, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0132] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0133] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0134] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0135] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in the embodiment of the present disclosure is executed in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a terminal, a user device or other programmable device. The computer program or instruction may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instruction may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server, data center, etc. that integrates one or more available media. The available medium may be a magnetic medium, for example, a floppy disk, a hard disk, a tape; it may also be an optical medium, for example, a digital video disc (DVD); it may also be a semiconductor medium, for example, a solid state drive (SSD).

[0136] Although the present disclosure has been described in conjunction with specific features and embodiments thereof, it is apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the present disclosure. Accordingly, this specification and the drawings are merely exemplary illustrations of the present disclosure as defined by the appended claims, and are deemed to have covered any and all modifications, variations, combinations or equivalents within the scope of the present disclosure. Obviously, those skilled in the art may make various modifications and variations to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is also intended to include these modifications and variations.

Claims

1. A method for identifying a dangerous situation, characterized in that: The method is applied to a danger identification device, comprising: Receiving a current video stream sent by an image acquisition device, wherein the current video stream includes a plurality of image frames to be processed; Inputting the image frame to be processed into the first danger recognition network group in the pre-trained danger recognition model to obtain a characteristic image and a recognition probability value, wherein the first danger recognition network group includes a danger recognition network and a threshold network, the danger recognition network is used to determine the characteristic image of the image frame to be processed, the threshold network is used to determine the recognition probability value according to the characteristic image, and the recognition probability value is used to represent the probability that the first danger recognition network group adjacent to the first danger recognition network group participates in danger recognition; When the recognition probability value is less than or equal to a preset threshold, determining a second danger recognition network group adjacent to the first danger recognition network group as the updated first danger recognition network group, and determining the feature image as the updated image frame to be processed; Starting from inputting the image frame to be processed into the first hazard recognition network group in the pre-trained hazard recognition model, the above process is repeated until each hazard recognition network group in the hazard recognition model is traversed to obtain the current hazard recognition result.

2. The method for identifying a dangerous situation according to claim 1, characterized in that: The danger identification model is constructed based on the SqueezeNet network, and the danger identification network is a Fire Module network. The step of inputting the image frame to be processed into the first hazard recognition network group in the pre-trained hazard recognition model to obtain a feature image and a recognition probability value comprises: Inputting the image frame to be processed into the Fire Module network in the first danger identification network group to obtain the feature image; The feature image is input into a threshold network adjacent to the Fire Module network in the first danger identification network group to obtain the identification probability value.

3. The method for identifying a dangerous situation according to claim 1, characterized in that: The step of inputting the feature image into a threshold network adjacent to the Fire Module network to obtain a recognition probability value includes: Inputting the feature image into a global average pooling layer in a threshold network adjacent to the Fire Module network to obtain a feature vector; The feature vector is processed based on the activation function in the threshold network to obtain the recognition probability value.

4. The method for identifying a dangerous situation according to claim 1, characterized in that: The method further comprises: When the recognition probability value is greater than a preset threshold, the first danger recognition network group is determined as the first danger recognition network group, and the feature image is determined as an updated image frame to be processed.

5. The method for identifying a dangerous situation according to claim 1, characterized in that: The method further comprises: Determine the current video stream as a sample video stream, and determine the current danger identification result as a video label, to obtain an updated sample video data set; After a preset time period, the hazard recognition model is trained based on the updated sample video data set to obtain an updated hazard recognition model.

6. The method for identifying a dangerous situation according to claim 1, characterized in that: The method further comprises: Acquire a sample video data set, wherein the sample video data set includes multiple sample video streams and a video label corresponding to each sample video stream; Inputting the sample image frame in the sample video stream into the first danger recognition network group in the danger recognition model to be trained, obtaining a sample feature map of the sample image and a sample recognition probability value of a first danger recognition network group adjacent to the first danger recognition network group; In the case where the sample recognition probability value is less than the preset threshold, a second danger recognition network group adjacent to the first danger recognition network group is determined as the updated first danger recognition network group, and the sample feature map is determined as the updated sample image frame, and the sample image frame in the sample video stream is input into the first danger recognition network group in the danger recognition model to be trained, and the above process is repeated until each danger recognition network group in the danger recognition model is traversed to obtain a sample danger recognition result of the sample video stream; Based on the video label, the sample hazard identification result and the loss function, a loss function value of the current iteration process is determined, and when the loss function value is less than or equal to a loss function threshold, it is determined that the hazard identification model has converged.

7. The method for identifying a dangerous situation according to claim 6, characterized in that: The determining of the loss function value of the current iteration process based on the video label, the sample danger identification result and the loss function includes: Determine a data loss value of a current iteration process based on the video label, the sample danger identification result, and the data loss term in the loss function; Determining the regularized loss value of the current iteration process based on the sample recognition probability value of each hazard recognition network group and the regularized loss term in the loss function; Determine the sum of the data loss value and the regularization loss value to obtain the loss function value of the current iteration process.

8. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images