Fire fighting truck fire identification method and system, electronic equipment and storage medium

Abstract

The invention relates to the technical field of image recognition, in particular to a fire fighting truck fire recognition method and system, electronic equipment and a storage medium, and the method comprises the steps: obtaining a to-be-recognized image which is obtained based on an original image shot by a vehicle-mounted camera of a fire fighting truck; inputting the to-be-recognized image into a preset feature extraction network, and performing feature extraction to obtain a one-dimensional feature vector; a target dictionary is obtained, the target dictionary comprises a plurality of target key value pairs, the target key value pairs comprise target keys and target values in one-to-one correspondence with the target keys, and the target keys refer to fire hazard types; performing linear processing on the target value to obtain a value embedding vector; the one-dimensional feature vector and the value embedding vector are spliced, and fire recognition is completed according to the spliced vector; according to the method, the fire type in the fire image can be accurately identified, the accuracy is high, and the cost is low.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a fire recognition method, system, electronic equipment and storage medium for a fire truck. Background Art

[0002] The fire environment that fire trucks face when performing their tasks is becoming increasingly complex and changeable. In order to discover and control fires as early as possible, it is necessary to identify and analyze fires based on the on-site images taken by fire trucks to help command terminals make correct decisions quickly.

[0003] However, fire image recognition is difficult due to the changeable and harsh fire environment. For example, when a fire truck drives near a fire scene, the angle between the preset camera on the fire truck and the target may change significantly due to the light being blocked by thick fog, which causes the appearance and shape of the same fire in the image to change, thereby increasing the difficulty of classification or detection tasks. In addition, due to the influence of light factors, the fire is often small in the early stages of the fire, and the contrast with the surrounding environment is low, further increasing the difficulty of fire recognition. Summary of the invention

[0004] The present invention provides a fire identification method, system, electronic equipment and storage medium for a fire truck, so as to solve the problem of low identification accuracy of fire images taken by a fire truck in the prior art.

[0005] The present invention provides a fire identification method for a fire truck, the method comprising: obtaining an image to be identified, the image to be identified being obtained based on an original image taken by a vehicle-mounted camera of the fire truck;

[0006] Inputting the image to be identified into a preset feature extraction network to perform feature extraction to obtain a one-dimensional feature vector;

[0007] Obtain a target dictionary, the target dictionary including a plurality of target key-value pairs, the target key-value pairs including a target key and a target value corresponding to the target key one by one, the target key indicating a fire type;

[0008] Performing linear processing on the target value to obtain a value embedding vector;

[0009] The one-dimensional feature vector and the value embedding vector are concatenated, and fire identification is completed according to the concatenated vector.

[0010] In one embodiment of the present invention, the step of acquiring an image to be recognized includes:

[0011] The plurality of images to be identified are obtained by randomly enhancing the original image, wherein the random enhancement includes: rotation, color change, translation, scaling, cropping, flipping, and sharpening.

[0012] In one embodiment of the present invention, the step of inputting the image to be identified into a preset feature extraction network to extract features and obtain a one-dimensional feature vector includes:

[0013] Input the image to be identified into the large-core hole convolution neural network of the feature extraction network, perform feature extraction, and obtain a target feature map. The large-core hole convolution neural network includes multiple large-core hole convolution layers. The large-core hole convolution layer is used to perform a large-core hole convolution on the input data to obtain a first feature vector; perform a second large-core hole convolution on the first feature vector to obtain a second feature vector; sum the first feature vector and the second feature vector to obtain a third feature vector; determine the product of the third feature vector and the input data as an output vector; output the output vector to the next layer of the network; the output vector output by the last large-core hole convolution layer is the target feature map;

[0014] The target feature map is input into the linear layer of the feature extraction network and linearly processed to obtain the one-dimensional feature vector.

[0015] In one embodiment of the present invention, the one-dimensional feature vector and the value embedding vector are concatenated, and the step of completing fire identification according to the concatenated vector includes:

[0016] Acquire a target position vector, where the target position vector is obtained based on position information of a shooting location of the original image;

[0017] Adding the target position vector to the one-dimensional feature vector to obtain a first intermediate vector;

[0018] Adding the target position vector to the value embedding vector to obtain a second intermediate vector;

[0019] splicing the first intermediate vector, the second intermediate vector, and a preset classification mark to obtain a target splicing vector;

[0020] Input the target concatenated vector into a preset multi-head attention mechanism network, perform global attention mechanism calculation, and obtain a target feature vector;

[0021] The target feature vector is input into a preset fully connected layer for fire classification and location, so as to obtain a fire classification result and a fire location result.

[0022] In one embodiment of the present invention, after the one-dimensional feature vector and the value embedding vector are concatenated and the fire identification step is completed according to the concatenated vector, the method further includes:

[0023] Encrypting and packaging the fire classification result and the fire location result to obtain a target data packet;

[0024] Sending a data upload request to a cloud server, wherein the data upload request is used to instruct the cloud server to perform authority verification on the user information carried in the data upload request, and if the verification is successful, establishing a data transmission channel;

[0025] The target data packet is sent to the cloud server using the data transmission channel. The target data packet is used to instruct the cloud server to decompress and decrypt the target data packet, store the decrypted data in a preset database, and send a fire notification to the target terminal.

[0026] In one embodiment of the present invention, the training step of the feature extraction network includes:

[0027] Acquire a training set, wherein the training set includes a plurality of sample images;

[0028] Inputting the sample image into the feature extraction network to perform feature extraction to obtain a plurality of one-dimensional feature vector samples;

[0029] Constructing a cosine similarity matrix, wherein the cosine similarity matrix includes cosine similarities between two of the plurality of one-dimensional feature vector samples;

[0030] According to a preset contrastive learning loss function and the cosine similarity, a similarity loss is obtained, wherein the similarity loss corresponds to the cosine similarity in a one-to-one manner;

[0031] By averaging all the similarity losses, an average loss is obtained;

[0032] The feature extraction network is trained based on the average loss.

[0033] In one embodiment of the present invention, the feature extraction network belongs to a preset fire recognition model, and the fire recognition model further includes: a multi-head attention mechanism network and a fully connected layer; the training set also includes real labels corresponding to the sample images one by one; the training steps of the fire recognition model include:

[0034] Acquire a target position vector sample, wherein the target position vector sample is obtained based on position information of a shooting location of the sample image;

[0035] Adding the target position vector sample to the one-dimensional feature vector sample to obtain a first intermediate vector sample;

[0036] Add a preset value embedding vector to the target position vector sample to obtain a second intermediate vector sample;

[0037] Splicing the first intermediate vector sample, the second intermediate vector sample, and a preset classification label to obtain a target spliced ​​vector sample;

[0038] Input the target concatenated vector sample into the multi-head attention mechanism network, perform global attention mechanism calculation, and obtain a target feature vector sample;

[0039] Inputting the target feature vector sample into the fully connected layer for fire classification and location to obtain fire classification information and fire location information;

[0040] Obtaining a classification loss based on a preset cross entropy loss function, the fire classification information, and a true label corresponding to the fire classification information;

[0041] The fire identification model is trained based on the classification loss.

[0042] The present invention also provides a fire identification system for a fire truck, the system comprising: an image acquisition module, used to acquire an image to be identified, the image to be identified being obtained based on an original image captured by a vehicle-mounted camera of the fire truck;

[0043] A feature extraction module, used for inputting the image to be identified into a preset feature extraction network to perform feature extraction and obtain a one-dimensional feature vector;

[0044] A dictionary acquisition module is used to acquire a target dictionary, wherein the target dictionary includes a plurality of target key-value pairs, wherein the target key-value pairs include a target key and a target value corresponding to the target key one by one, wherein the target key refers to a fire type;

[0045] A linear processing module, used for performing linear processing on the target value to obtain a value embedding vector;

[0046] The fire identification module is used to concatenate the one-dimensional feature vector with the value embedding vector, and complete fire identification according to the concatenated vector.

[0047] The present invention also provides an electronic device, comprising a processor, a memory and a communication bus; the communication bus is used to connect the processor and the memory; the processor is used to execute a computer program stored in the memory to implement a fire identification method for a fire truck as provided in any one of the above embodiments.

[0048] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is used to enable a computer to execute the fire identification method for a fire truck as provided in any one of the above embodiments.

[0049] Beneficial effects of the present invention: The fire identification method, system, electronic device and storage medium provided by the present invention for fire trucks, the method comprises: obtaining an image to be identified, the image to be identified is obtained based on the original image taken by the onboard camera of the fire truck; inputting the image to be identified into a preset feature extraction network, performing feature extraction, and obtaining a one-dimensional feature vector; obtaining a target dictionary, the target dictionary includes multiple target key-value pairs, the target key-value pairs include a target key, and a target value corresponding to the target key one by one, the target key refers to the fire type; performing linear processing on the target value to obtain a value embedding vector; splicing the one-dimensional feature vector and the value embedding vector, and completing fire identification according to the spliced ​​vector. The method can more accurately realize the identification of the fire type in the fire image, with high accuracy and low cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A schematic diagram of a flow chart of a fire identification method for a fire truck provided in one embodiment of the present invention;

[0051] Figure 2 A schematic diagram of the structure of a feature extraction network in a fire identification method for a fire truck provided in one embodiment of the present invention;

[0052] Figure 3 An exemplary schematic diagram of a cosine similarity matrix in a fire identification method for a fire truck provided by an embodiment of the present invention;

[0053] Figure 4 It is a structural diagram of a multi-head attention mechanism network in a fire identification method for a fire truck provided by an embodiment of the present invention;

[0054] Figure 5 It is a structural schematic diagram of a fire identification system for a fire truck provided by an embodiment of the present invention;

[0055] Figure 6 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0057] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0058] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0059] Combine the following Figures 1 to 6 , the fire identification method, system, electronic device and storage medium of the fire truck provided by the present invention are explained.

[0060] See also Figure 1 , Figure 1 A flow chart of a fire identification method for a fire truck provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method includes:

[0061] S110: Acquire an image to be identified, wherein the image to be identified is obtained based on an original image captured by a vehicle-mounted camera of a fire truck. It should be noted that the original image refers to an image of a fire scene captured by a vehicle-mounted camera of a fire truck.

[0062] S120: Input the image to be identified into a preset feature extraction network to perform feature extraction to obtain a one-dimensional feature vector.

[0063] S130: Obtain a target dictionary, wherein the target dictionary includes a plurality of target key-value pairs, wherein the target key-value pairs include a target key and a target value corresponding to the target key one by one, wherein the target key refers to a fire type. For example, a target key-value pair {solid material fire: 34}, wherein solid material fire is a target key in the target key-value pair, and 34 is a target value in the target key-value pair.

[0064] S140: linearly process the target value to obtain a value embedding vector. It should be noted that by linearly processing the target value to obtain a value embedding vector, subsequent processing can be facilitated. The value embedding vector includes text prompt information of flames and dense fog.

[0065] S150: splicing the one-dimensional feature vector and the value embedding vector, and completing fire identification according to the spliced ​​vector. It should be noted that by splicing the one-dimensional feature vector and the value embedding vector, and completing fire identification according to the spliced ​​vector, the accuracy of fire identification or classification can be effectively improved, the cost is low, and the credibility is high.

[0066] In some embodiments, the step of acquiring the image to be recognized includes:

[0067] By randomly enhancing the original image, a plurality of images to be identified are obtained, wherein the random enhancement includes: rotation, color change, translation, scaling, cropping, flipping, and sharpening. It should be noted that by randomly enhancing the original image, the richness of the features subsequently extracted can be improved, which helps to improve the accuracy of subsequent fire identification.

[0068] In some embodiments, the step of inputting the image to be identified into a preset feature extraction network to extract features and obtain a one-dimensional feature vector includes:

[0069] 1. Input the image to be identified into the large-core hole convolutional neural network of the feature extraction network to perform feature extraction to obtain a target feature map. The large-core hole convolutional neural network includes multiple large-core hole convolutional layers. The large-core hole convolutional layer is used to perform a large-core hole convolution on the input data to obtain a first feature vector; perform a second large-core hole convolution on the first feature vector to obtain a second feature vector; sum the first feature vector and the second feature vector to obtain a third feature vector; determine the product of the third feature vector and the input data as an output vector; output the output vector to the next layer of the network; the output vector output by the last large-core hole convolution layer is the target feature map.

[0070] It should be noted that the mathematical expression of the large-core atrous convolution layer is:

[0071]

[0072] U i+1 =U i *S i

[0073] U0=X

[0074] Among them, S i represents the third eigenvector, Represents large-kernel dilated convolution, which refers to a deep convolution with a convolution kernel of k and a dilation rate of d. The values ​​of k and d can be set according to actual needs. irepresents the data input to the i-th large-core hole convolution layer, U0 represents the data input to the 0-th large-core hole convolution layer, X represents the image to be recognized, and U i+1 It represents the data input to the i+1th large-kernel dilated convolutional layer, that is, the data output by the i-th large-kernel dilated convolutional layer.

[0075] It should also be noted that the first eigenvector and the second eigenvector are added according to the spatial dimension to obtain the third eigenvector.

[0076] It is worth mentioning that through the above steps, rare fire features (such as specific color changes and smoke shapes, etc.) in the image to be identified can be extracted, which helps to improve the accuracy of subsequent fire identification or classification. By inputting the image to be identified into a large-core atrous convolutional neural network for feature extraction, multi-scale feature analysis of the image to be identified can be achieved and the demand for computing resources can be reduced.

[0077] 2. Input the target feature map into the linear layer of the feature extraction network, perform linear processing, and obtain the one-dimensional feature vector.

[0078] It should be noted that the mathematical expression of the step of inputting the target feature map into the linear layer of the feature extraction network and performing linear processing to obtain the one-dimensional feature vector is:

[0079] P x =Linear(U x )

[0080] Among them, P x represents a one-dimensional feature vector, Linear(·) represents linear processing, U x represents the target feature map, that is, the output vector of the x-th large-kernel dilated convolutional layer, and x represents the number of the last large-kernel dilated convolutional layer.

[0081] Figure 2 FIG. 1 is a schematic diagram of a structure of a feature extraction network in a fire truck fire identification method provided in an embodiment of the present invention. Please refer to FIG. Figure 2 The feature extraction network includes a large-core hole convolutional neural network and a linear layer, and the large-core hole convolutional neural network includes multiple large-core hole convolutional layers. The input of the feature extraction network is multiple images to be recognized (such as Figure 2 The output is multiple one-dimensional feature vectors. Figure 2 LK in Spatial Selection means vector splicing.

[0082] It should also be noted that by performing linear processing on the target feature map, the difficulty and complexity of subsequent processing can be reduced.

[0083] In some embodiments, the one-dimensional feature vector is concatenated with the value embedding vector, and the step of completing fire identification according to the concatenated vector includes:

[0084] 1. Obtain a target position vector, where the target position vector is obtained based on the position information of the shooting location of the original image.

[0085] 2. Add the target position vector to the one-dimensional feature vector to obtain a first intermediate vector. The mathematical expression of the first intermediate vector is: P′ x =P x +w, where P′ x represents the first intermediate vector, and w represents the target position vector.

[0086] 3. Add the target position vector to the value embedding vector to obtain a second intermediate vector. The mathematical expression of the second intermediate vector is: E′ j =E j +w, where E′ j represents the second intermediate vector, E j Represents a value embedding vector.

[0087] Fourth, concatenate the first intermediate vector, the second intermediate vector and a preset classification mark (CLS mark) to obtain a target concatenated vector.

[0088] It should be noted that by splicing the first intermediate vector, the second intermediate vector and the preset classification mark (CLS mark), it is possible to facilitate subsequent learning of the correlation between the image features (first intermediate vector) and the text vector (second intermediate vector), thereby improving the accuracy of fire identification or classification.

[0089] 5. Input the target splicing vector into a preset multi-head attention mechanism network, perform global attention mechanism calculation, and obtain the target feature vector.

[0090] It should be noted that the calculation formula of the multi-head attention mechanism network is as follows:

[0091] Q=W q Y

[0092] K=W k Y

[0093] V=W v Y

[0094]

[0095] Among them, Q represents the query matrix, K represents the key matrix, V represents the value matrix, and W qrepresents the first learned weight matrix, W k represents the learned second weight matrix, W v represents the learned third weight matrix, Attention(Q, K, V) represents the target feature vector, softmax(·) represents the softmax (soft maximum function) function, (·) T represents transpose, d k Indicates the preset scaling factor, QK T Indicates the dot product operation of Q and KT to obtain the attention weight of Q on V. By setting Scaling can avoid Q and K T The dot product of is too large to prevent the gradient value from becoming extremely small. In addition, by setting the softmax function, it is easy to perform gradient calculation for back propagation.

[0096] Figure 4 This is a schematic diagram of the structure of a multi-head attention mechanism network in a fire truck fire identification method provided by an embodiment of the present invention. Please refer to Figure 4 , the input of the multi-head attention mechanism network includes: target position vector, CLS tag, value embedding vector, and one-dimensional feature vector (feature map embedding). Figure 4 The Encoder Layer in the figure represents the encoder layer of the multi-head attention mechanism network, which is used to embed the target position vector, CLS tag, one-dimensional feature vector (feature map embedding) (such as Figure 4 P1, P2, ... in ), and value embedding vectors (such as Figure 4 E0, E1, E2...) in the code, and then the encoded data ( Figure 4 The encoded data (referred to as encoded data in the text) is input into another encoder layer for vector concatenation and multi-head global attention mechanism calculation. Figure 4 The Head in represents the attention head.

[0097] 6. Input the target feature vector into a preset fully connected layer for fire classification and location, so as to obtain fire classification results and fire location results. It should be noted that through the above steps, fire classification results and fire location results with high accuracy can be obtained.

[0098] In some embodiments, after concatenating the one-dimensional feature vector and the value embedding vector and completing the step of fire identification according to the concatenated vector, the method further includes:

[0099] 1. Encrypt and package the fire classification result and the fire location result to obtain a target data packet.

[0100] Specifically, the fire classification result and the fire location result are encrypted and packaged through a preset communication protocol (such as 4G / 5G mobile communication network, satellite communication or dedicated wireless transmission network, etc.).

[0101] 2. Send a data upload request to the cloud server, wherein the data upload request is used to instruct the cloud server to perform authority verification on the user information carried in the data upload request. If the verification is successful, a data transmission channel is established.

[0102] 3. Using the data transmission channel, the target data packet is sent to the cloud server, and the target data packet is used to instruct the cloud server to decompress and decrypt the target data packet, store the decrypted data in a preset database, and send a fire notification to the target terminal.

[0103] It should be noted that through the above steps, fire disaster warning can be better achieved, and fire alarm information, i.e., fire notification, can be sent to the relevant fire command center, rescue team or person in charge in time so that they can take response measures in time and quickly mobilize resources to go to the fire scene for fire fighting.

[0104] In some embodiments, the step of training the feature extraction network includes:

[0105] 1. Obtain a training set, wherein the training set includes a plurality of sample images. The plurality of sample images include fire scene images at different scales (resolutions / sizes) and at different angles and under different illuminations.

[0106] 2. Input the sample image into the feature extraction network to perform feature extraction to obtain multiple one-dimensional feature vector samples.

[0107] 3. Construct a cosine similarity matrix, wherein the cosine similarity matrix includes cosine similarities between two of the plurality of one-dimensional feature vector samples.

[0108] It should be noted that the mathematical expression for obtaining the cosine similarity between two one-dimensional feature vector samples (including themselves) is:

[0109]

[0110] Among them, S i，j represents the cosine similarity between the i-th one-dimensional feature vector sample and the j-th one-dimensional feature vector sample, μ is the adjustable first Temperature parameter, which can extend the cosine similarity to the range [-1, 1], ||·|| represents the modulus of the orientation vector, and g i represents a one-dimensional feature vector sample, g j Represents another one-dimensional feature vector sample.

[0111] Figure 3 For an exemplary schematic diagram of a cosine similarity matrix in a fire truck fire identification method provided by an embodiment of the present invention, please refer to Figure 3 , taking four one-dimensional feature vector samples as an example, by Figure 3 The cosine similarity calculation is performed between samples 1, 2, 3, and 4 (including themselves) to obtain S1,1, S1,2, S1,3, S1,4, S2,1, S2,2, S 2，3 , S 2，4 , S 3，1 , S 3，2 , S 3，3 , S 3，4 , S 4，1 , S 4，2 , S 4，3 , and S 4，4 , forming a cosine similarity matrix.

[0112] 4. According to the preset contrastive learning loss function and the cosine similarity, a similarity loss is obtained, and the similarity loss corresponds to the cosine similarity one by one.

[0113] It should be noted that the mathematical expression of the contrastive learning loss function is:

[0114]

[0115] Among them, loss i，j represents the similarity loss of the cosine similarity of the i-th row and j-th column, S i，k represents the cosine similarity between the i-th one-dimensional feature vector sample and the k-th one-dimensional feature vector sample. log represents the logarithmic function, exp represents the exponential function, 2N represents the number of one-dimensional feature vector samples, "!=" represents not equal to, and ρ is the adjustable second Temperature parameter.

[0116] 5. Obtain an average loss by averaging all the similarity losses. Through the above steps, the distribution and aggregation degree of the fire type features of the same category can be improved.

[0117] It should be noted that, by averaging all the similarity losses, the mathematical expression of the average loss is:

[0118]

[0119] Among them, L represents the average loss, g represents the number of similarity losses, and loss i，j (e) represents the e-th similarity loss.

[0120] 6. Based on the average loss, the feature extraction network is trained. It should be noted that the above steps can effectively improve the accuracy of the feature extraction network.

[0121] In some embodiments, the feature extraction network belongs to a preset fire recognition model, and the fire recognition model further includes: a multi-head attention mechanism network and a fully connected layer; the training set also includes real labels corresponding to the sample images one by one; the training steps of the fire recognition model include:

[0122] 1. Obtain a target position vector sample, wherein the target position vector sample is obtained based on the position information of the shooting location of the sample image.

[0123] 2. Add the target position vector sample to the one-dimensional feature vector sample to obtain a first intermediate vector sample.

[0124] 3. Add a preset value to the target position vector sample to embed the vector to obtain a second intermediate vector sample.

[0125] 4. Splice the first intermediate vector sample, the second intermediate vector sample and a preset classification label to obtain a target spliced ​​vector sample.

[0126] 5. Input the target splicing vector sample into the multi-head attention mechanism network, perform global attention mechanism calculation, and obtain the target feature vector sample.

[0127] 6. Input the target feature vector sample into the fully connected layer for fire classification and location to obtain fire classification information and fire location information.

[0128] 7. Based on a preset cross entropy loss function, the fire classification information, and a true label corresponding to the fire classification information, a classification loss is obtained.

[0129] It should be noted that the mathematical expression of the cross entropy loss function is:

[0130]

[0131] Among them, L CE represents the classification loss, l represents the amount of fire classification information, y uc is 0 or 1. When the fire classification information u is the same as the true label, y uc Take 1, when the fire classification information u is different from the true label, y uc Take 0, p uc It refers to the probability that the fire classification information u is of category c, and M represents the total number of fire categories.

[0132] 8. Based on the classification loss, the fire identification model is trained.

[0133] It should be noted that by training the fire identification model based on the classification loss, the accuracy of the fire identification model can be improved.

[0134] The fire identification system for a fire truck provided by the present invention is described below. The fire identification system for a fire truck described below and the fire identification method for a fire truck described above can be referred to each other.

[0135] Please refer to Figure 5 The fire identification system for fire trucks provided in this embodiment includes:

[0136] An image acquisition module 510 is used to acquire an image to be recognized, where the image to be recognized is obtained based on an original image captured by a vehicle-mounted camera of a fire truck;

[0137] The feature extraction module 520 is used to input the image to be identified into a preset feature extraction network to perform feature extraction to obtain a one-dimensional feature vector;

[0138] A dictionary acquisition module 530 is used to acquire a target dictionary, wherein the target dictionary includes a plurality of target key-value pairs, wherein the target key-value pairs include a target key and a target value corresponding to the target key one by one, wherein the target key refers to a fire type;

[0139] A linear processing module 540, configured to perform linear processing on the target value to obtain a value embedding vector;

[0140] The fire identification module 550 is used to concatenate the one-dimensional feature vector with the value embedding vector, and complete fire identification according to the concatenated vector. The image acquisition module 510, the feature extraction module 520, the dictionary acquisition module 530, the linear processing module 540, and the fire identification module 550 are connected. The fire identification system of the fire truck in this embodiment can accurately identify the fire type in the fire image with high accuracy and low cost.

[0141] In some embodiments, the image acquisition module 510 is specifically used to obtain multiple images to be identified by randomly enhancing the original image, and the random enhancement includes: rotation, color change, translation, scaling, cropping, flipping, and sharpening.

[0142] In some embodiments, the feature extraction module 520 is specifically used to input the image to be identified into the large-core hole convolutional neural network of the feature extraction network, perform feature extraction, and obtain a target feature map. The large-core hole convolutional neural network includes multiple large-core hole convolutional layers. The large-core hole convolutional layer is used to perform a large-core hole convolution on the input data to obtain a first feature vector; perform a second large-core hole convolution on the first feature vector to obtain a second feature vector; sum the first feature vector and the second feature vector to obtain a third feature vector; determine the product of the third feature vector and the input data as an output vector; output the output vector to the next layer of the network; the output vector output by the last layer of large-core hole convolution layer is the target feature map;

[0143] The target feature map is input into the linear layer of the feature extraction network and linearly processed to obtain the one-dimensional feature vector.

[0144] In some embodiments, the fire identification module 550 is specifically used to obtain a target position vector, and the target position vector is obtained based on the position information of the shooting location of the original image;

[0145] Adding the target position vector to the one-dimensional feature vector to obtain a first intermediate vector;

[0146] Adding the target position vector to the value embedding vector to obtain a second intermediate vector;

[0147] splicing the first intermediate vector, the second intermediate vector, and a preset classification mark to obtain a target splicing vector;

[0148] Input the target concatenated vector into a preset multi-head attention mechanism network, perform global attention mechanism calculation, and obtain a target feature vector;

[0149] The target feature vector is input into a preset fully connected layer for fire classification and location, so as to obtain a fire classification result and a fire location result.

[0150] In some embodiments, it further includes: an uploading module, used to encrypt and package the fire classification result and the fire location result to obtain a target data packet;

[0151] Sending a data upload request to a cloud server, wherein the data upload request is used to instruct the cloud server to perform authority verification on the user information carried in the data upload request, and if the verification is successful, establishing a data transmission channel;

[0152] The target data packet is sent to the cloud server using the data transmission channel. The target data packet is used to instruct the cloud server to decompress and decrypt the target data packet, store the decrypted data in a preset database, and send a fire notification to the target terminal.

[0153] In some embodiments, it further includes: a training module, configured to obtain a training set, wherein the training set includes a plurality of sample images;

[0154] Inputting the sample image into the feature extraction network to perform feature extraction to obtain a plurality of one-dimensional feature vector samples;

[0155] Constructing a cosine similarity matrix, wherein the cosine similarity matrix includes cosine similarities between two of the plurality of one-dimensional feature vector samples;

[0156] According to a preset contrastive learning loss function and the cosine similarity, a similarity loss is obtained, wherein the similarity loss corresponds to the cosine similarity in a one-to-one manner;

[0157] By averaging all the similarity losses, an average loss is obtained;

[0158] The feature extraction network is trained based on the average loss.

[0159] In some embodiments, the training module is further used to obtain a target position vector sample, wherein the target position vector sample is obtained based on the position information of the shooting location of the sample image;

[0160] Adding the target position vector sample to the one-dimensional feature vector sample to obtain a first intermediate vector sample;

[0161] Add a preset value embedding vector to the target position vector sample to obtain a second intermediate vector sample;

[0162] Splicing the first intermediate vector sample, the second intermediate vector sample, and a preset classification label to obtain a target spliced ​​vector sample;

[0163] Input the target concatenated vector sample into the multi-head attention mechanism network, perform global attention mechanism calculation, and obtain a target feature vector sample;

[0164] Inputting the target feature vector sample into the fully connected layer for fire classification and location to obtain fire classification information and fire location information;

[0165] Obtaining a classification loss based on a preset cross entropy loss function, the fire classification information, and a true label corresponding to the fire classification information;

[0166] The fire identification model is trained based on the classification loss.

[0167] In some embodiments, an electronic device is also provided, which may be a server, and its internal structure is shown in FIG. Figure 6 As shown. The electronic device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, the functions or steps on the server side of the above method are implemented.

[0168] In some embodiments, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program: obtaining an image to be identified, wherein the image to be identified is obtained based on an original image captured by an onboard camera of a fire truck; inputting the image to be identified into a preset feature extraction network to perform feature extraction to obtain a one-dimensional feature vector; obtaining a target dictionary, wherein the target dictionary includes multiple target key-value pairs, wherein the target key-value pairs include a target key and a target value corresponding to the target key one by one, wherein the target key refers to a fire type; performing linear processing on the target value to obtain a value embedding vector; and splicing the one-dimensional feature vector and the value embedding vector, and completing fire identification according to the spliced ​​vector.

[0169] In some embodiments, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: obtaining an image to be identified, where the image to be identified is obtained based on an original image taken by a vehicle-mounted camera of a fire truck; inputting the image to be identified into a preset feature extraction network to perform feature extraction to obtain a one-dimensional feature vector; obtaining a target dictionary, where the target dictionary includes multiple target key-value pairs, where the target key-value pairs include a target key and a target value corresponding to the target key one by one, where the target key refers to the type of fire; performing linear processing on the target value to obtain a value embedding vector; splicing the one-dimensional feature vector and the value embedding vector, and completing fire identification based on the spliced ​​vector.

[0170] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or electronic device can refer to the relevant descriptions on the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0171] The flow chart and block diagram in the accompanying drawings illustrate the possible implementation architecture, function and operation of the method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0172] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A fire identification method for a fire truck, characterized in that: include: Acquire an image to be identified, where the image to be identified is obtained based on an original image captured by a vehicle-mounted camera of a fire truck; Inputting the image to be identified into a preset feature extraction network to perform feature extraction to obtain a one-dimensional feature vector; Obtain a target dictionary, the target dictionary including a plurality of target key-value pairs, the target key-value pairs including a target key and a target value corresponding to the target key one by one, the target key indicating a fire type; Performing linear processing on the target value to obtain a value embedding vector; The one-dimensional feature vector and the value embedding vector are concatenated, and fire identification is completed according to the concatenated vector.

2. The fire identification method for a fire truck according to claim 1, characterized in that: The steps of obtaining the image to be recognized include: The plurality of images to be identified are obtained by randomly enhancing the original image, wherein the random enhancement includes: rotation, color change, translation, scaling, cropping, flipping, and sharpening.

3. The fire identification method for a fire truck according to claim 1, characterized in that: The step of inputting the image to be identified into a preset feature extraction network to extract features and obtain a one-dimensional feature vector comprises: Input the image to be identified into the large-core hole convolution neural network of the feature extraction network, perform feature extraction, and obtain a target feature map. The large-core hole convolution neural network includes multiple large-core hole convolution layers. The large-core hole convolution layer is used to perform a large-core hole convolution on the input data to obtain a first feature vector; perform a second large-core hole convolution on the first feature vector to obtain a second feature vector; sum the first feature vector and the second feature vector to obtain a third feature vector; determine the product of the third feature vector and the input data as an output vector; output the output vector to the next layer of the network; the output vector output by the last large-core hole convolution layer is the target feature map; The target feature map is input into the linear layer of the feature extraction network and linearly processed to obtain the one-dimensional feature vector.

4. The fire identification method for a fire truck according to claim 1, characterized in that: The one-dimensional feature vector is concatenated with the value embedding vector, and the steps of completing fire identification according to the concatenated vector include: Acquire a target position vector, where the target position vector is obtained based on position information of a shooting location of the original image; Adding the target position vector to the one-dimensional feature vector to obtain a first intermediate vector; Adding the target position vector to the value embedding vector to obtain a second intermediate vector; splicing the first intermediate vector, the second intermediate vector, and a preset classification mark to obtain a target splicing vector; Input the target concatenated vector into a preset multi-head attention mechanism network, perform global attention mechanism calculation, and obtain a target feature vector; The target feature vector is input into a preset fully connected layer for fire classification and location to obtain a fire classification result and a fire location result.

5. The fire identification method for a fire truck according to claim 4, characterized in that: After the step of concatenating the one-dimensional feature vector and the value embedding vector and completing the fire identification step according to the concatenated vector, the method further includes: Encrypting and packaging the fire classification result and the fire location result to obtain a target data packet; Sending a data upload request to a cloud server, wherein the data upload request is used to instruct the cloud server to perform authority verification on the user information carried in the data upload request, and if the verification is successful, establishing a data transmission channel; The target data packet is sent to the cloud server using the data transmission channel. The target data packet is used to instruct the cloud server to decompress and decrypt the target data packet, store the decrypted data in a preset database, and send a fire notification to the target terminal.

6. The fire identification method for a fire truck according to claim 1, characterized in that: The training steps of the feature extraction network include: Acquire a training set, wherein the training set includes a plurality of sample images; Inputting the sample image into the feature extraction network to perform feature extraction to obtain a plurality of one-dimensional feature vector samples; Constructing a cosine similarity matrix, wherein the cosine similarity matrix includes cosine similarities between two of the plurality of one-dimensional feature vector samples; According to a preset contrastive learning loss function and the cosine similarity, a similarity loss is obtained, wherein the similarity loss corresponds to the cosine similarity in a one-to-one manner; By averaging all the similarity losses, an average loss is obtained; The feature extraction network is trained based on the average loss.

7. The fire identification method for a fire truck according to claim 6, characterized in that: The feature extraction network belongs to a preset fire recognition model, and the fire recognition model also includes: a multi-head attention mechanism network and a fully connected layer; the training set also includes real labels corresponding to the sample images one by one; the training steps of the fire recognition model include: Acquire a target position vector sample, wherein the target position vector sample is obtained based on position information of a shooting location of the sample image; Adding the target position vector sample to the one-dimensional feature vector sample to obtain a first intermediate vector sample; Add a preset value embedding vector to the target position vector sample to obtain a second intermediate vector sample; Splicing the first intermediate vector sample, the second intermediate vector sample, and a preset classification label to obtain a target spliced ​​vector sample; Input the target concatenated vector sample into the multi-head attention mechanism network, perform global attention mechanism calculation, and obtain a target feature vector sample; Inputting the target feature vector sample into the fully connected layer for fire classification and location to obtain fire classification information and fire location information; Obtaining a classification loss based on a preset cross entropy loss function, the fire classification information, and a true label corresponding to the fire classification information; The fire identification model is trained based on the classification loss.

8. A fire identification system for a fire truck, characterized in that: include: An image acquisition module is used to acquire an image to be identified, where the image to be identified is obtained based on an original image taken by a vehicle-mounted camera of a fire truck; A feature extraction module, used for inputting the image to be identified into a preset feature extraction network to perform feature extraction and obtain a one-dimensional feature vector; A dictionary acquisition module is used to acquire a target dictionary, wherein the target dictionary includes a plurality of target key-value pairs, wherein the target key-value pairs include a target key and a target value corresponding to the target key one by one, wherein the target key refers to a fire type; A linear processing module, used for performing linear processing on the target value to obtain a value embedding vector; The fire identification module is used to concatenate the one-dimensional feature vector with the value embedding vector, and complete fire identification according to the concatenated vector.

9. An electronic device, characterized in that: It comprises a processor, a memory and a communication bus; the communication bus is used to connect the processor and the memory; the processor is used to execute the computer program stored in the memory to implement the fire identification method for a fire truck as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is used to enable a computer to execute the fire identification method for a fire truck as claimed in any one of claims 1 to 7.

Images