Hazardous chemical article early warning method and system based on image recognition

Abstract

The invention discloses a dangerous chemical early warning method and system based on image recognition, and the method comprises the steps: collecting image data of chemicals in a preset environment, and carrying out the preprocessing of the image data; performing stable feature extraction on the grouped data to obtain image features, and identifying and screening the image features to obtain chemical features; annotating the image data according to the complete character features to obtain annotated images, and grouping and integrating the ambiguity features according to the similarity to obtain corrected features; and carrying out danger identification on the corrected features to obtain a danger coefficient, constructing a dangerous chemical early warning model according to the annotated image and the danger coefficient, inputting to-be-identified data into the dangerous chemical early warning model, and outputting an identification result. The method not only can improve the accuracy of dangerous chemical early warning based on image recognition, but also has good interpretability, and can be directly applied to a dangerous chemical early warning system.

Description

Technical Field

[0001] The present invention relates to the field of early warning, and in particular to an early warning method and system for hazardous chemicals based on image recognition. Background Art

[0002] With the rapid development of the industrial and chemical industries, the types and quantities of chemicals have increased dramatically, and the potential safety hazards in their storage, transportation and use have become increasingly prominent. The traditional manual inspection method is not only inefficient, but also difficult to accurately identify potential hazardous chemicals and their potential risks in real time. Therefore, it is particularly important to develop a system that can automatically, quickly and accurately identify hazardous chemicals and issue warnings.

[0003] In the existing technology, some image recognition technologies have been applied to the field of chemical identification, but they mostly focus on static image analysis under specific conditions, lack robustness in complex environments, and are not effective when dealing with blurred, damaged or ambiguous chemical labels. In addition, most systems lack the ability to comprehensively assess the hazards of chemicals and cannot directly generate effective warning information. Summary of the invention

[0004] The purpose of the present invention is to provide a chemical hazardous substances early warning method based on image recognition.

[0005] To achieve the above object, the present invention is implemented according to the following technical solutions:

[0006] The present invention comprises the following steps:

[0007] Collecting image data of chemicals in a preset environment and preprocessing the image data;

[0008] Extracting stable features from the image data to obtain image features, and identifying and screening the image features to obtain chemical features; the chemical features include complete text features and ambiguous features;

[0009] Annotating the image data according to the complete text features to obtain an annotated image, and grouping and integrating the ambiguous features according to similarities to obtain corrected features;

[0010] The modified features are subjected to hazard identification to obtain a hazard coefficient, a chemical hazardous substance early warning model is constructed according to the annotated image and the hazard coefficient, the data to be identified is input into the chemical hazardous substance early warning model, and an identification result is output.

[0011] Furthermore, the method of extracting stable features from the image data to obtain image features includes:

[0012] Arrange the image data in chronological order, use local binary patterns to extract the features of the image data, and obtain the features of the object;

[0013] The weights of object features are assigned through the channel attention module of the attention mechanism to obtain the weights of object features. The expression is:

[0014] k c =σ(M1(M0(c ag ))+M1(M0(c max )))

[0015] The weight of the cth thing feature is k c , the activation function is σ(·), the initial perceptual weight is M0, the secondary perceptual weight is M1, and the average pooling image feature is c ag , the maximum pooled image feature is c max ;

[0016] Perform maximum pooling and average pooling on the image features to obtain the spatial weight coefficients, and calculate the channel weight coefficients between the neighborhood channels:

[0017]

[0018] The number of channels is N, and the weight coefficient of the rth channel is w r , the ratio of the number of channels adjusted by the hyperparameter is ψ, the ratio of the convolution kernel adjusted by the hyperparameter is β, the number of adjacent channels is μ, and the odd term is |·| odd , the rth channel f r The μ adjacent channels are The weight coefficient of the xth feature of the rth channel is

[0019] Calculate the offset of the channel weight coefficient:

[0020]

[0021] The control coefficient is The upper limit of observation time is The weight coefficient of the rth channel of the xth object image at time t is w r (t), the weight coefficient of the rth channel of the xth object image at time t+1 is w r (t+1), the natural constant is e, the adjustment factor is ε, and the offset of the channel weight coefficient of the x-th object image at time t and time t+1 is Δw c,x (t,t+1);

[0022] The image features with an offset less than 0.359 are output as image features.

[0023] Furthermore, the method of identifying and screening the image features to obtain chemical features includes:

[0024] The image features are input into deep learning, and the multi-layer perceptron is used to project the image features into the descriptor space, and the initial matching vector is obtained by fusing with the descriptor. The expression is:

[0025] g a =θ a +MLP(v a )

[0026] The ath initial matching vector is g a , the ath descriptor is θ a , the a-th projection feature of the multilayer perceptron is MLP(v a ), the multi-layer perceptron operation is MLP(·), and the a-th image feature is v a ;

[0027] A multi-layer neural network is used for message passing to update the residual message passing. The expression is:

[0028]

[0029] The feature of the ath image in the image l at the y+1th layer is The feature of the ath image of the yth layer is The information transmitted is h C→a , the image feature set is C, the connection operation is (·||·), and the a-th initial matching vector on the c query image is The first bias vector is δ1, the The value vector is The a-th bond vector is β a , the first projection parameter of the yth layer is λ 1,y , transposed to T;

[0030] Perform linear projection on image features, the expression is:

[0031]

[0032] The matching descriptor of the ath image feature of image l is Compute the score matrix:

[0033]

[0034] The inner product function is <·>, and the image B The matching descriptor of image features is The ath image feature and the The score matrix of image features is

[0035] Given the discriminant function, the expression is:

[0036]

[0037] The importance parameter is The resolution coefficient is ζ, the ath image feature and the The average score matrix of image features is The a+1th image feature and the The score matrix of image features is The tuning factor is The ath image feature and the The correlation degree of image features is The ath image feature and the The discriminant function of image features is

[0038] The image features with discriminant function values ​​greater than 0.514 are output as complete text features, and the image features with discriminant function values ​​less than 0.514 are output as ambiguous features.

[0039] Furthermore, the method for obtaining the annotated image automatically extracts complete text features through a recurrent neural network and learns the mapping relationship between the complete text features and image data, and annotates according to the mapping relationship.

[0040] Furthermore, the method of grouping and integrating the ambiguous features according to similarity to obtain corrected features includes:

[0041] The fusion feature is obtained by summing up the ambiguous features from different perspectives. The expression is:

[0042]

[0043] The number of perspectives is n, and the ambiguous feature of the bth perspective is z b , the height of the b-th viewing angle is H b , the length of the b-th perspective is V b , the fusion feature is q o ;

[0044] The feature information is obtained through global average pooling, and the expression is:

[0045]

[0046] The fusion feature q o The characteristic information is The height is D, the width is S, the fusion feature of the i-th row and d-th column is q(i,d), and the number of rows is The number of columns is

[0047] Calculate the similarity of ambiguous features:

[0048]

[0049] The ambiguous feature of the τth perspective is z τ , the maximum function is max{·}, the similarity tolerance parameter is u, the calibration error is κ, and the ambiguous feature z b and the ambiguous feature z τ The similarity is

[0050] The ambiguous data with similarity greater than 0.546 is used as the fusion data set, and the fusion features are updated through weighted operations to obtain the corrected features. The expression is:

[0051]

[0052] The corrected feature is q, and the number of ambiguous features of the fused dataset is The first learning parameter of the jth ambiguous feature is Ambiguous feature z b and the ambiguous feature z j The similarity is The second learning parameter of the jth ambiguous feature is η j , the deviation factor is γ.

[0053] Furthermore, the method of performing risk identification on the correction feature to obtain a risk coefficient includes:

[0054] The target detection algorithm is used to learn the visual features of hazardous chemicals and identify hazardous chemicals based on the comparison with the visual features;

[0055] Calculate the risk factor based on:

[0056]

[0057] The risk factor is The number of visual features is p, and the genetic factor is No. The visual features are No. The correction feature is The linear coefficient is e ξ .

[0058] Furthermore, the method of constructing a chemical hazardous goods warning model according to the annotated image and the hazard coefficient includes:

[0059] Compare the labeled image with hazardous chemicals to obtain a comparison value. If the comparison value is 1, it means it is a hazardous substance, otherwise it is 0.

[0060] The objective function is constructed based on the comparison value and the risk coefficient, and the expression is:

[0061]

[0062] The objective function of image l is The loss function is The number of ambiguous features is P, and the comparison value is The optimization constant is α, and the matching degree between the ath ambiguous feature and the a+1th ambiguous feature is The a+1th ambiguous feature is s a+1 , the ath ambiguous feature is s a ;

[0063] When the comparison value is 1, a dangerous goods warning is directly issued; when the comparison value is 0 and the objective function value is greater than 0.731, a dangerous goods warning is issued;

[0064] The chemical hazardous materials warning model includes the holdout method, adaptive threshold segmentation algorithm, texture analysis algorithm, dual attention network, and attention perception network;

[0065] The holdout method divides the input data into two mutually exclusive sets to obtain the training set and the test set;

[0066] The adaptive threshold segmentation algorithm dynamically calculates the threshold according to the grayscale characteristics of the local area of ​​the training set, and takes the pixels in the training set that are greater than the threshold as the foreground area;

[0067] The texture analysis algorithm obtains foreground features by analyzing the spatial arrangement pattern and local variation characteristics of pixels in the foreground area and extracting quantitative descriptors that characterize the image texture structure;

[0068] The dual attention network strengthens the important features in the foreground features through the channel attention and spatial attention mechanisms to obtain enhanced features, and performs weighted fusion of the enhanced features to obtain fused features;

[0069] The attention perception network learns the feature representation of chemical hazardous materials images, uses the attention mechanism to focus on the fusion features, matches the feature representation and the fusion features, calculates the target function value, and issues warnings based on the target function value.

[0070] The test set is input into the chemical hazardous goods warning model to obtain the predicted warning results, and the learning rate of the chemical hazardous goods warning model is adjusted according to the variance between the predicted warning results and the actual warning results.

[0071] In the second aspect, a chemical hazardous goods early warning system based on image recognition includes:

[0072] Data acquisition module: used to collect image data of chemicals in a preset environment and pre-process the image data;

[0073] Feature processing module: used for extracting stable features from the image data to obtain image features, and identifying and screening the image features to obtain chemical features; the chemical features include complete text features and ambiguous features;

[0074] Annotation integration module: used for annotating the image data according to the complete text features to obtain annotated images, and grouping and integrating the ambiguous features according to similarity to obtain corrected features;

[0075] Danger warning module: used to perform hazard identification on the modified features to obtain a hazard coefficient, construct a chemical hazardous substance warning model based on the annotated image and the hazard coefficient, input the data to be identified into the chemical hazardous substance warning model, and output the identification result.

[0076] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0077] A processor; and a memory arranged to store computer executable instructions, which when executed cause the processor to perform the method steps described in the first aspect.

[0078] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple applications, the electronic device executes the method steps described in the first aspect.

[0079] The beneficial effects of the present invention are:

[0080] The present invention is a chemical hazardous goods early warning method and system based on image recognition. Compared with the prior art, the present invention has the following technical effects:

[0081] The present invention can improve the accuracy of chemical hazardous substance warning based on image recognition through preprocessing, stable feature extraction, identification and screening, image annotation, grouping integration, hazard identification and model construction steps, thereby improving the precision of chemical hazardous substance warning based on image recognition, optimizing chemical hazardous substance warning based on image recognition, greatly saving resources, and improving work efficiency, and can realize automatic early warning of chemical hazardous substances, and can identify, screen and group integrate chemical hazardous substance warning based on image recognition in real time, which is of great significance to grouping integration, and can adapt to image recognition chemical hazardous substance warnings of different standards and different image recognition chemical hazardous substance warning requirements, and has a certain universality. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1 A flowchart of a method for early warning of hazardous chemicals based on image recognition according to the present invention;

[0083] Figure 2 It is a schematic diagram of the structure of an electronic device in an embodiment of this specification. DETAILED DESCRIPTION

[0084] The present invention is further described below by means of specific embodiments. The illustrative embodiments and descriptions of the present invention are used to explain the present invention but are not intended to limit the present invention.

[0085] The present invention provides a chemical hazardous substance early warning method and system based on image recognition, comprising the following steps:

[0086] like Figure 1 As shown, in this embodiment, the following steps are included:

[0087] Collecting image data of chemicals in a preset environment and preprocessing the image data;

[0088] In the actual evaluation, the XX Chemical Laboratory was taken as the research object, and two sets of image data of the XX Chemical Laboratory were obtained;

[0089] Extracting stable features from the image data to obtain image features, and identifying and screening the image features to obtain chemical features; the chemical features include complete text features and ambiguous features;

[0090] In the actual evaluation, the image features of the first set of image data are transparent light yellow liquid, flammable liquid sign, flammable liquid text, well-ventilated area, and placed in a transparent glass bottle; the image features of the second set of image data are high-pressure gas cylinders, poisonous gas signs, white background, black skull and crossbones, poisonous gas text, stay away from fire sources, and wear a protective mask;

[0091] The first group of complete text features is the flammable liquid text, and the first group of ambiguous features is the flammable liquid sign and the flammable liquid text;

[0092] The second group of complete text features are poisonous gas text, and the second group of ambiguous features are poisonous gas symbol, white background, black skull and crossbones, poisonous gas text, stay away from fire, and wear protective mask;

[0093] Annotating the image data according to the complete text features to obtain an annotated image, and grouping and integrating the ambiguous features according to similarities to obtain corrected features;

[0094] In the actual assessment, the first set of revised features is the complete flammable liquid sign and flammable liquid text; the second set of revised features is the complete poisonous gas sign, white background, black skull and crossbones, poisonous gas text, stay away from fire, and wear a protective mask;

[0095] Performing hazard identification on the modified features to obtain a hazard coefficient, constructing a chemical hazardous substance warning model according to the annotated image and the hazard coefficient, inputting the data to be identified into the chemical hazardous substance warning model, and outputting an identification result;

[0096] In the actual evaluation, the hazard coefficient of the first group was 0.531, the comparison value was 0 and the objective function value was 0.6011, so no dangerous goods warning was issued; the comparison value of the second group was 1, and a dangerous goods warning was issued directly.

[0097] In this embodiment, the method of extracting stable features from the image data to obtain image features includes:

[0098] Arrange the image data in chronological order, use local binary patterns to extract the features of the image data, and obtain the features of the object;

[0099] The weights of object features are assigned through the channel attention module of the attention mechanism to obtain the weights of object features. The expression is:

[0100] k c =σ(M1(M0(c ag ))+M1(M0(c max )))

[0101] The weight of the cth thing feature is k c , the activation function is σ(·), the initial perceptual weight is M0, the secondary perceptual weight is M1, and the average pooling image feature is c ag , the maximum pooled image feature is c max ;

[0102] Perform maximum pooling and average pooling on the image features to obtain the spatial weight coefficients, and calculate the channel weight coefficients between the neighborhood channels:

[0103]

[0104] The number of channels is N, and the weight coefficient of the rth channel is w r , the ratio of the number of channels adjusted by the hyperparameter is ψ, the ratio of the convolution kernel adjusted by the hyperparameter is β, the number of adjacent channels is μ, and the odd term is |·| odd , the rth channel f r The μ adjacent channels are The weight coefficient of the xth feature of the rth channel is

[0105] Calculate the offset of the channel weight coefficient:

[0106]

[0107] The control coefficient is The upper limit of observation time is The weight coefficient of the rth channel of the xth object image at time t is w r (t), the weight coefficient of the rth channel of the xth object image at time t+1 is w r (t+1), the natural constant is e, the adjustment factor is ε, and the offset of the channel weight coefficient of the x-th object image at time t and time t+1 is Δw c,x (t,t+1);

[0108] The image features with an offset less than 0.359 are output as image features.

[0109] In this embodiment, the method of identifying and screening the image features to obtain chemical features includes:

[0110] The image features are input into deep learning, and the multi-layer perceptron is used to project the image features into the descriptor space, and the initial matching vector is obtained by fusing with the descriptor. The expression is:

[0111] g a =θ a +MLP(v a )

[0112] The ath initial matching vector is g a , the ath descriptor is θ a , the a-th projection feature of the multilayer perceptron is MLP(v a ), the multi-layer perceptron operation is MLP(·), and the a-th image feature is v a ;

[0113] A multi-layer neural network is used for message passing to update the residual message passing. The expression is:

[0114]

[0115] The feature of the ath image in the image l at the y+1th layer is The feature of the ath image of the yth layer is The information transmitted is h C→a , the image feature set is C, the connection operation is (·||·), and the a-th initial matching vector on the c query image is The first bias vector is δ1, the The value vector is The a-th bond vector is β a , the first projection parameter of the yth layer is λ 1,y , transposed to T;

[0116] Perform linear projection on image features, the expression is:

[0117]

[0118] The matching descriptor of the ath image feature of image l is Compute the score matrix:

[0119]

[0120] The inner product function is <·>, and the image B The matching descriptor of image features is The ath image feature and the The score matrix of image features is

[0121] Given the discriminant function, the expression is:

[0122]

[0123] The importance parameter is The resolution coefficient is ζ, the ath image feature and the The average score matrix of image features is The a+1th image feature and the The score matrix of image features is The tuning factor is The ath image feature and the The correlation degree of image features is The ath image feature and the The discriminant function of image features is

[0124] The image features with discriminant function values ​​greater than 0.514 are output as complete text features, and the image features with discriminant function values ​​less than 0.514 are output as ambiguous features.

[0125] In this embodiment, the method for obtaining the annotated image automatically extracts complete text features through a recurrent neural network and learns the mapping relationship between the complete text features and image data, and annotates according to the mapping relationship.

[0126] In this embodiment, the method for obtaining a corrected feature by grouping and integrating the ambiguous features according to similarity includes:

[0127] The fusion feature is obtained by summing up the ambiguous features from different perspectives. The expression is:

[0128]

[0129] The number of perspectives is n, and the ambiguous feature of the bth perspective is z b , the height of the b-th viewing angle is H b , the length of the b-th perspective is V b , the fusion feature is q o ;

[0130] The feature information is obtained through global average pooling, and the expression is:

[0131]

[0132] The fusion feature q o The characteristic information is The height is D, the width is S, the fusion feature of the i-th row and d-th column is q(i,d), and the number of rows is The number of columns is

[0133] Calculate the similarity of ambiguous features:

[0134]

[0135] The ambiguous feature of the τth perspective is z τ , the maximum function is max{·}, the similarity tolerance parameter is u, the calibration error is κ, and the ambiguous feature z b and the ambiguous feature z τ The similarity is

[0136] The ambiguous data with similarity greater than 0.546 is used as the fusion data set, and the fusion features are updated through weighted operations to obtain the corrected features. The expression is:

[0137]

[0138] The corrected feature is q, and the number of ambiguous features of the fused dataset is The first learning parameter of the jth ambiguous feature is Ambiguous feature z b and the ambiguous feature z j The similarity is The second learning parameter of the jth ambiguous feature is η j , the deviation factor is γ.

[0139] In this embodiment, the method of performing risk identification on the correction feature to obtain the risk coefficient includes:

[0140] The target detection algorithm is used to learn the visual features of hazardous chemicals and identify hazardous chemicals based on the comparison with the visual features;

[0141] Calculate the risk factor based on:

[0142]

[0143] The risk factor is The number of visual features is p, and the genetic factor is No. The visual features are No. The correction feature is The linear coefficient is e ξ .

[0144] In this embodiment, the method for constructing a chemical hazardous article warning model according to the annotated image and the hazard coefficient includes:

[0145] Compare the labeled image with hazardous chemicals to obtain a comparison value. If the comparison value is 1, it means it is a hazardous substance, otherwise it is 0.

[0146] The objective function is constructed based on the comparison value and the risk coefficient, and the expression is:

[0147]

[0148] The objective function of image l is The loss function is The number of ambiguous features is P, and the comparison value is The optimization constant is α, and the matching degree between the ath ambiguous feature and the a+1th ambiguous feature is The a+1th ambiguous feature is s a+1 , the ath ambiguous feature is s a ;

[0149] When the comparison value is 1, a dangerous goods warning is directly issued; when the comparison value is 0 and the objective function value is greater than 0.731, a dangerous goods warning is issued;

[0150] The chemical hazardous materials warning model includes the holdout method, adaptive threshold segmentation algorithm, texture analysis algorithm, dual attention network, and attention perception network;

[0151] The holdout method divides the input data into two mutually exclusive sets to obtain the training set and the test set;

[0152] The adaptive threshold segmentation algorithm dynamically calculates the threshold according to the grayscale characteristics of the local area of ​​the training set, and takes the pixels in the training set that are greater than the threshold as the foreground area;

[0153] The texture analysis algorithm obtains foreground features by analyzing the spatial arrangement pattern and local variation characteristics of pixels in the foreground area and extracting quantitative descriptors that characterize the image texture structure;

[0154] The dual attention network strengthens the important features in the foreground features through the channel attention and spatial attention mechanisms to obtain enhanced features, and performs weighted fusion of the enhanced features to obtain fused features;

[0155] The attention perception network learns the feature representation of chemical hazardous materials images, uses the attention mechanism to focus on the fusion features, matches the feature representation and the fusion features, calculates the target function value, and issues warnings based on the target function value.

[0156] The test set is input into the chemical hazardous goods warning model to obtain the predicted warning results, and the learning rate of the chemical hazardous goods warning model is adjusted according to the variance between the predicted warning results and the actual warning results.

[0157] In the second aspect, a chemical hazardous goods early warning system based on image recognition includes:

[0158] Data acquisition module: used to collect image data of chemicals in a preset environment and pre-process the image data;

[0159] Feature processing module: used for extracting stable features from the image data to obtain image features, and identifying and screening the image features to obtain chemical features; the chemical features include complete text features and ambiguous features;

[0160] Annotation integration module: used for annotating the image data according to the complete text features to obtain annotated images, and grouping and integrating the ambiguous features according to similarity to obtain corrected features;

[0161] Danger warning module: used to perform hazard identification on the modified features to obtain a hazard coefficient, construct a chemical hazardous substance warning model based on the annotated image and the hazard coefficient, input the data to be identified into the chemical hazardous substance warning model, and output the identification result.

[0162] Figure 2 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 2 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0163] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 2 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0164] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0165] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a chemical hazardous early warning device based on image recognition at the logical level. The processor executes the program stored in the memory and is specifically used to execute any of the above-mentioned chemical hazardous early warning methods based on image recognition.

[0166] The above application Figure 1The chemical hazardous goods warning method based on image recognition disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0167] The electronic device may also perform Figure 1 A chemical hazardous materials early warning method based on image recognition is implemented Figure 1 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.

[0168] An embodiment of the present application also proposes a computer-readable storage medium, which stores one or more programs, and the one or more programs include instructions. When the instructions are executed by an electronic device including multiple application programs, any one of the aforementioned chemical hazardous goods warning methods based on image recognition is executed.

[0169] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0170] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0171] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0172] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0173] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0174] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0175] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0176] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0177] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0178] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A chemical hazardous goods early warning method based on image recognition, characterized in that: The following steps are involved: Collecting image data of chemicals in a preset environment and preprocessing the image data; Extracting stable features from the image data to obtain image features, and identifying and screening the image features to obtain chemical features; the chemical features include complete text features and ambiguous features; Annotating the image data according to the complete text features to obtain an annotated image, and grouping and integrating the ambiguous features according to similarities to obtain corrected features; The modified features are subjected to hazard identification to obtain a hazard coefficient, a chemical hazardous substance early warning model is constructed according to the annotated image and the hazard coefficient, the data to be identified is input into the chemical hazardous substance early warning model, and an identification result is output.

2. According to claim 1, a chemical hazardous substance early warning method based on image recognition is characterized in that: The method for extracting stable features from the image data to obtain image features comprises: Arrange the image data in chronological order, use local binary patterns to extract the features of the image data, and obtain the features of the object; The weights of object features are assigned through the channel attention module of the attention mechanism to obtain the weights of object features. The expression is: k c =σ(M1(M0(c ag ))+M1(M0(c max ))) The weight of the cth thing feature is k c , the activation function is σ(·), the initial perceptual weight is M0, the secondary perceptual weight is M1, and the average pooling image feature is c ag , the maximum pooled image feature is c max ; Perform maximum pooling and average pooling on the image features to obtain the spatial weight coefficients, and calculate the channel weight coefficients between the neighborhood channels: The number of channels is N, and the weight coefficient of the rth channel is w r , the ratio of the number of channels adjusted by the hyperparameter is ψ, the ratio of the convolution kernel adjusted by the hyperparameter is β, the number of adjacent channels is μ, and the odd term is |·| odd , the rth channel f r The μ adjacent channels are The weight coefficient of the xth feature of the rth channel is Calculate the offset of the channel weight coefficient: The control coefficient is The upper limit of observation time is The weight coefficient of the rth channel of the xth object image at time t is w r (t), the weight coefficient of the rth channel of the xth object image at time t+1 is w r (t+1), the natural constant is e, the adjustment factor is ε, and the offset of the channel weight coefficient of the x-th object image at time t and time t+1 is Δw c,x (t, t+1); The image features with an offset less than 0.359 are output as image features.

3. According to claim 1, a chemical hazardous substance early warning method based on image recognition is characterized in that: The method for identifying and screening the image features to obtain chemical features includes: The image features are input into deep learning, and the multi-layer perceptron is used to project the image features into the descriptor space, and the initial matching vector is obtained by fusing with the descriptor. The expression is: g a =θ a +MLP(v a ) The ath initial matching vector is g a , the ath descriptor is θ a , the a-th projection feature of the multilayer perceptron is MLP(v a ), the multi-layer perceptron operation is MLP(·), and the a-th image feature is v a ; A multi-layer neural network is used for message passing to update the residual message passing. The expression is: The feature of the ath image in the image l at the y+1th layer is The feature of the ath image of the yth layer is The information transmitted is h C→a , the image feature set is C, the connection operation is (·||·), and the a-th initial matching vector on the c query image is The first bias vector is δ1, the The value vector is The ath bond vector is β a , the first projection parameter of the yth layer is λ 1，y , transposed to T; Perform linear projection on image features, the expression is: The matching descriptor of the ath image feature of image l is Compute the score matrix: The inner product function is (·>, image B The matching descriptor of image features is The ath image feature and the The score matrix of image features is Given the discriminant function, the expression is: The importance parameter is The resolution coefficient is ζ, the ath image feature and the The average score matrix of image features is The a+1th image feature and the The score matrix of image features is The tuning factor is The ath image feature and the The correlation degree of image features is The ath image feature and the The discriminant function of image features is The image features with discriminant function values ​​greater than 0.514 are output as complete text features, and the image features with discriminant function values ​​less than 0.514 are output as ambiguous features.

4. According to claim 1, a chemical hazardous substance early warning method based on image recognition is characterized in that: The method for obtaining the annotated image automatically extracts complete text features through a recurrent neural network and learns the mapping relationship between the complete text features and image data, and annotates according to the mapping relationship.

5. According to claim 1, a chemical hazardous substance early warning method based on image recognition is characterized in that: The method for obtaining a corrected feature by grouping and integrating the ambiguous features according to similarity includes: The fusion feature is obtained by summing up the ambiguous features from different perspectives. The expression is: The number of perspectives is n, and the ambiguous feature of the bth perspective is z b , the height of the b-th viewing angle is Hb, and the length of the b-th viewing angle is V b , the fusion feature is q o ; The feature information is obtained through global average pooling, and the expression is: The fusion feature q o The characteristic information is The height is D, the width is S, the fusion feature of the i-th row and d-th column is q(i, d), and the number of rows is The number of columns is Calculate the similarity of ambiguous features: The ambiguous feature of the τth perspective is z τ , the maximum function is max{·}, the similarity tolerance parameter is u, the calibration error is κ, and the ambiguous feature z b and the ambiguous feature z τ The similarity is The ambiguous data with similarity greater than 0.546 is used as the fusion data set, and the fusion features are updated through weighted operations to obtain the corrected features. The expression is: The corrected feature is q, and the number of ambiguous features of the fused dataset is The first learning parameter of the jth ambiguous feature is θ j , ambiguous feature z b and the ambiguous feature z j The similarity is d(b, j), and the second learning parameter of the jth ambiguous feature is η j , the deviation factor is γ.

6. According to claim 1, a chemical hazardous substance early warning method based on image recognition is characterized in that: The method for performing risk identification on the correction feature to obtain a risk coefficient includes: The target detection algorithm is used to learn the visual features of hazardous chemicals and identify hazardous chemicals based on the comparison with the visual features; Calculate the risk factor based on: The risk factor is The number of visual features is p, and the genetic factor is No. The visual features are The cth modified feature is q c , the linear coefficient is e ξ .

7. According to claim 1, a chemical hazardous substance early warning method based on image recognition is characterized in that: The method for constructing a chemical hazardous article warning model according to the annotated image and the hazard coefficient comprises: Compare the labeled image with hazardous chemicals to obtain a comparison value. If the comparison value is 1, it means it is a hazardous substance, otherwise it is 0. The objective function is constructed based on the comparison value and the risk coefficient, and the expression is: The objective function of image l is The loss function is The number of ambiguous features is P, and the comparison value is The optimization constant is α, and the matching degree between the ath ambiguous feature and the a+1th ambiguous feature is The a+1th ambiguous feature is s a+1 , the ath ambiguous feature is s a ; When the comparison value is 1, a dangerous goods warning is directly issued; when the comparison value is 0 and the objective function value is greater than 0.731, a dangerous goods warning is issued; The chemical hazardous materials warning model includes the holdout method, adaptive threshold segmentation algorithm, texture analysis algorithm, dual attention network, and attention perception network; The holdout method divides the input data into two mutually exclusive sets to obtain the training set and the test set; The adaptive threshold segmentation algorithm dynamically calculates the threshold according to the grayscale characteristics of the local area of ​​the training set, and takes the pixels in the training set that are greater than the threshold as the foreground area; The texture analysis algorithm obtains foreground features by analyzing the spatial arrangement pattern and local variation characteristics of pixels in the foreground area and extracting quantitative descriptors that characterize the image texture structure; The dual attention network strengthens the important features in the foreground features through the channel attention and spatial attention mechanisms to obtain enhanced features, and performs weighted fusion of the enhanced features to obtain fused features; The attention perception network learns the feature representation of chemical hazardous materials images, uses the attention mechanism to focus on the fusion features, matches the feature representation and the fusion features, calculates the target function value, and issues warnings based on the target function value. The test set is input into the chemical hazardous goods warning model to obtain the predicted warning results, and the learning rate of the chemical hazardous goods warning model is adjusted according to the variance between the predicted warning results and the actual warning results.

8. A chemical hazardous substance early warning system based on image recognition, characterized in that: include: Data acquisition module: used to collect image data of chemicals in a preset environment and pre-process the image data; Feature processing module: used for extracting stable features from the image data to obtain image features, and identifying and screening the image features to obtain chemical features; the chemical features include complete text features and ambiguous features; Annotation integration module: used for annotating the image data according to the complete text features to obtain annotated images, and grouping and integrating the ambiguous features according to similarity to obtain corrected features; Danger warning module: used to perform hazard identification on the modified features to obtain a hazard coefficient, construct a chemical hazardous substance warning model based on the annotated image and the hazard coefficient, input the data to be identified into the chemical hazardous substance warning model, and output the identification result.

9. An electronic device, comprising: processor; as well as A memory arranged to store computer executable instructions, which when executed cause the processor to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, enables the electronic device to execute the method according to any one of claims 1 to 7.

Images