Water mist identification method, device, equipment and medium

Abstract

The invention provides a water mist recognition method and device, equipment and a medium, and the method comprises the steps: obtaining a to-be-recognized image which is an image of a to-be-recognized region; acquiring target information, wherein the target information comprises at least one of the following items: position information of the to-be-identified area, radar acquisition data of the to-be-identified area, meteorological information of the to-be-identified area and acquisition time information of the target information; and inputting the to-be-recognized image and the target information into a pre-trained water mist recognition model to obtain a water mist recognition result. According to the multi-information fusion end-to-end water mist recognition scheme, water mist recognition is carried out based on the multi-modal information source (the to-be-recognized image and the target information), and compared with single-modal information recognition, water mist recognition precision can be improved.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and in particular, relates to a water mist recognition method, device, equipment and medium. Background Art

[0002] In the field of unmanned locomotive transportation, the key to realizing unmanned operation is the comprehensive perception of the environment along the route during the transportation operation. Water mist will affect the perception system of the locomotive. Accurately identifying water mist through water mist recognition technology is conducive to filtering water mist interference, thereby achieving safe driving of vehicles in complex environments. At present, most environmental perception solutions for water mist recognition use single-modal information (such as image information). The disadvantages of single-modal information environmental perception include incomplete information, limited adaptability, susceptibility to interference, lack of redundancy and fault tolerance, etc. Summary of the invention

[0003] In view of the above-mentioned shortcomings of the prior art, the purpose of the present application is to provide a water mist identification method, device, equipment and medium to solve the above-mentioned problems.

[0004] The water mist identification method provided in this application includes:

[0005] Acquire an image to be identified, where the image to be identified is an image of the area to be identified;

[0006] Acquire target information, the target information including at least one of the following: location information of the area to be identified, radar collection data of the area to be identified, meteorological information of the area to be identified, and acquisition time information of the target information;

[0007] The image to be identified and the target information are input into a pre-trained water mist recognition model to obtain a water mist recognition result.

[0008] In one embodiment of the present application, the water mist recognition model includes a first feature extraction network for extracting image features, a target feature extraction network for extracting target features, a feature fusion network, and a water mist recognition result output network, the output data of the first feature extraction network and the target feature extraction network are transmitted to the feature fusion network, and the output data of the feature fusion network is transmitted to the water mist recognition result output network;

[0009] The step of inputting the image to be identified and the target information into a pre-trained water mist identification model to obtain a water mist identification result includes:

[0010] The image to be identified is input into the first feature extraction network, and the target information is input into the target feature extraction network to obtain the water mist identification result output by the water mist identification result output network.

[0011] In one embodiment of the present application, the first feature extraction network adopts a ResNet network structure, wherein the first feature extraction network includes a maximum pooling layer and multiple bottleneck layers, the maximum pooling layer is used to output a feature map with reduced spatial size, and pass the feature map with reduced spatial size into the multiple bottleneck layers.

[0012] In one embodiment of the present application, the target information includes radar acquisition data of the area to be identified, the target feature extraction network includes a second feature extraction network, and inputting the target information into the target feature extraction network includes:

[0013] The radar collected data is input into the second feature extraction network, and the second feature extraction network is used to extract features of the radar collected data.

[0014] In one embodiment of the present application, the target information includes location information of the to-be-identified area, the target feature extraction network includes a third feature extraction network, and inputting the target information into the target feature extraction network includes:

[0015] The position information of the area to be identified is input into the third feature extraction network, and the third feature extraction network is used to extract features of the position information.

[0016] In one embodiment of the present application, the target information includes meteorological information of the area to be identified and acquisition time information of the target information, the target feature extraction network includes a fourth feature extraction network, and inputting the target information into the target feature extraction network includes:

[0017] The meteorological information and the acquisition time information are input into the fourth feature extraction network, and the fourth feature extraction network is used to extract features of the meteorological information and the time information.

[0018] In one embodiment of the present application, the area to be identified is an area along the line where molten iron is transported.

[0019] The water mist identification device provided in this application includes:

[0020] A first acquisition module is used to acquire an image to be identified, where the image to be identified is an image of a region to be identified;

[0021] A second acquisition module is used to acquire target information, wherein the target information includes at least one of the following: location information of the area to be identified, radar acquisition data of the area to be identified, meteorological information of the area to be identified, and acquisition time information of the target information;

[0022] The determination module is used to input the image to be identified and the target information into a pre-trained water mist recognition model to obtain a water mist recognition result.

[0023] In one embodiment of the present application, the water mist recognition model includes a first feature extraction network for extracting image features, a target feature extraction network for extracting target features, a feature fusion network, and a water mist recognition result output network, the output data of the first feature extraction network and the target feature extraction network are transmitted to the feature fusion network, and the output data of the feature fusion network is transmitted to the water mist recognition result output network;

[0024] The determination module is specifically used for:

[0025] The image to be identified is input into the first feature extraction network, and the target information is input into the target feature extraction network to obtain the water mist identification result output by the water mist identification result output network.

[0026] In one embodiment of the present application, the first feature extraction network adopts a ResNet network structure, wherein the first feature extraction network includes a maximum pooling layer and multiple bottleneck layers, the maximum pooling layer is used to output a feature map with reduced spatial size, and pass the feature map with reduced spatial size into the multiple bottleneck layers.

[0027] In one embodiment of the present application, the target information includes radar acquisition data of the area to be identified, the target feature extraction network includes a second feature extraction network, and the determination module is further configured to:

[0028] The radar collected data is input into the second feature extraction network, and the second feature extraction network is used to extract features of the radar collected data.

[0029] In one embodiment of the present application, the target information includes location information of the area to be identified, the target feature extraction network includes a third feature extraction network, and the determination module is further configured to:

[0030] The position information of the area to be identified is input into the third feature extraction network, and the third feature extraction network is used to extract features of the position information.

[0031] In one embodiment of the present application, the target information includes meteorological information of the area to be identified and acquisition time information of the target information, the target feature extraction network includes a fourth feature extraction network, and the determination module is further specifically used for:

[0032] The meteorological information and the acquisition time information are input into the fourth feature extraction network, and the fourth feature extraction network is used to extract features of the meteorological information and the time information.

[0033] In one embodiment of the present application, the area to be identified is an area along the line where molten iron is transported.

[0034] The electronic device provided by the present application includes:

[0035] one or more processors;

[0036] A storage device is used to store one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device implements the water mist recognition method.

[0037] The computer-readable storage medium provided in the present application stores a computer program thereon, and when the computer program is executed by a processor of a computer, the computer is enabled to execute the water mist identification method.

[0038] Beneficial effects of the present technical solution: The present technical solution provides an end-to-end water mist recognition solution with multi-information fusion, which performs water mist recognition based on multi-modal information sources (image to be recognized and target information), which is beneficial to improving the accuracy of water mist recognition compared with single-modal information recognition.

[0039] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0041] Figure 1 is one of the flow charts of a water mist identification method shown in an exemplary embodiment of the present application;

[0042] Figure 2 is a structural diagram of a ResNet network shown in an exemplary embodiment of the present application;

[0043] Figure 3 is a structural diagram of a PointNet feature extraction network shown in an exemplary embodiment of the present application;

[0044] Figure 4is a structural diagram of an LSTM network shown in an exemplary embodiment of the present application;

[0045] Figure 5 is a structural diagram of a three-layer neural network shown in an exemplary embodiment of the present application;

[0046] Figure 6 is a structural diagram of a Transformer architecture shown in an exemplary embodiment of the present application;

[0047] Figure 7 is a block diagram of a water mist identification device shown in an exemplary embodiment of the present application;

[0048] Figure 8 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0049] The following will describe the implementation methods of the present application with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application, not for limiting the scope of protection of the present application.

[0050] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application, and thus the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.

[0051] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.

[0052] See also Figure 1 , Figure 1 FIG. 1 is a flow chart of a water mist identification method shown in an exemplary embodiment of the present application. Figure 1 As shown, in an exemplary embodiment, the water mist identification method includes steps 110 to S130, which are described in detail as follows:

[0053] Step S110, obtaining an image to be identified, wherein the image to be identified is an image of a region to be identified;

[0054] Step S120, acquiring target information, wherein the target information includes at least one of the following: location information of the area to be identified, radar acquisition data of the area to be identified, meteorological information of the area to be identified, and acquisition time information of the target information;

[0055] Step S130, inputting the image to be recognized and the target information into a pre-trained water mist recognition model to obtain a water mist recognition result.

[0056] The image of the area to be identified is collected by a camera. As an example, the camera can be a neural network encoding camera.

[0057] The location information of the above-mentioned area to be identified can be obtained by relevant positioning systems, such as the Global Navigation Satellite System (GNSS), the Inertial Measurement Unit (IMU) and the Simultaneous Localization and Mapping (SLAM) system.

[0058] The radar data collected in the above-mentioned area to be identified can be collected by laser radar, 4D millimeter wave radar and ultrasonic radar.

[0059] The meteorological information of the area to be identified may include weather temperature, whether it is raining, etc. Water mist may have a higher probability of appearing in specific weather or at specific times. Based on this, the meteorological information of the area to be identified and the acquisition time information of the target information have a certain influence on water mist recognition. Among them, the acquisition time information of the target information can be accurate to season or hour, etc.

[0060] The water mist recognition model is a trained model, and each sample in its training samples includes an image to be recognized and other training data. The data types included in the other training data are consistent with the data types included in the target information. For example, when the training data includes location information, radar acquisition data, meteorological information, and time information, the target information includes the location information of the area to be recognized, the radar acquisition data of the area to be recognized, the meteorological information of the area to be recognized, and the acquisition time information of the target information. Each sample in the training samples also includes a corresponding label, which is used to indicate whether water mist exists.

[0061] In an embodiment of the present application, multimodal information, i.e., the image to be identified and the target information, is obtained, and the multimodal information is input into a pre-trained water mist recognition model to implement an end-to-end water mist recognition solution with highly robust multimodal information fusion. Compared with the single modal information recognition in the related art, this is beneficial to improving the accuracy of water mist recognition.

[0062] In one embodiment of the present application, the water mist recognition model includes a first feature extraction network for extracting image features, a target feature extraction network for extracting target features, a feature fusion network, and a water mist recognition result output network, the output data of the first feature extraction network and the target feature extraction network are transmitted to the feature fusion network, and the output data of the feature fusion network is transmitted to the water mist recognition result output network;

[0063] The step of inputting the image to be identified and the target information into a pre-trained water mist identification model to obtain a water mist identification result includes:

[0064] The image to be identified is input into the first feature extraction network, and the target information is input into the target feature extraction network to obtain the water mist identification result output by the water mist identification result output network.

[0065] In this embodiment, features of different input parameters are extracted based on different feature extraction networks. Specifically, the first feature extraction network of the water mist recognition model is used to extract image features, and the image to be recognized is input into the first feature extraction network; the target feature extraction network is used to extract target features, and the above target information is input into the target feature extraction network. The features extracted by the first feature extraction network and the target feature extraction network are both input into the feature fusion network, and the feature extraction network is used to fuse the input features, and the fused features are input into the water mist recognition result output network, which outputs the water mist recognition result.

[0066] The loss function of the water mist recognition result output network can be set as required, for example, as a cross entropy loss. The cross entropy loss function is based on the concept of cross entropy in information theory, which measures the difference between the predicted probability distribution and the true label probability distribution. The embodiment of the present application minimizes the loss through back propagation, so that the water mist recognition performance is jointly globally optimized in each network of the model.

[0067] In the related art, in the field of multi-sensor fusion, there are solutions through pre-fusion and post-fusion. Specifically, pre-fusion is to process data of multiple scales after splicing them together. However, the pre-fusion solution is difficult to implement. If there is a problem with a part of the data, it may affect the performance of the entire system. Post-fusion means that each data is processed separately first, generating their own perception results or decisions, and then fusing these results. The post-fusion solution has many shortcomings, including insufficient information utilization, complex system links, and high maintenance costs after rule stacking. In the embodiment of the present application, multimodal information is first introduced in the field of water mist recognition, and data features of information of different scales are extracted based on different feature extraction networks. Then, water mist recognition is performed after the multi-scale features are fused, making full use of multimodal information, improving the accuracy of water mist recognition and improving robustness.

[0068] In one embodiment of the present application, the first feature extraction network adopts a ResNet network structure, wherein the first feature extraction network includes a maximum pooling layer and multiple bottleneck layers, the maximum pooling layer is used to output a feature map with reduced spatial size, and pass the feature map with reduced spatial size into the multiple Bottleneck layers.

[0069] The ResNet network structure used by the first feature extraction network is as follows Figure 2 As shown, see Figure 2 , the first feature extraction network includes: an input layer (Input), the above-mentioned image to be recognized first enters the input layer; a convolution layer, using a 7x7 convolution kernel (Conv7x7), the number of input channels is 3 and the number of output channels is 64, the stride of the convolution kernel movement is 2 (stride=2), and 3 pixels are padded at the edge of the input data (padding=3); a batch normalization layer (Batch Normalization, BN), which is used to accelerate the training process and improve the stability of the model; an activation function layer ReLU (Rectified Linear Unit), which introduces nonlinearity so that the model can learn more complex features; a maximum pooling layer (MaxPool2d, 3x3, stride=2, padding=1), using a 3x3 pooling kernel and a stride of 2, padding=1 means padding 1 pixel at the edge of the input data to reduce the spatial size of the feature map and extract important features; multiple BottleNeck layers (4 in the figure, which can be set according to needs) are used to gradually extract deeper features.

[0070] In an embodiment of the present application, after the feature map passes through the maximum pooling layer, the maximum pooling layer reduces the spatial size of the feature map and extracts important features, and then inputs multiple BottleNeck layers, which helps to reduce the spatial size of the feature map and can provide richer feature representation for subsequent BottleNeck modules.

[0071] In one embodiment of the present application, the target information includes radar acquisition data of the area to be identified, the target feature extraction network includes a second feature extraction network, and inputting the target information into the target feature extraction network includes:

[0072] The radar collected data is input into the second feature extraction network, and the second feature extraction network is used to extract features of the radar collected data.

[0073] In this implementation, the second feature extraction network is used to extract features of radar data. As an example, the second feature extraction network can adopt the architecture of a point cloud network (PointNet), which is a deep learning model for processing point cloud data. Figure 3 , Figure 3 This is a schematic diagram of the PointNet architecture. Figure 3 As shown in Figure 1, the PointNet architecture includes input transformation (Input Transform), a multilayer perceptron with two hidden layers and 64 neurons in each layer (MLP(64,64)), feature extraction (Feature Transform), and a multilayer perceptron with three hidden layers, with 64, 128, and 1024 neurons respectively (MLP(64,128,1024)). See Figure 3 , the output of the PointNet architecture is a matrix of n×1024. The second feature extraction network adopts the PointNet network structure, and extracts local features through a multi-layer perceptron after grouping the input data.

[0074] In an embodiment of the present application, the target information includes radar acquisition data. The features of the radar acquisition data are extracted by a second feature extraction network, and the extracted features are further processed by a feature fusion network, etc., which is beneficial to improving the accuracy of water mist recognition.

[0075] In one embodiment of the present application, the target information includes location information of the to-be-identified area, the target feature extraction network includes a third feature extraction network, and inputting the target information into the target feature extraction network includes:

[0076] The position information of the area to be identified is input into the third feature extraction network, and the third feature extraction network is used to extract features of the position information.

[0077] In this implementation, the third feature extraction network is used to extract the features of the location information. As an example, the third feature extraction network may adopt a long short-term memory network (Long Short-Term Memory, LSTM) structure. Figure 4 , Figure 4 Schematic diagram of LSTM structure. LSTM controls the flow of information by introducing a gating mechanism, so that it can learn long-term dependencies. In the embodiment of the present application, the output gate of LSTM can be removed, that is, the output of the output gate area is set to zero, and the remaining inputs (x) and outputs (h) remain unchanged.

[0078] In the embodiment of the present application, the target information includes the location information of the area to be identified. In the UAV transportation environment, water mist may appear frequently in a specific production area. The location information of the area to be identified is obtained, and the features of the location information are extracted by the third feature extraction network, and then the extracted features are further processed by the feature fusion network, etc., which is conducive to improving the accuracy of water mist identification.

[0079] In one embodiment of the present application, the target information includes meteorological information of the area to be identified and acquisition time information of the target information, the target feature extraction network includes a fourth feature extraction network, and inputting the target information into the target feature extraction network includes:

[0080] The meteorological information and the acquisition time information are input into the fourth feature extraction network, and the fourth feature extraction network is used to extract features of the meteorological information and the time information.

[0081] The fourth feature extraction network can adopt a three-layer neural network. Figure 5 , Figure 5 The architecture diagram of the three-layer neural network includes an input layer (input x), a hidden layer, and an output layer (each input x corresponds to an output y). Its learning algorithm uses backpropagation, and wij and wjk represent connection weights. Among them, the fourth feature extraction network in the embodiment of the present application uses a three-layer neural network, and a discrete three-value neural network can be used, that is, three discrete values ​​are set to represent the weights in the network and the output of the activation function, thereby simplifying the calculation and improving the robustness.

[0082] In the embodiment of the present application, the target information includes meteorological information and time information. Water mist may appear frequently under specific meteorological conditions and on specific dates. Acquiring meteorological information and time information, extracting features of the meteorological information and time information through the fourth feature extraction network, and further processing the extracted features through the feature fusion network, etc., is conducive to improving the accuracy of water mist recognition.

[0083] In one embodiment of the present application, the feature fusion network may adopt a Transformer architecture, use an independent convolutional encoder to process input features, and interconnect features at different scales with different encoders to generate staged feature fusion. Figure 6 As shown, Figure 6 Figure 1 shows a schematic diagram of the structure of the Transformer architecture, where different encoders are used to process the outputs of different feature extraction networks. Figure 6 , the internal working mechanism of the Transformer structure, including Multi-head Attention, Feedforward Fully Connected Networks (MLPs), Add&Norm, and Masked Multi-head Attention. By adopting the Transformer architecture, the attention mechanism can extract the pattern information that the neural network considers important and update the information from other patterns.

[0084] In the field of steel transportation, the transportation of molten iron between iron mills and steel mills has an important impact on the stability of the production process and the control of cost and quality. At present, the molten iron transportation operation of steel mills mainly involves the transportation of molten iron tanks by manually driven locomotives. In order to realize the unmanned operation of molten iron transportation, a comprehensive perception of the environment along the railway line for molten iron transportation is required. The water mist generated by the production environment of the steel mill and the surrounding natural environment will have a great impact on the results of environmental perception. Based on this, the water mist recognition method in the embodiment of the present application can be applied to the field of molten iron transportation, and the above-mentioned area to be identified can be the area along the molten iron transportation line.

[0085] In the embodiment of the present application, the water mist recognition method is used to identify the area along the molten iron transportation line, which can achieve highly robust multi-information fusion end-to-end water mist recognition. Real-time water mist recognition results are helpful for calibrating the environmental perception results of the automated molten iron transportation locomotive and improving the accuracy and robustness of environmental perception.

[0086] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0087] Figure 7FIG. 1 is a block diagram of a water mist identification device shown in an exemplary embodiment of the present application. Figure 7 As shown, the exemplary water mist identification device includes:

[0088] A first acquisition module 710 is used to acquire an image to be recognized, where the image to be recognized is an image of a region to be recognized;

[0089] The second acquisition module 720 is used to acquire target information, wherein the target information includes at least one of the following: location information of the area to be identified, radar collection data of the area to be identified, meteorological information of the area to be identified, and acquisition time information of the target information;

[0090] The determination module 730 is used to input the image to be identified and the target information into a pre-trained water mist recognition model to obtain a water mist recognition result.

[0091] In one embodiment of the present application, the water mist recognition model includes a first feature extraction network for extracting image features, a target feature extraction network for extracting target features, a feature fusion network, and a water mist recognition result output network, the output data of the first feature extraction network and the target feature extraction network are transmitted to the feature fusion network, and the output data of the feature fusion network is transmitted to the water mist recognition result output network;

[0092] The determination module 730 is specifically used for:

[0093] The image to be identified is input into the first feature extraction network, and the target information is input into the target feature extraction network to obtain the water mist identification result output by the water mist identification result output network.

[0094] In one embodiment of the present application, the first feature extraction network adopts a ResNet network structure, wherein the first feature extraction network includes a maximum pooling layer and multiple bottleneck layers, the maximum pooling layer is used to output a feature map with reduced spatial size, and pass the feature map with reduced spatial size into the multiple bottleneck layers.

[0095] In one embodiment of the present application, the target information includes radar acquisition data of the area to be identified, the target feature extraction network includes a second feature extraction network, and the determination module 730 is further specifically used for:

[0096] The radar collected data is input into the second feature extraction network, and the second feature extraction network is used to extract features of the radar collected data.

[0097] In one embodiment of the present application, the target information includes location information of the to-be-identified area, the target feature extraction network includes a third feature extraction network, and the determination module 730 is further specifically configured to:

[0098] The position information of the area to be identified is input into the third feature extraction network, and the third feature extraction network is used to extract features of the position information.

[0099] In one embodiment of the present application, the target information includes meteorological information of the area to be identified and acquisition time information of the target information, the target feature extraction network includes a fourth feature extraction network, and the determination module is further specifically used for:

[0100] The meteorological information and the acquisition time information are input into the fourth feature extraction network, and the fourth feature extraction network is used to extract features of the meteorological information and the time information.

[0101] In one embodiment of the present application, the area to be identified is an area along the line where molten iron is transported.

[0102] It should be noted that the water mist identification device provided in the above embodiment and the water mist identification method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment, and will not be repeated here. In practical applications, the water mist identification device provided in the above embodiment can allocate the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0103] An embodiment of the present application also provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device implements the water mist recognition method provided in the above-mentioned embodiments.

[0104] Figure 8 The structure diagram of the computer system suitable for implementing the electronic device of the embodiment of the present application is shown. It should be noted that: Figure 8 The computer system 800 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0105] like Figure 8As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 802 or the program loaded from the storage part 808 to the random access memory (RAM) 803, such as executing the method described in the above embodiment. In the RAM 803, various programs and data required for system operation are also stored. The CPU 801, ROM 802 and RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0106] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed so that a computer program read therefrom is installed into the storage section 808 as needed.

[0107] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 809, and / or installed from a removable medium 811. When the computer program is executed by a central processing unit (CPU) 1201, various functions defined in the system of the present application are executed.

[0108] It should be noted that the computer-readable medium shown in the embodiment of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. This propagated data signal can take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. A computer program contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0109] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0110] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0111] Another aspect of the present application further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor of a computer, the computer executes the water mist identification method as described above. The computer-readable storage medium may be included in the electronic device described in the above embodiment, or may exist independently without being assembled into the electronic device.

[0112] Another aspect of the present application further provides a computer program product or a computer program, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the water mist identification method provided in each of the above embodiments.

[0113] The above embodiments are merely illustrative of the principles and effects of the present application, and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A water mist identification method, characterized in that: include: Acquire an image to be identified, where the image to be identified is an image of the area to be identified; Acquire target information, the target information including at least one of the following: location information of the area to be identified, radar collection data of the area to be identified, meteorological information of the area to be identified, and acquisition time information of the target information; The image to be identified and the target information are input into a pre-trained water mist recognition model to obtain a water mist recognition result.

2. The water mist identification method according to claim 1, characterized in that: The water mist recognition model includes a first feature extraction network for extracting image features, a target feature extraction network for extracting target features, a feature fusion network, and a water mist recognition result output network. The output data of the first feature extraction network and the target feature extraction network are both transmitted to the feature fusion network, and the output data of the feature fusion network is transmitted to the water mist recognition result output network. The step of inputting the image to be identified and the target information into a pre-trained water mist identification model to obtain a water mist identification result includes: The image to be identified is input into the first feature extraction network, and the target information is input into the target feature extraction network to obtain the water mist identification result output by the water mist identification result output network.

3. The water mist identification method according to claim 2, characterized in that: The first feature extraction network adopts a ResNet network structure, wherein the first feature extraction network includes a maximum pooling layer and multiple bottleneck layers, the maximum pooling layer is used to output a feature map with reduced spatial size, and pass the feature map with reduced spatial size into the multiple bottleneck layers.

4. The water mist identification method according to claim 2, characterized in that: The target information includes radar acquisition data of the to-be-identified area, the target feature extraction network includes a second feature extraction network, and inputting the target information into the target feature extraction network includes: The radar collected data is input into the second feature extraction network, and the second feature extraction network is used to extract features of the radar collected data.

5. The water mist identification method according to claim 2, characterized in that: The target information includes location information of the to-be-identified area, the target feature extraction network includes a third feature extraction network, and inputting the target information into the target feature extraction network includes: The position information of the area to be identified is input into the third feature extraction network, and the third feature extraction network is used to extract features of the position information.

6. The water mist identification method according to claim 2, characterized in that: The target information includes meteorological information of the to-be-identified area and acquisition time information of the target information, the target feature extraction network includes a fourth feature extraction network, and inputting the target information into the target feature extraction network includes: The meteorological information and the acquisition time information are input into the fourth feature extraction network, and the fourth feature extraction network is used to extract features of the meteorological information and the time information.

7. The water mist identification method according to claim 1, characterized in that: The area to be identified is an area along the line where molten iron is transported.

8. A water mist identification device, characterized in that: include: A first acquisition module is used to acquire an image to be identified, where the image to be identified is an image of a region to be identified; A second acquisition module is used to acquire target information, wherein the target information includes at least one of the following: location information of the area to be identified, radar acquisition data of the area to be identified, meteorological information of the area to be identified, and acquisition time information of the target information; The determination module is used to input the image to be identified and the target information into a pre-trained water mist recognition model to obtain a water mist recognition result.

9. A device, characterized in that: include: one or more processors and memory, A computer program is stored in the memory, and when the one or more processors execute the computer program, the device executes the water mist identification method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, which, when executed by one or more processors, enables the device to perform the water mist identification method as described in any one of claims 1-7.

Images