Model training method, image display method, control device, equipment and medium
By constructing a convolutional neural network edge extraction model for infrared thermal images, the problem that image acquisition devices in thick smoke environments is difficult to acquire clear target outlines, and efficient and accurate target edges in thick smoke, thick fog and dark environments are achieved. Inspection has improved the efficiency of fire rescue.
Patent Information
- Application Number
- CN202411351590.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-26
- Publication Date
- 2025-02-14
AI Technical Summary
During the fire rescue process, thick smoke leads to low visibility on site, making it difficult for the image acquisition device to collect clear target outline images, affecting the rescue efficiency.
A model training method is used to process the characteristics of infrared thermal images to construct an edge extraction model of convolutional neural network structure. This model can efficiently and accurately detect target edges by training infrared thermal images in thick smoke, thick fog and dark environments.
It realizes efficient and accurate target edge detection of infrared thermal images in thick smoke, thick fog and dark environments, improves the ability to quickly and accurately analyze targets in the on-site area, and improves fire rescue efficiency.
Smart Images
Figure CN119445276A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and specifically to a model training method, an image display method, a control device, equipment and a medium. Background Art
[0002] When actually conducting firefighting and rescue, for areas such as fire scenes where there is a lot of thick smoke, the thick smoke causes extremely low visibility in the area, and it is difficult for rescuers to clearly observe the targets to be rescued in the area with the naked eye. Rescuers need to spend a lot of time searching for the targets to be rescued, which in turn affects the efficiency of firefighting and rescue. With the rapid development of image acquisition technology, image acquisition devices such as infrared thermal cameras and night vision cameras are widely used in emergency areas such as firefighting and rescue. Rescue detection equipment commonly used in emergency areas includes image acquisition devices such as infrared thermal cameras and night vision cameras, as well as display devices such as AR display screens. The image acquisition device is used to collect images of the scene area, and the display device is used to display the image.
[0003] Rescuers can improve rescue efficiency and success rate by observing the images displayed by the display device. However, when the smoke density in the scene area is too high, the visibility in the scene area is too low, and the image acquisition device can only capture a fuzzy target outline image with poor recognition. The display device cannot obtain a clear target outline image, resulting in the inability to quickly and accurately analyze the target in the scene area, which in turn affects the efficiency of firefighting and rescue. Summary of the invention
[0004] The purpose of the embodiments of the present application is to provide a model training method, an image display method, a control device, equipment and a medium. The model training method is used to solve the problem in the prior art that it is impossible to quickly and accurately analyze targets in a field area.
[0005] In order to achieve the above-mentioned purpose, the first aspect of the present application provides a model training method, which includes:
[0006] Processing the edge detection image data set according to the characteristics of the infrared thermal image to obtain a first sample image data set;
[0007] Capturing images of a preset area in a target environment to obtain a sample image of the target environment, wherein the target environment includes a dense smoke environment, a dense fog environment, and a dark environment;
[0008] According to the target environment sample image, a second sample image data set is obtained;
[0009] Build a preset model based on the convolutional neural network structure;
[0010] Using the first sample image data set to train the preset model to obtain an initial model;
[0011] The initial model is trained using the second sample image data set to obtain an edge extraction model.
[0012] In an embodiment of the present application, the edge detection image data set is processed according to the characteristics of the infrared thermal image to obtain a first sample image data set, including:
[0013] According to the characteristics of the infrared thermal image, the edge detection image data set is fuzzy processed and noise added to obtain a first sample image data set.
[0014] In an embodiment of the present application, a preset model is trained using a first sample image data set to obtain an initial model, including:
[0015] Inputting the first sample image data set into a preset number of preset models respectively to obtain a preset number of trained preset models;
[0016] Determine the performance parameters of each trained preset model respectively;
[0017] The target preset model is determined as the initial model, wherein the target preset model is the model with the best performance parameters among all trained preset models.
[0018] In an embodiment of the present application, the initial model is trained using the second sample image data set to obtain an edge extraction model, including:
[0019] Performing a first-stage training on the initial model using the second sample image data set to obtain an iterated initial model, wherein parameters of a backbone feature extraction network of the initial model are frozen in the first-stage training;
[0020] The second sample image data set is used to perform the second stage training on the iterated initial model to obtain an edge extraction model, wherein the parameters of the backbone feature extraction network of the iterated initial model are unfrozen in the second stage training.
[0021] A second aspect of the present application provides an image display method, which is applied to a detection device, wherein the detection device includes an image acquisition device and a display device, and the control method of the detection device includes:
[0022] In response to a request for displaying a dense smoke environment or a request for displaying a dense fog environment, an infrared thermal image of a target area is acquired by using an image acquisition device;
[0023] Inputting the infrared thermal image into the edge extraction model, and using the edge extraction model to extract the contour of the target in the infrared thermal image to obtain the target contour image, wherein the edge extraction model is obtained according to the above-mentioned model training method;
[0024] Performing image enhancement processing on the target contour image to obtain a target contour enhanced image;
[0025] Based on the target contour enhanced image, the display device is controlled to generate an image interface.
[0026] In an embodiment of the present application, performing image enhancement processing on a target contour image to obtain a target contour enhanced image includes:
[0027] The color of the target contour in the target contour image is enhanced to a preset color to obtain a target contour enhanced image.
[0028] In an embodiment of the present application, the image display method further includes:
[0029] In response to a dark environment display request, a night vision image of a target area is acquired by using an image acquisition device;
[0030] The contour of the target in the night vision image is extracted to obtain the target contour image.
[0031] In an embodiment of the present application, based on the target contour enhanced image, controlling the display device to generate an image interface includes:
[0032] In response to an image overlay request, the target contour enhancement image is overlaid with the infrared thermal image to obtain an overlaid image;
[0033] Based on the superimposed image, the display device is controlled to generate an image interface.
[0034] A third aspect of the present application provides a control device, comprising:
[0035] a memory configured to store instructions;
[0036] A processor is configured to call instructions from a memory and to implement the above-mentioned model training method or the above-mentioned image display method when executing the instructions.
[0037] A fourth aspect of the present application provides a detection device, including:
[0038] According to the control device described above;
[0039] an image acquisition device configured to acquire an image of a target area;
[0040] The display device is configured to generate a graphic interface.
[0041] A fifth aspect of the present application provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the above-mentioned model training method or the above-mentioned image display method.
[0042] The present application provides a model training method, including: processing an edge detection image data set according to the characteristics of an infrared thermal image to obtain a first sample image data set; collecting images of a preset area in a target environment to obtain a sample image of the target environment; obtaining a second sample image data set according to the sample image of the target environment; constructing a preset model based on a convolutional neural network structure; training the preset model using the first sample image data set to obtain an initial model; and training the initial model using the second sample image data set to obtain an edge extraction model. A sample data set is obtained according to the characteristics of the infrared thermal image and the image in the actual target environment. The edge extraction model obtained through the sample data set can perform efficient and accurate target edge detection according to the characteristics of the infrared thermal image in the current environment, and can then quickly and accurately analyze the targets in the field area.
[0043] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present application, but do not constitute a limitation on the embodiments of the present application. In the accompanying drawings:
[0045] Figure 1 A schematic diagram of a process flow of a model training method according to an embodiment of the present application is schematically shown;
[0046] Figure 2 A schematic diagram of a process of an image display method according to an embodiment of the present application is schematically shown;
[0047] Figure 3 An example diagram of an image with added target contour according to an embodiment of the present application is schematically shown. DETAILED DESCRIPTION
[0048] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application, and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0049] It should be noted that if directional indication is involved in the embodiments of the present application, the directional indication is only used to explain the relative position relationship, movement status, etc. between the components in a certain specific posture. If the specific posture changes, the directional indication will also change accordingly.
[0050] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.
[0051] Example 1
[0052] Figure 1 The following schematically shows a flow chart of a model training method according to an embodiment of the present application. Figure 1 As shown, the embodiment of the present application provides a model training method, and the model training method includes:
[0053] S110, processing the edge detection image data set according to the characteristics of the infrared thermal image to obtain a first sample image data set.
[0054] When the scene area is the target environment, the visibility of the scene area is low, and it is difficult for the user to search for the target in the scene area with the naked eye. The target environment is one of a thick smoke environment, a thick fog environment and a dark environment. Usually, infrared thermal images are collected for the scene area, and then the target is searched through the collected infrared thermal images.
[0055] It should be understood that the type of edge detection image dataset is set according to actual needs and can be composed of any visible light edge detection dataset, which is not limited here. For ease of understanding, the edge detection image dataset in the embodiment of the present application includes the BSDS500 dataset and the BIPED dataset.
[0056] When images are collected in a non-target environment, clear images can be collected due to the high visibility in the on-site area. However, the infrared thermal images collected in the target environment have the characteristics of blurred edges and low contrast. The edge detection image dataset is processed according to the characteristics of the infrared thermal images, that is, the BSDS500 dataset and the BIPED dataset are specifically processed respectively to obtain the first sample image dataset.
[0057] In an embodiment of the present application, the edge detection image data set is processed according to the characteristics of the infrared thermal image to obtain a first sample image data set, including:
[0058] According to the characteristics of the infrared thermal image, the edge detection image data set is fuzzy processed and noise added to obtain a first sample image data set.
[0059] Infrared thermal images differ from visible light images in imaging principles and image characteristics, and the image data in the edge detection image datasets usually obtained are all visible light images. According to the characteristics of infrared thermal images, the edge detection image datasets are fuzzy processed and noise added to obtain the first sample image dataset. The sample image data in the first sample image dataset conforms to the characteristics of infrared thermal images, and can provide the training effect of the machine learning model.
[0060] S120, collecting images of a preset area in a target environment to obtain a sample image of the target environment, wherein the target environment includes a dense smoke environment, a dense fog environment, and a dark environment.
[0061] It should be understood that smoke cakes can be used to generate thick smoke in the preset area, or a smoke machine can be used to generate thick fog in the preset area to simulate the complex visual conditions in the actual target environment, which will not be elaborated here. Use infrared cameras and other devices to collect images of the preset area in the target environment to obtain sample images of the target environment as training sample data for the model.
[0062] S130, obtaining a second sample image data set according to the target environment sample image;
[0063] Machine learning model training relies on a large amount of sample data, and image collection can be performed in preset areas of multiple target environments to obtain a large number of actual environment images. A large number of actual environment images are annotated, and then a second sample image data set is obtained based on the target environment sample images.
[0064] Sample data labeling is an important step in machine learning model training. The target environment sample images are labeled to obtain the second sample image data so that the machine learning model can recognize and learn the sample data. By effectively labeling the target environment sample images to obtain the second sample image data set, the training quality and performance of the machine learning model can be improved, thereby more accurately analyzing the infrared thermal images in the target environment.
[0065] S140, builds a preset model based on the convolutional neural network structure;
[0066] Edge detection operators such as the Canny operator, Laplace operator, and Sobel operator are usually used for edge infrared image detection. However, some parameters of edge detection operators usually need to be manually selected, resulting in poor generalization ability. In addition, edge detection operators use local information and underlying information for detection, making it difficult to achieve robust detection in infrared thermal image detection under target environments.
[0067] Based on the Convolutional Neural Networks (CNN) structure, a preset model is constructed. When edge extraction is performed based on the preset model of the convolutional neural network, rich hierarchical features can be automatically learned through nested multi-scale feature fusion, which has better performance than the edge detection operator.
[0068] Aiming at the edge extraction accuracy and real-time requirements of infrared thermal images acquired in indoor target environments, as well as the edge model caused by heat crossover, noise introduced by thermal sensors, low resolution and low contrast of infrared thermal images, a preset model for global edge extraction is constructed. The preset model based on the convolutional neural network structure adds dense connections and channel space attention mechanism (Convolutional Block Attention Module, CBAM) in the feature stage to transfer the shallow contextual information and texture information of the infrared thermal image to the deep layer, thereby capturing more detailed features. At the same time, the preset model obtains the importance of each feature channel and feature space through learning, thereby extracting important features, and finally fuses the edge maps output by each block to obtain the final edge feature extraction map.
[0069] In an embodiment of the present application, a preset model is trained using a first sample image data set to obtain an initial model, including:
[0070] Inputting the first sample image data set into a preset number of preset models respectively to obtain a preset number of trained preset models;
[0071] Determine the performance parameters of each trained preset model respectively;
[0072] The target preset model is determined as the initial model, wherein the target preset model is the model with the best performance parameters among all trained preset models.
[0073] The first sample image data set is input into a preset number of preset models respectively to obtain a preset number of trained preset models. For ease of understanding, the model parameters in the embodiment of the present application are initialized using a Gaussian random distribution with a mean of 0 and a standard deviation of 0.02. The total number of iterations for training based on the first sample image data set is 300 times, the batch_size is 16, and the initial learning rate is 0.0005. During the preset model iteration process, the current learning rate is the initial learning rate multiplied by (1-iter / max_iter) power , where iter is the current iteration number of the preset model, max_iter is the total iteration number, and power is the momentum parameter.
[0074] After each preset model completes the iteration, a preset number of trained preset models are obtained. The performance parameters of each trained preset model are determined respectively. The target preset model with the best performance parameters among all the trained preset models is determined as the initial model.
[0075] S150, training a preset model using the first sample image data set to obtain an initial model;
[0076] Usually, edge detection models based on deep networks are used for edge detection of visible light images. However, due to the differences in imaging principles and image characteristics between infrared thermal images and visible light images, edge detection models based on deep networks cannot accurately extract edges of infrared thermal images in the target environment. At the same time, when the edge detection model based on the deep network is running on a computing device, the inference frame rate is low, resulting in visual delays. In this embodiment, no pre-training is used, but the edge extraction model is trained from scratch. The preset model is trained using the first sample image data set to obtain an initial model.
[0077] S160, using the second sample image data set to train the initial model to obtain an edge extraction model.
[0078] The initial model is trained using the second sample image data set, and the trained initial model analyzes and identifies infrared thermal images in a dense fog environment to obtain an edge extraction model. It should be understood that the training steps of the machine learning model are based on actual needs and are not limited here. A sample data set is obtained based on the characteristics of infrared thermal images and images in the actual target environment. The edge extraction model obtained through the sample data set can perform efficient and accurate target edge detection based on the characteristics of infrared thermal images in the target environment, and thus can quickly and accurately analyze targets in the field area.
[0079] In an embodiment of the present application, the initial model is trained using the second sample image data set to obtain an edge extraction model, including:
[0080] Performing a first-stage training on the initial model using the second sample image data set to obtain an iterated initial model, wherein parameters of a backbone feature extraction network of the initial model are frozen in the first-stage training;
[0081] The second sample image data set is used to perform the second stage training on the iterated initial model to obtain an edge extraction model, wherein the parameters of the backbone feature extraction network of the iterated initial model are unfrozen in the second stage training.
[0082] The model parameters of the initial model are used as pre-training parameters, and the second sample image data set is used to perform two-stage model training to obtain an edge extraction model. Specifically, the initial model is trained in the first stage using the second sample image data set to obtain an iterated initial model. The first stage training is a freezing stage, that is, the parameters of the backbone feature extraction network of the initial model are frozen in the first stage training, the weights of the model feature extraction part do not change, and only the entire network is fine-tuned. For ease of understanding, the number of iterations of the first stage training in the embodiment of the present application is 600, batch_size is 16, the learning rate is 0.0002, and the learning rate update strategy adopts fixed step decay.
[0083] In the first phase of training, the backbone feature extraction network is frozen so that it is not updated, and the backbone feature extraction network is ensured to remain stable to focus on extracting features. Fine-tuning the entire network of the initial model can reduce the complexity of training and avoid overfitting.
[0084] The second sample image data set is used to perform the second stage training on the iterated initial model to obtain the edge extraction model. The second stage is the thawing stage, that is, the parameters of the backbone feature extraction network of the iterated initial model are thawed in the second stage training, and the weights of the backbone feature extraction network are also updated. For ease of understanding, the number of iterations of the first stage training in the embodiment of the present application is 60, batch_size is 8, the learning rate is 0.00002, and the learning rate update strategy adopts fixed step decay.
[0085] The present application provides a model training method, including: processing an edge detection image data set according to the characteristics of an infrared thermal image to obtain a first sample image data set; collecting images of a preset area in a target environment to obtain a sample image of the target environment; obtaining a second sample image data set according to the sample image of the target environment; constructing a preset model based on a convolutional neural network structure; training the preset model using the first sample image data set to obtain an initial model; and training the initial model using the second sample image data set to obtain an edge extraction model. A sample data set is obtained according to the characteristics of the infrared thermal image and the image in the actual target environment. The edge extraction model obtained through the sample data set can perform efficient and accurate target edge detection according to the characteristics of the infrared thermal image in the current environment, and can then quickly and accurately analyze the targets in the field area.
[0086] Example 2
[0087] Figure 2 The flowchart of an image display method according to an embodiment of the present application is schematically shown. Figure 2 As shown, an embodiment of the present application provides an image display method, and the image display method includes:
[0088] S210, in response to a request for displaying a dense smoke environment or a request for displaying a dense fog environment, using an image acquisition device to acquire an infrared thermal image of a target area.
[0089] In the scenario of target search for the target area, if the current environment of the target area is a dense smoke environment or a dense fog environment, a display request is generated according to the user's operation. The detection device responds to the dense smoke environment display request or the dense fog environment display request, and uses the image acquisition device to acquire the infrared thermal image of the target area. It should be understood that the image acquisition device can be used to acquire the infrared thermal video of the target area, and the infrared thermal image of the target area can be obtained through the infrared thermal video, which will not be elaborated here.
[0090] S220, inputting the infrared thermal image into an edge extraction model, and using the edge extraction model to extract the contour of the target in the infrared thermal image to obtain a target contour image, wherein the edge extraction model is obtained according to the above-mentioned model training method.
[0091] Targets radiated by heat sources, such as electrical appliances, are visible thermal images in infrared thermal images, and users can quickly identify targets radiated by heat sources through infrared thermal images. However, it is difficult to obtain visible thermal images of targets without heat sources or without heat radiation in infrared thermal images. Since infrared thermal images in smoke or foggy environments have blurred edges and low contrast, it is difficult for users to identify targets without heat sources or without heat radiation through infrared thermal images, which in turn affects the efficiency of target search.
[0092] In this embodiment, the infrared thermal image is input into the edge extraction model, and the edge extraction model is used to extract the contour of the target in the infrared thermal image to obtain the target contour image. The sample data set is obtained through the characteristics of the infrared thermal image and the image in the actual environment for model training, so that the edge extraction model obtained through the sample data set can perform efficient and accurate target edge detection according to the characteristics of the infrared thermal image in the actual environment, and then extract an accurate target contour image through the edge extraction model. The target in the target area changes according to the actual area, and can be a door, stairs, road, tree, and rescue personnel, etc., which are not limited here.
[0093] S230, performing image enhancement processing on the target contour image to obtain a target contour enhanced image.
[0094] The target contour image is enhanced to obtain a target contour enhanced image. The image enhancement processing strategy is set according to actual needs and is not limited here. By performing image enhancement processing on the target contour image, the target contour in the image is made clearer, so that the user can quickly distinguish different targets based on the target contour enhanced image.
[0095] In an embodiment of the present application, performing image enhancement processing on a target contour image to obtain a target contour enhanced image includes:
[0096] The color of the target contour in the target contour image is enhanced to a preset color to obtain a target contour enhanced image.
[0097] Figure 3 An example diagram of an image with added target contour according to an embodiment of the present application is schematically shown.
[0098] In this embodiment, the target contour image is color enhanced, that is, the color of the target contour in the target contour image is enhanced to a preset color, and a target contour enhanced image with a clearer contour is obtained. It should be understood that the preset color is set according to actual needs and is not limited here. As shown in the figure, for ease of understanding, the preset color in the embodiment of the present application is green. Through the target contour enhanced image, the user can observe the clear contour of the target, so that the user can quickly and accurately analyze the target in the target area.
[0099] It should be understood that the target contour image can also be enhanced by other image enhancement processing methods, such as denoising, contrast stretching and edge sharpening, which will not be elaborated here.
[0100] S240, based on the target contour enhanced image, controlling the display device to generate an image interface.
[0101] The type of detection equipment is set according to actual needs and is not limited here. For ease of understanding, the detection equipment in the embodiment of the present application is a head-mounted AR (Augmented Reality) device. When the target area is a target environment, the user can wear the detection equipment in this embodiment, wherein the target environment includes a dense smoke environment, a dense fog environment, and a dark environment. The detection equipment enhances the image based on the target contour and controls the display device to generate an image interface. The user can quickly identify different targets such as doors, stairs, and people to be rescued through the image interface.
[0102] It should be understood that the noisy edge contour in the target contour added image can also be de-noised to make the target contour in the image clearer. Based on the de-noised target contour enhanced image, the display device is controlled to generate an image interface. The de-noising method is set according to actual needs, and can be morphological filtering processing, etc., which is not limited here.
[0103] The edge extraction model can perform efficient and accurate target edge detection based on the characteristics of infrared thermal images in the target environment, and enhance the target contour image obtained by the edge extraction model to generate an image interface with a clear target contour. In the case of indoor areas and other target environments, users can quickly and accurately analyze the targets in the target area through the image interface, thereby improving the efficiency of target search. In addition, the image interface can also be input into the target recognition model to quickly identify the target using the target recognition model.
[0104] In an embodiment of the present application, the image display method further includes:
[0105] In response to a dark environment display request, a night vision image of a target area is acquired by using an image acquisition device;
[0106] The contour of the target in the night vision image is extracted to obtain the target contour image.
[0107] In this embodiment, the detection device is used for target search. Usually, there is also a situation where the target search is performed in a dark environment, and a dark environment display request is generated according to the user's operation on the detection device. It should be understood that the detection device responds to the dark environment display request, and the image acquisition device can collect the night vision image of the target area, and can also collect the infrared thermal image of the target area. For ease of understanding, in the embodiments of the present application, when the actual environment is a dark environment, and the actual environment is not a dense smoke environment or a dense fog environment, the image acquisition device is used to collect the night vision image of the target area; when the actual environment is a dark environment, and the actual environment is also a dense smoke environment or a dense fog environment, the image acquisition device is used to collect the infrared thermal image of the target area.
[0108] When a night vision image of the target area is acquired, the contour of the target in the night vision image is extracted to obtain a target contour image. Since night vision images have a higher contrast than infrared thermal images in dense smoke and dense fog environments, the method of extracting the contour of the target in the night vision image is set according to actual needs. The contour of the target in the night vision image can be extracted using an edge extraction model, or an edge detection operator can be used to extract the contour of the target in the night vision image, which is not limited here. When the edge extraction model is used to extract the contour of the target in the night vision image, since the second sample image data set includes sample images of a dark environment, the edge extraction model in this embodiment can directly extract the contour of the target in the night vision image.
[0109] In an embodiment of the present application, based on the target contour enhanced image, controlling the display device to generate an image interface includes:
[0110] In response to an image overlay request, the target contour enhancement image is overlaid with the infrared thermal image to obtain an overlaid image;
[0111] Based on the superimposed image, the display device is controlled to generate an image interface.
[0112] The image acquisition device in this embodiment includes an infrared thermal camera, a night vision camera and a visible light camera. In response to an image superposition request, the target contour enhanced image is superimposed with the infrared thermal image collected by the infrared thermal camera to obtain a superimposed image. It should be understood that the detection device can also superimpose the target contour enhanced image with the night vision image collected by the night vision camera, or superimpose the target contour enhanced image with the visible light image collected by the visible light camera to obtain a superimposed image according to the user's operation.
[0113] Based on the superimposed image, the display device is controlled to generate an image interface. According to the image interface generated by the superimposed image, the user can quickly obtain different targets in the target area and the environment around the target through the image interface, thereby improving the target search efficiency.
[0114] The present application also provides a control device, including:
[0115] a memory configured to store instructions;
[0116] A processor is configured to call instructions from a memory and to implement the above-mentioned model training method or the above-mentioned image display method when executing the instructions.
[0117] The processor includes a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and the kernel parameters can be adjusted to solve the problem of being unable to quickly and accurately analyze the target in the field area.
[0118] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.
[0119] The present application also provides a detection device, including:
[0120] According to the control device described above;
[0121] an image acquisition device configured to acquire an image of a target area;
[0122] The display device is configured to generate a graphic interface.
[0123] The control device obtains the edge extraction model in advance, and when searching for the target area, the image acquisition device acquires the image of the target area. The edge extraction model is used to extract the edge contour to obtain the target contour image. The target contour image is subjected to image enhancement processing to obtain the target contour enhanced image. The display device generates and displays the image interface based on the target contour enhanced image. The user can quickly identify different targets such as doors, stairs and people to be rescued through the image interface, and then quickly determine the location of the target to be searched.
[0124] An embodiment of the present application also provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the above-mentioned model training method or the above-mentioned image display method.
[0125] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0126] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.
[0127] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0129] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0130] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0131] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0132] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0133] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.
Claims
1. A model training method, characterized in that: The model training method comprises: Processing the edge detection image data set according to the characteristics of the infrared thermal image to obtain a first sample image data set; Capturing images of a preset area in a target environment to obtain a sample image of the target environment, wherein the target environment includes a dense smoke environment, a dense fog environment, and a dark environment; Obtaining a second sample image data set according to the target environment sample image; Build a preset model based on the convolutional neural network structure; Using the first sample image data set to train the preset model to obtain an initial model; The initial model is trained using the second sample image data set to obtain an edge extraction model.
2. The model training method according to claim 1, characterized in that: The edge detection image data set is processed according to the characteristics of the infrared thermal image to obtain a first sample image data set, including: According to the characteristics of the infrared thermal image, the edge detection image data set is fuzzy processed and noise added to obtain a first sample image data set.
3. The model training method according to claim 1, characterized in that: The step of training the preset model using the first sample image data set to obtain an initial model includes: Inputting the first sample image data set into a preset number of preset models respectively to obtain a preset number of trained preset models; Respectively determining the performance parameters of each of the trained preset models; The target preset model is determined as the initial model, wherein the target preset model is the model with the best performance parameters among all trained preset models.
4. The model training method according to claim 1, characterized in that: The step of training the initial model using the second sample image data set to obtain an edge extraction model includes: Performing a first-stage training on the initial model using the second sample image data set to obtain an iterated initial model, wherein parameters of a backbone feature extraction network of the initial model are frozen in the first-stage training; The second sample image data set is used to perform a second stage of training on the iterated initial model to obtain an edge extraction model, wherein the parameters of the backbone feature extraction network of the iterated initial model are unfrozen in the second stage of training.
5. An image display method, characterized in that: Applied to a detection device, the detection device includes an image acquisition device and a display device, and the control method of the detection device includes: In response to a request for displaying a dense smoke environment or a request for displaying a dense fog environment, using the image acquisition device to acquire an infrared thermal image of a target area; Inputting the infrared thermal image into an edge extraction model, and using the edge extraction model to extract the contour of the target in the infrared thermal image to obtain a target contour image, wherein the edge extraction model is obtained according to the model training method described in any one of claims 1 to 4; Performing image enhancement processing on the target contour image to obtain a target contour enhanced image; Based on the target contour enhanced image, the display device is controlled to generate an image interface.
6. The image display method according to claim 5, characterized in that: The step of performing image enhancement processing on the target contour image to obtain a target contour enhanced image includes: The color of the target contour in the target contour image is enhanced to a preset color to obtain a target contour enhanced image.
7. The image display method according to claim 5, characterized in that: The image display method further includes: In response to a dark environment display request, using the image acquisition device to acquire a night vision image of a target area; The contour of the target in the night vision image is extracted to obtain a target contour image.
8. The image display method according to claim 5, characterized in that: The step of controlling the display device to generate an image interface based on the target contour enhanced image comprises: In response to an image overlay request, the target contour enhanced image is overlaid with the infrared thermal image to obtain an overlaid image; Based on the superimposed image, the display device is controlled to generate an image interface.
9. A control device, characterized in that: include: a memory configured to store instructions; A processor is configured to call the instructions from the memory and to implement the model training method according to any one of claims 1 to 4, or the image display method according to any one of claims 5 to 8 when executing the instructions.
10. A detection device, characterized in that: include: A control device according to claim 9; an image acquisition device configured to acquire an image of a target area; The display device is configured to generate a graphic interface.
11. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores instructions for enabling a machine to execute the model training method according to any one of claims 1 to 4, or the image display method according to any one of claims 5 to 8.