Dual-light image display method and apparatus, device, and medium

Abstract

A dual-light image display method and apparatus, a device, and a medium, relating to the technical field of image display. The method comprises: acquiring a current infrared original image and visible light original image in a target scene, and respectively determining a bottom-layer image and a top-layer image from the infrared original image and the visible light original image; performing target detection on the bottom-layer image to obtain a bottom-layer target position, and determining a region to be displayed from the top-layer image on the basis of the bottom-layer target position; and displaying said region in the top-layer image on an upper layer of the bottom-layer image in a picture-in-picture form. By means of the described solution, automatic and intelligent dual-light image display is achieved without manually selecting a region to be displayed, thereby reducing the difficulty of and time required for dual-light image display, and greatly improving user experience.

Description

Dual-light image display method, device, equipment and medium

[0001] The present application claims priority to the Chinese patent application No. 202410800052.0, filed on June 20, 2024, and entitled "Dual-light image display method, device, equipment and medium", the content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the technical field of image display, in particular to a dual-light image display method, device, equipment and medium. BACKGROUND

[0003] With the development of infrared technology, more and more infrared product forms appear, in addition to traditional single infrared equipment, there are visible light and infrared dual-light equipment. The dual-light equipment is beneficial to distinguish detailed color information while maintaining the infrared discovery of the heat source target, and specifically identify the target. The modes of dual-light image display include single infrared mode, single visible light mode, fusion mode, picture-in-picture mode, etc., among which the picture-in-picture display commonly includes full-frame infrared and full-frame visible light display on the same screen, or one light source image as the background and the other light source image as the picture-in-picture display.

[0004] In the existing dual-light picture-in-picture display scheme, it is usually necessary to manually select the region of interest, such as selecting by selecting a frame or using an erasing action, and then displaying the region of interest as a picture-in-picture with another light source image; that is, the region that needs to be displayed as a picture-in-picture cannot be automatically and intelligently determined, and manual selection wastes time and is difficult to operate with low accuracy.

[0005] In summary, how to automatically and intelligently display dual-light images and reduce the difficulty and time required for dual-light image display is a problem to be solved in the art. SUMMARY

[0006] Therefore, the purpose of the present application is to provide a dual-light image display method, device, equipment and medium, which can automatically and intelligently display dual-light images and reduce the difficulty and time required for dual-light image display. The specific scheme is as follows:

[0007] In a first aspect, the present application discloses a dual-light image display method, comprising:

[0008] obtaining an infrared original image and a visible light original image under a target scene, and determining a bottom layer image and a top layer image from the infrared original image and the visible light original image, respectively;

[0009] performing target detection on the bottom layer image to obtain a bottom layer target position, and determining a to-be-displayed region from the top layer image based on the bottom layer target position;

[0010] display the to-be-displayed region in the top layer image on the bottom layer image in a picture-in-picture manner.

[0011] Optionally, in the process of acquiring the current infrared raw image and the visible light raw image under the target scene, the method further includes:

[0012] registering the infrared raw image and the visible light raw image to align the infrared raw image and the visible light raw image.

[0013] Optionally, the target detection on the bottom layer image to obtain the bottom layer target position and the determination of the to-be-displayed region from the top layer image based on the bottom layer target position include:

[0014] target detection based on the bottom layer image to obtain an initial target position;

[0015] screening, from the initial target position, a bottom layer target position satisfying a preset true target condition;

[0016] determination of the to-be-displayed region from the top layer image based on the bottom layer target position.

[0017] Optionally, the target detection based on the bottom layer image to obtain an initial target position includes:

[0018] target detection on the bottom layer image by using a deep learning algorithm and / or a traditional machine learning algorithm to obtain an initial target position; wherein the traditional machine learning algorithm includes a connected region detection algorithm and a moving object detection algorithm.

[0019] Optionally, the screening, from the initial target position, of a bottom layer target position satisfying a preset true target condition includes:

[0020] acquisition of indication information issued by a user by using a preset interface, and determination of a position corresponding to the indication information in the initial target position as the bottom layer target position.

[0021] Optionally, the screening, from the initial target position, of a bottom layer target position satisfying a preset true target condition includes:

[0022] generating, in a preset form, prompt information of a target corresponding to the initial target position;

[0023] acquisition of a target selection instruction issued by the user based on the prompt information by using a preset interface, and screening, from the initial target position, of a bottom layer target position satisfying a preset true target condition based on the target selection instruction.

[0024] Optionally, the filtering of the bottom target position from the initial target position that meets the preset true target condition comprises:

[0025] determining each target corresponding to the initial target position and a gyroscope stabilization time and position information of each target;

[0026] filtering of the bottom target position from the initial target position that meets the preset true target condition according to a size relationship between the gyroscope stabilization time and a preset stabilization time threshold and the position information.

[0027] Optionally, the determining of the to-be-displayed region from the top image based on the bottom target position comprises:

[0028] target detection on the entire top image to obtain a top target position;

[0029] determining of the to-be-displayed region from the top image based on the bottom target position and the top target position.

[0030] Optionally, the determining of the to-be-displayed region from the top image based on the bottom target position comprises:

[0031] filtering of a to-be-detected region from the top image based on the bottom target position, and target detection on the to-be-detected region in the top image to obtain a top target position;

[0032] determining of the to-be-displayed region from the top image based on the bottom target position and the top target position.

[0033] Optionally, the determining of the to-be-displayed region from the top image based on the bottom target position and the top target position comprises:

[0034] determining of an initial display region from the top image based on the top target position;

[0035] aligning of a bottom anchor frame corresponding to the bottom target position with a top anchor frame corresponding to the top target position, and recording of scaling information and offset information in the aligning process of the bottom anchor frame and the top anchor frame;

[0036] processing of the initial display region according to the scaling information and the offset information to obtain a to-be-displayed region.

[0037] Optionally, the displaying of the to-be-displayed region in the top image on the bottom image in a picture-in-picture form comprises:

[0038] If there are multiple areas to be displayed in the top-level image, the target area to be displayed is selected from the areas to be displayed according to the display command given by the user, and the target area to be displayed in the top-level image is displayed on the top layer of the bottom layer image in a picture-in-picture format.

[0039] Secondly, this application discloses a dual-light image display device, comprising:

[0040] The image acquisition module is used to acquire the current infrared raw image and visible light raw image in the target scene, and to determine the bottom layer image and the top layer image from the infrared raw image and the visible light raw image, respectively.

[0041] The region determination module is used to perform target detection on the bottom layer image to obtain the position of the bottom layer target, and determine the region to be displayed from the top layer image based on the position of the bottom layer target;

[0042] The area display module is used to display the area to be displayed in the top layer image on top of the bottom layer image in a picture-in-picture format.

[0043] Thirdly, this application discloses an electronic device, including:

[0044] Memory, used to store computer programs;

[0045] A processor is configured to execute the computer program to implement the steps of the aforementioned disclosed dual-light image display method.

[0046] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the aforementioned disclosed dual-light image display method.

[0047] The beneficial effects of this application are as follows: This application acquires the current original infrared image and original visible light image in the target scene, and determines the bottom layer image and the top layer image from the original infrared image and the original visible light image, respectively; it performs target detection on the bottom layer image to obtain the bottom target position, and determines the area to be displayed from the top layer image based on the bottom target position; it then displays the area to be displayed in the top layer image in a picture-in-picture format on top of the bottom layer image. Therefore, this application performs target detection on the bottom layer image to obtain the bottom target position, determines the area to be displayed from the top layer image based on the bottom target position, and then displays the area to be displayed in the top layer image in a picture-in-picture format on top of the bottom layer image. This achieves automatic and intelligent dual-light image display without the need for manual selection of the area to be displayed, reducing the difficulty and time required for dual-light image display, and greatly improving the user experience. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0049] Figure 1 is a flowchart of a dual-light image display method disclosed in this application;

[0050] Figure 2 is a flowchart of the first specific method for determining the area to be displayed disclosed in this application;

[0051] Figure 3 is a flowchart of the second specific method for determining the area to be displayed disclosed in this application;

[0052] Figure 4 is a flowchart of the third specific method for determining the area to be displayed disclosed in this application;

[0053] Figure 5 is a specific picture-in-picture display diagram disclosed in this application;

[0054] Figure 6 is a schematic diagram of another specific picture-in-picture display disclosed in this application;

[0055] Figure 7 is a flowchart of the first specific dual-light image display method disclosed in this application;

[0056] Figure 8 is a flowchart of the second specific dual-light image display method disclosed in this application;

[0057] Figure 9 is a schematic diagram of the structure of a dual-light image display device disclosed in this application;

[0058] Figure 10 is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0059] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0060] With the development of infrared technology, more and more infrared product forms have emerged. In addition to traditional single infrared devices, there are also dual-light devices that combine visible light and infrared. Dual-light devices, while maintaining infrared detection of heat sources, are beneficial for distinguishing detailed color information and specifically identifying targets. Dual-light image display modes include single infrared mode, single visible light mode, fusion mode, picture-in-picture mode, etc. Among them, picture-in-picture displays commonly include full-frame infrared and full-frame visible light simultaneous display, or using one light source image as a background and displaying another light source image picture-in-picture.

[0061] In existing dual-light picture-in-picture display solutions, it is usually necessary to manually select the area of ​​interest, such as by using a selection box or erasing action, and then display the area of ​​interest in picture-in-picture with another light source image. In other words, it is impossible to automatically and intelligently determine the area that needs to be displayed in picture-in-picture. Manual selection is time-consuming, difficult to operate, and has low accuracy.

[0062] Therefore, this application provides a dual-light image display scheme that can automatically and intelligently display dual-light images, reducing the difficulty and time required for dual-light image display.

[0063] Referring to Figure 1, this application discloses a dual-light image display method, including:

[0064] Step S11: Obtain the current infrared raw image and visible light raw image in the target scene, and determine the bottom layer image and top layer image from the infrared raw image and the visible light raw image, respectively.

[0065] In this embodiment, the process of acquiring the current infrared and visible light original images in the target scene further includes: registering the infrared and visible light original images to align them. When the dual-light device structure design is unreasonable, the acquired infrared and visible light original images differ significantly. However, due to the high complexity and cost of hardware adjustments, to improve the subsequent detection effect, the infrared and visible light original images can be registered in software to ensure their consistency and avoid the inability to determine the corresponding display area from the top-level image based on the underlying target position. Furthermore, if the dual-light device structure design is reasonable, with parallel optical axes, suitable focal lengths, and similar center distances, the step of registering the infrared and visible light original images can be omitted, reducing process complexity and cost.

[0066] Understandably, the raw infrared image can be used as the bottom layer image and the raw visible light image as the top layer image, or vice versa. Users can choose between the bottom layer image and the top layer image based on the specific target scene.

[0067] Step S12: Perform target detection on the bottom layer image to obtain the bottom layer target position, and determine the display area from the top layer image based on the bottom layer target position.

[0068] In a specific embodiment of the first method for determining the area to be displayed, the step of performing target detection on the underlying image to obtain the underlying target position, and determining the area to be displayed from the top image based on the underlying target position, includes: performing target detection on the underlying image to obtain an initial target position; filtering out underlying target positions that meet preset true target conditions from the initial target positions; and determining the area to be displayed from the top image based on the underlying target positions. As shown in the flowchart of the first specific method for determining the area to be displayed in Figure 2, firstly, target detection is performed on the underlying image to obtain the initial target position. Then, underlying target positions that meet preset true target conditions are filtered out from the initial target positions. In other words, the user can confirm whether the initial target position is indeed the underlying target position, avoiding the situation where the target corresponding to the detected initial target position is not the true target observed by the user. Therefore, underlying target positions that meet preset true target conditions are filtered out from the initial target positions. After determining the underlying target positions, the area to be displayed can be directly determined from the top image based on the underlying target positions.

[0069] Furthermore, the step of performing target detection based on the underlying image to obtain the initial target position includes: performing target detection on the underlying image using deep learning algorithms and / or traditional machine learning algorithms to obtain the initial target position; wherein, the traditional machine learning algorithms include connected component detection algorithms and moving object detection algorithms.

[0070] Object detection in the underlying image is performed using deep learning algorithms and / or traditional machine learning algorithms to obtain the initial object location. The detection model in the deep learning algorithm can be the common YOLO (You Only Look Once) series model. The YOLO series model is an industry-proven model that meets efficiency and effectiveness requirements. It can achieve good detection accuracy by training on a dedicated dataset. For example, in outdoor or birdwatching scenarios, data for that scenario can be collected and labeled to build a large-scale dataset for training the object detection model used in that scenario. The model can then be deployed to a device to detect objects. Alternatively, traditional machine learning algorithms can be used to perform object detection in the underlying image, such as connected component detection algorithms and moving object detection algorithms. By using AI (Artificial Intelligence) methods, the tedious process of searching for objects by humans can be avoided, making the entire process efficient and accurate. Ultimately, the initial object location in the underlying image can be obtained, specifically the anchor box.

[0071] In a first specific embodiment of filtering underlying target locations, filtering underlying target locations that satisfy preset true target conditions from the initial target locations includes: obtaining user-sent instruction information using a preset interface, and determining the location in the initial target locations corresponding to the instruction information as the underlying target location. Specifically, the user-sent instruction information may be a moving arrow in the display interface, and the initial target location indicated by this arrow is the underlying target location, meaning that the initial target location satisfies the preset true target conditions.

[0072] In a second specific embodiment of filtering underlying target locations, the step of filtering underlying target locations from the initial target locations that meet preset true target conditions includes: generating prompt information for the targets corresponding to the initial target locations in a preset form; obtaining a target selection instruction issued by the user based on the prompt information using a preset interface; and filtering underlying target locations from the initial target locations that meet the preset true target conditions based on the target selection instruction. Generating prompt information for the targets corresponding to the initial target locations in a preset form, such as a UI (User Interface) color change and sound announcement, serves as a reminder to the user. The location corresponding to the target selected by the user from the initial target locations is the target selection instruction issued by the user based on the prompt information. Therefore, underlying target locations that meet the preset true target conditions are filtered from the initial target locations based on the target selection instruction.

[0073] In the third specific embodiment of bottom-level target location filtering, the step of filtering bottom-level target locations that meet preset true target conditions from the initial target locations includes: determining each target corresponding to the initial target location and the gyroscope stabilization time and position information of each target; filtering bottom-level target locations that meet preset true target conditions from the initial target locations based on the relationship between the gyroscope stabilization time and a preset stabilization time threshold, and the position information. The filtering of bottom-level target locations that meet preset true target conditions is based on the user's gyroscope stabilization time and the target's position in the image. For example, for target 1, which is in the middle of the image, and the user's gyroscope stabilization time is 4 seconds; for target 2, which is in the upper left corner of the image, and the user's gyroscope stabilization time is 0.9 seconds, with a stabilization time threshold of 3 seconds; targets whose gyroscope stabilization time is greater than the preset stabilization time threshold and are located in the middle area of ​​the image are true targets, and their corresponding initial target locations are the bottom-level target locations. In other words, the initial target location of target 1 is the bottom-level target location.

[0074] In a second specific embodiment of determining the area to be displayed, determining the area to be displayed from the top-level image based on the bottom-level target position includes: performing target detection on the entire top-level image to obtain the top-level target position; and determining the area to be displayed from the top-level image based on the bottom-level target position and the top-level target position. As shown in the flowchart of the second specific method for determining the area to be displayed in Figure 3, due to the different characteristics of infrared and visible light sources, as well as issues related to structure, lens, and assembly processes, even if the original infrared image and the original visible light image are registered, it is impossible to achieve precise pixel-by-pixel registration. If the area to be displayed is directly determined from the original visible light image based on the bottom-level target position, the area to be displayed may not be a visible light target. Therefore, target detection needs to be performed on the top-level image again. In other words, after determining the bottom-level target position, target detection can be performed on the entire top-level image to obtain the top-level target position, and then the area to be displayed can be determined from the top-level image based on the bottom-level target position and the top-level target position, making the determined area to be displayed more accurate.

[0075] In the third specific embodiment of determining the display area, determining the display area from the top-level image based on the bottom-level target position includes: filtering the detection area from the top-level image based on the bottom-level target position, and performing target detection on the detection area in the top-level image to obtain the top-level target position; determining the display area from the top-level image based on the bottom-level target position and the top-level target position. As shown in the flowchart of the third specific display area determination in Figure 4, the detection area is filtered from the top-level image according to the bottom-level target position, that is, the position in the top-level image corresponding to the bottom-level target position and a certain area of ​​that position are all taken as the detection area. In other words, the bottom-level target position provides prior information for target detection in the top-level image. For example, in some camouflage scenes, the target is not obvious in visible light scenes, but the target is obvious in infrared scenes. The bottom-level target position can provide prior information for target detection in the top-level image and can also filter out irrelevant interference information, which is beneficial to improving detection accuracy.

[0076] In this embodiment, determining the area to be displayed from the top-level image based on the bottom target position and the top target position includes: determining an initial display area from the top-level image based on the top target position; aligning the bottom anchor frame corresponding to the bottom target position and the top anchor frame corresponding to the top target position, and recording the scaling information and offset information during the alignment process of the bottom anchor frame and the top anchor frame; and processing the initial display area according to the scaling information and the offset information to obtain the area to be displayed. By combining the target position information from infrared and visible light, alignment optimization is performed. The dual-light target position, i.e., the bottom anchor frame corresponding to the bottom target position and the top anchor frame corresponding to the top target position, can reflect the target's position and size. The target is scaled to the same size according to the size of the dual-light anchor frames, and the scaling information is recorded. Then, the offset information is recorded according to the coordinates of the top left corner of the bottom anchor frame and the top left corner of the top anchor frame. The initial display area is processed based on the scaling and offset information. The anchor frame of the area to be displayed can be x1.5, x2, etc. of the bottom anchor frame, and the resolution of the area to be displayed can be x1, x2, etc. of the resolution of the bottom target position part in the original infrared image.

[0077] Step S13: Display the area to be displayed in the top layer image on top of the bottom layer image in a picture-in-picture format.

[0078] In this embodiment, displaying the area to be displayed in the top layer image in a picture-in-picture format on top of the bottom layer image includes: determining the target display position of the area to be displayed in the top layer image in the bottom layer image; wherein, the target display position is the bottom target position in the bottom layer image or a position in the bottom layer image other than the bottom target position; and displaying the area to be displayed in the top layer image in a picture-in-picture format on top of the bottom layer image based on the target display position.

[0079] Visible light is advantageous for distinguishing detailed information about specific targets. For example, in birdwatching scenarios, after identifying a target using infrared light, a picture-in-picture view of the bird in visible light allows for easy observation of the bird's species, feathers, details, and other information. Figure 5 illustrates a specific picture-in-picture display, using the original infrared image as the bottom layer and the original visible light image as the top layer. The area to be displayed in the top layer image is shown as a picture-in-picture on top of the bottom layer image. In this picture-in-picture display, the visible light area is above the target position in the bottom layer of the original infrared image. This allows the user to simultaneously observe the target's state in both infrared and visible light. Figure 6 illustrates another specific picture-in-picture display, using the original infrared image as the bottom layer and the original visible light image as the top layer. When the area to be displayed in the visible light image is shown as a picture-in-picture on top of the original infrared image, the visible light area overlaps with the target position in the bottom layer of the original infrared image, covering the original infrared area. This allows the user to observe the target's state in visible light as well as the surrounding area in infrared light.

[0080] In this embodiment, displaying the area to be displayed in the top layer image on top of the bottom layer image in a picture-in-picture format includes: if there are multiple areas to be displayed in the top layer image, selecting a target area to be displayed from the areas to be displayed according to the user's display command, and displaying the target area to be displayed in the top layer image on top of the bottom layer image in a picture-in-picture format. It can be understood that if there are multiple areas to be displayed in the top layer image, the user can determine the target area to be displayed from all the areas to be displayed, or all the areas to be displayed can be determined as the target area to be displayed, and then display the target area to be displayed in the top layer image on top of the bottom layer image in a picture-in-picture format.

[0081] The beneficial effects of this application are as follows: This application acquires the current original infrared image and original visible light image in the target scene, and determines the bottom layer image and the top layer image from the original infrared image and the original visible light image, respectively; it performs target detection on the bottom layer image to obtain the bottom target position, and determines the area to be displayed from the top layer image based on the bottom target position; it then displays the area to be displayed in the top layer image in a picture-in-picture format on top of the bottom layer image. Therefore, this application performs target detection on the bottom layer image to obtain the bottom target position, determines the area to be displayed from the top layer image based on the bottom target position, and then displays the area to be displayed in the top layer image in a picture-in-picture format on top of the bottom layer image. This achieves automatic and intelligent dual-light image display without the need for manual selection of the area to be displayed, reducing the difficulty and time required for dual-light image display, and greatly improving the user experience.

[0082] For example, the first specific dual-light image display flowchart shown in Figure 7 is used as an example. Compared with the previous embodiment, this embodiment uses the original infrared image as the bottom layer image and the original visible light image as the top layer image for corresponding explanation.

[0083] Step S21: Obtain the current infrared raw image and visible light raw image in the target scene, determine the infrared raw image as the bottom layer image, and determine the visible light raw image as the top layer image.

[0084] Infrared imaging utilizes infrared thermal imaging technology to detect the infrared radiation of objects in the environment. Through signal processing and photoelectric conversion, it transforms the temperature distribution of these objects into a visible image. This makes infrared images unaffected by lighting conditions or weather, naturally suitable for target searching. Therefore, in nighttime or very low-light environments, infrared imaging can capture the thermal radiation of objects, while visible light imaging may not provide a clear image. Infrared sensors can provide sufficient information to locate potential targets in nighttime or low-light conditions. Thus, in nighttime or low-light scenarios, the raw infrared image can be used as the bottom layer, and the raw visible light image as the top layer. Furthermore, in target concealment detection scenarios, targets may deliberately hide in shadows or disguise themselves as part of the background to evade visible light detection. Infrared imaging can penetrate these disguises because it is based on the object's thermal radiation rather than reflected light. Therefore, the infrared image, as the bottom layer, can help quickly locate these concealed targets, while the visible light image is used for further confirmation and identification.

[0085] Step S22: Perform target detection on the infrared raw image to obtain the position of the underlying target, and determine the area to be displayed from the visible light raw image based on the position of the underlying target.

[0086] When both the infrared raw image and the visible light raw image are used, after determining the location of the underlying target from the infrared raw image, the area to be displayed is directly determined from the visible light raw image based on the location of the underlying target. It can be understood that when both the visible light raw image and the infrared raw image are used, the area to be displayed can be determined in the following way:

[0087] The first method involves target detection based on the raw infrared image to obtain initial target positions. From these initial positions, lower-level target positions that meet preset true target criteria are selected. Then, based on these lower-level target positions, the area to be displayed is determined from the raw visible light image. In other words, after determining the lower-level target positions from the raw visible light image using the target detection method, the area to be displayed is directly determined from the raw infrared image based on these lower-level target positions. To improve scene adaptability, a large amount of data specific to outdoor and birdwatching scenarios can be collected as training data for visible light target detection. This training data is then used to improve the accuracy of target detection.

[0088] The second approach is to consider that infrared and visible light are light sources with different characteristics, as well as issues related to structure, lens, and assembly processes. After determining the position of the bottom target, target detection is performed on the entire original visible light image to obtain the position of the top target. Then, based on the bottom target position and the top target position, the area to be displayed is determined from the original visible light image.

[0089] The third method: The bottom target position provides prior information for target detection in the visible light original image. Therefore, the region to be detected can be screened from the visible light original image based on the bottom target position, and target detection can be performed on the region to be detected in the visible light original image to obtain the top target position. Then, the region to be displayed can be determined from the visible light original image based on the bottom target position and the top target position, which is conducive to improving the detection accuracy.

[0090] Step S23: Display the area to be displayed in the original visible light image on top of the original infrared image in a picture-in-picture format.

[0091] It is understandable that when using the infrared original image and the visible light original image as the infrared original image, after determining the area to be displayed in the visible light original image, the area to be displayed in the visible light original image is displayed on top of the infrared original image in a picture-in-picture format. This display format can be simultaneous overlay or staggered display. That is to say, when the area to be displayed in the visible light original image is displayed on top of the infrared original image, the display position of the area to be displayed can be the position of the underlying target or not. For example, if target A is to be displayed simultaneously in a dual-light format, then the area to be displayed of target A in the visible light original image will be displayed at the position of target A in the infrared original image. Or, if target A is to be displayed staggered in a dual-light format, then the area to be displayed of target A in the visible light original image will be displayed at a position other than target A in the infrared original image. Specifically, it can be displayed at the position of target B in the infrared original image or at a position without a target.

[0092] For example, the second specific dual-light image display flowchart shown in Figure 8 is used as an example. Compared with the previous embodiment, this embodiment uses the original visible light image as the bottom layer image and the original infrared image as the top layer image for corresponding explanation.

[0093] Step S31: Obtain the current infrared raw image and visible light raw image in the target scene, determine the visible light raw image as the bottom layer image, and determine the infrared raw image as the top layer image.

[0094] Visible light images are based on light reflection imaging and can reveal rich geometric and textural details. Under high light conditions or in daytime scenes, such as in well-lit environments, visible light imaging can provide clear images and rich color information. In such cases, visible light images, as the bottom layer, can provide more intuitive contextual information, helping observers understand the environment in which the target is located. Infrared images can be used as the top layer to highlight targets that may be distinguishable from other objects due to temperature differences. Furthermore, in specific target feature detection scenarios, targets may have specific infrared features, such as high-temperature areas. These features may not be obvious or difficult to identify in visible light images. By using infrared images as the top layer, these features can be highlighted, making it easier to identify the target. For example, in fire detection, infrared images can show the high-temperature areas of the fire source, while visible light images are used to provide an overall view of the fire scene.

[0095] Step S32: Perform target detection on the raw visible light image to obtain the position of the underlying target, and determine the area to be displayed from the raw infrared image based on the position of the underlying target.

[0096] Target detection is performed on the raw visible light image to obtain the location of the underlying target, and the area to be displayed is determined from the raw infrared image based on the location of the underlying target. Specifically, the underlying target locations that meet the preset true target conditions can be selected from the initial target locations, and the area to be displayed can be determined from the raw infrared image based on the location of the underlying target; or, target detection is performed on the entire raw infrared image to obtain the location of the top-level target, and the area to be displayed can be determined from the raw infrared image based on the location of the underlying target; or, the area to be detected can be selected from the raw infrared image based on the location of the underlying target, and target detection is performed on the area to be detected in the raw infrared image to obtain the location of the top-level target, and then the area to be displayed can be determined from the raw infrared image based on the location of the underlying target and the location of the top-level target.

[0097] Furthermore, the specific process of determining the display area from the infrared raw image based on the bottom target position and the top target position is as follows: According to the top target position, a preliminary display area, i.e., the initial display area, is first determined in the infrared raw image. Then, the bottom anchor frame associated with the bottom target position in the visible light raw image is aligned with the top anchor frame corresponding to the top target position in the infrared raw image. During the alignment process, the scaling changes and offset of the bottom anchor frame relative to the top anchor frame are recorded in detail, i.e., scaling information and offset information. Finally, the previously determined initial display area is adjusted using these recorded scaling information and offset information to obtain the final display area.

[0098] Step S33: Display the area to be displayed in the original infrared image on top of the original visible light image in a picture-in-picture format.

[0099] If the raw infrared image contains multiple areas to be displayed, the user-specified target area can be selected from these areas according to the display instructions given by the user. Then, the selected target area is displayed on top of the raw visible light image in a picture-in-picture manner.

[0100] Referring to Figure 9, this application discloses a dual-light image display device, comprising:

[0101] Image acquisition module 11 is used to acquire the current infrared raw image and visible light raw image in the target scene, and to determine the bottom layer image and the top layer image from the infrared raw image and the visible light raw image, respectively;

[0102] The region determination module 12 is used to perform target detection on the bottom layer image to obtain the position of the bottom layer target, and determine the region to be displayed from the top layer image based on the position of the bottom layer target.

[0103] The area display module 13 is used to display the area to be displayed in the top layer image on the top layer image in a picture-in-picture format.

[0104] The beneficial effects of this application are as follows: This application acquires the current original infrared image and original visible light image in the target scene, and determines the bottom layer image and the top layer image from the original infrared image and the original visible light image, respectively; it performs target detection on the bottom layer image to obtain the bottom target position, and determines the area to be displayed from the top layer image based on the bottom target position; it then displays the area to be displayed in the top layer image in a picture-in-picture format on top of the bottom layer image. Therefore, this application performs target detection on the bottom layer image to obtain the bottom target position, determines the area to be displayed from the top layer image based on the bottom target position, and then displays the area to be displayed in the top layer image in a picture-in-picture format on top of the bottom layer image. This achieves automatic and intelligent dual-light image display without the need for manual selection of the area to be displayed, reducing the difficulty and time required for dual-light image display, and greatly improving the user experience.

[0105] Furthermore, embodiments of this application also provide an electronic device. Figure 10 is a structural diagram of an electronic device 20 according to an exemplary embodiment; the content in the figure should not be construed as limiting the scope of this application.

[0106] Figure 10 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Specifically, it may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the dual-light image display method performed by the electronic device disclosed in any of the foregoing embodiments.

[0107] In this embodiment, the power supply 23 is used to provide operating voltage for various hardware devices on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0108] The processor 21 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 21 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0109] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored on it include operating system 221, computer program 222 and data 223, etc., and the storage method can be temporary storage or permanent storage.

[0110] The operating system 221 manages and controls the various hardware devices and computer programs 222 on the electronic device to enable the processor 21 to perform calculations and processing on the massive amounts of data 223 in the memory 22. The operating system can be Windows, Unix, Linux, etc. The computer program 222, in addition to including a computer program capable of performing the dual-light image display method executed by the electronic device as disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks. The data 223 may include data received by the electronic device from external devices, as well as data collected by its own input / output interface 25.

[0111] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned dual-light image display method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0112] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0113] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly in hardware, software modules executed by a processor, or a combination of both. The software module may be located in random access memory (RAM), memory, read-only memory (ROM), electrically programmable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), register, hard disk, removable disk, CD-ROM (Compact Disc Read-Only Memory), or any other form of storage medium known in the art.

[0114] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0115] The present invention provides a detailed description of a dual-light image display method, apparatus, device, and medium. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A dual-light image display method, characterized in that, include: Acquire the current raw infrared image and raw visible light image in the target scene, and determine the bottom layer image and top layer image from the raw infrared image and the raw visible light image, respectively; Target detection is performed on the bottom layer image to obtain the location of the bottom layer target, and the area to be displayed is determined from the top layer image based on the location of the bottom layer target; The area to be displayed in the top layer image is displayed on top of the bottom layer image in a picture-in-picture format.

2. The dual-light image display method according to claim 1, characterized in that, The process of acquiring the current raw infrared image and raw visible light image in the target scene also includes: The infrared original image and the visible light original image are registered to align them.

3. The dual-light image display method according to claim 1, characterized in that, The step of performing target detection on the bottom layer image to obtain the position of the bottom layer target, and determining the area to be displayed from the top layer image based on the bottom layer target position, includes: Target detection is performed based on the underlying image to obtain the initial target location; Select the underlying target locations that meet the preset true target conditions from the initial target locations; The area to be displayed is determined from the top-level image based on the location of the underlying target.

4. The dual-light image display method according to claim 3, characterized in that, The step of performing target detection based on the underlying image to obtain the initial target location includes: The underlying image is subjected to target detection using deep learning algorithms and / or traditional machine learning algorithms to obtain the initial target position; wherein the traditional machine learning algorithms include connected component detection algorithms and moving object detection algorithms.

5. The dual-light image display method according to claim 3, characterized in that, The step of filtering out the underlying target locations that meet the preset true target conditions from the initial target locations includes: The system uses a preset interface to obtain the instruction information issued by the user, and determines the position corresponding to the instruction information in the initial target position as the underlying target position.

6. The dual-light image display method according to claim 3, characterized in that, The step of filtering out the underlying target locations that meet the preset true target conditions from the initial target locations includes: Generate prompt information for the target corresponding to the initial target location in a preset format; The system uses a preset interface to obtain the target selection command issued by the user based on the prompt information, and filters out the underlying target positions that meet the preset true target conditions from the initial target positions based on the target selection command.

7. The dual-light image display method according to claim 3, characterized in that, The step of filtering out the underlying target locations that meet the preset true target conditions from the initial target locations includes: Determine each target corresponding to the initial target position, as well as the gyroscope stabilization time and position information of each target; Based on the relationship between the gyroscope stabilization time and the preset stabilization time threshold, the position information is used to filter out the underlying target positions that meet the preset true target conditions from the initial target positions.

8. The dual-light image display method according to claim 1, characterized in that, Determining the area to be displayed from the top-level image based on the bottom-level target location includes: Target detection is performed on the entire top-level image to obtain the location of the top-level target; The area to be displayed is determined from the top-level image based on the bottom-level target position and the top-level target position.

9. The dual-light image display method according to claim 1, characterized in that, Determining the area to be displayed from the top-level image based on the bottom-level target location includes: Based on the bottom target location, a region to be detected is selected from the top image, and target detection is performed on the region to be detected in the top image to obtain the top target location; The area to be displayed is determined from the top-level image based on the bottom-level target position and the top-level target position.

10. The dual-light image display method according to claim 8 or 9, characterized in that, The step of determining the area to be displayed from the top-level image based on the bottom-level target position and the top-level target position includes: The initial display area is determined from the top-level image based on the top-level target position; Align the bottom anchor frame corresponding to the bottom target position and the top anchor frame corresponding to the top target position, and record the scaling and offset information during the alignment process of the bottom anchor frame and the top anchor frame; The initial display area is processed according to the scaling information and the offset information to obtain the area to be displayed.

11. The dual-light image display method according to any one of claims 1 to 10, characterized in that, The step of displaying the area to be displayed in the top layer image on top of the bottom layer image in a picture-in-picture format includes: If there are multiple areas to be displayed in the top-level image, the target area to be displayed is selected from the areas to be displayed according to the display command given by the user, and the target area to be displayed in the top-level image is displayed on the top layer of the bottom layer image in a picture-in-picture format.

12. The dual-light image display method according to claim 1, characterized in that, The step of displaying the area to be displayed in the top layer image on top of the bottom layer image in a picture-in-picture format includes: The target display position of the area to be displayed in the top layer image is determined in the bottom layer image; wherein, the target display position is the bottom layer target position in the bottom layer image or a position in the bottom layer image other than the bottom layer target position; Based on the target display position, the area to be displayed in the top layer image is displayed on top of the bottom layer image in a picture-in-picture format.

13. A dual-light image display device, characterized in that, include: The image acquisition module is used to acquire the current infrared raw image and visible light raw image in the target scene, and to determine the bottom layer image and the top layer image from the infrared raw image and the visible light raw image, respectively. The region determination module is used to perform target detection on the bottom layer image to obtain the position of the bottom layer target, and determine the region to be displayed from the top layer image based on the position of the bottom layer target; The area display module is used to display the area to be displayed in the top layer image on top of the bottom layer image in a picture-in-picture format.

14. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the dual-light image display method as described in any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that, Used to store a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the dual-light image display method as described in any one of claims 1 to 12.

Images