Image display processing method, device and electronic equipment

Abstract

The application discloses a display processing method of an image, comprising: acquiring a thermal imaging image and a visible light image obtained by shooting the same scene; detecting an operation event for a first image currently displayed in an image presentation interface; determining a first target region selected in the first image based on position information of the detected operation event; searching for a second target region corresponding to the same position as the first target region in a second image based on position calibration information between the first image and the second image; and displaying an image in the second target region in the second image. According to the application, the interested part can be conveniently switched and displayed among the thermal imaging image, the visible light image and the dual-light fusion image, and the experience of a user is improved.

Description

Technical Field

[0001] This application relates to thermal imaging technology, and more particularly to an image display processing method, apparatus, and electronic device. Background Technology

[0002] With the continuous development and progress of infrared imaging technology, methods and systems for generating and processing thermal images using infrared imaging technology have been increasingly widely applied.

[0003] Infrared imaging technology works by detecting the radiation energy of an object and processing it into a thermal image of the target object. The temperature distribution of the target can be obtained through the thermal imaging spectrum.

[0004] When users take photos using thermal imaging cameras, the camera often generates thermal images, visible light images, and dual-light fused images for the same scene. Dual-light fused images are simply thermal and visible light images superimposed together for display. To view image information, users can switch between modes: visible light mode displays the visible light image; thermal imaging mode displays the thermal image; and dual-light fused mode displays the dual-light fused image.

[0005] The aforementioned method of switching between visible light images, thermal images, and dual-light fusion images is inconvenient for intuitive information viewing. For example, if an abnormal temperature point is found in the thermal image, and the user wants to view the corresponding visible light image to identify the object, the current solution requires switching to the visible light image and manually searching for the corresponding point on the visible light image. This search is prone to errors, as the locations may not match. If multiple such phenomena exist in a single thermal image, the switching becomes even more cumbersome, making it difficult to accurately locate the corresponding position and resulting in a poor user experience. While dual-light fusion images can display two images simultaneously, the superimposed display makes it difficult to clearly distinguish between the visible light and thermal imaging portions. Summary of the Invention

[0006] This application provides an image display processing method, apparatus, and electronic device that can conveniently switch between displaying the part of interest in thermal imaging images, visible light images, and dual-light fusion images, thereby improving the user experience.

[0007] To achieve the above objectives, this application adopts the following technical solution:

[0008] An image display processing method, comprising:

[0009] Acquire thermal and visible light images of the same scene.

[0010] Detect operation events on the currently displayed first image in the image presentation interface; wherein the first image is the thermal imaging image, the visible light image, or a dual-light fusion image generated by superimposing the thermal imaging image and the visible light image;

[0011] Based on the location information of the detected operation event, a first target region is determined in the first image;

[0012] Based on the positional calibration information between the first image and the second image, a second target region corresponding to the same position as the first target region is found in the second image; wherein, the second image is another image other than the first image among the thermal imaging image, the visible light image and the dual-light fusion image, and the first positional calibration information includes auxiliary information to ensure that the pixels corresponding to the same position in the first image and the second image coincide.

[0013] Display the image within the second target region of the second image.

[0014] Preferably, after determining the first target region, the method further includes:

[0015] Based on the second positional calibration information between the first image and the third image, a third target region corresponding to the same position as the first target region is found in the third image; wherein, the third image is another image other than the first image and the second image among the thermal imaging image, the visible light image and the dual-light fusion image, and the second positional calibration information includes auxiliary information to ensure that the pixels corresponding to the same position in the first image and the third image are coincident;

[0016] Display the image within the third target region of the third image.

[0017] Preferably, the first position calibration information includes the coordinates of the first reference point, the image size of the image where the first reference point is located, and the relative angle between the first image and the second image when the corresponding pixels in the thermal imaging image, the first image, and the second image are guaranteed to overlap.

[0018] Wherein, the coordinates of the first reference point are the pixel coordinates of the corresponding position of the first reference point on the other image, which is either the first image or the second image;

[0019] The second position calibration information includes the coordinates of the second reference point, the image size of the image where the second reference point is located, and the relative angle between the first image and the third image when the pixels at the same position in the first image and the third image are guaranteed to overlap;

[0020] Wherein, the second reference point coordinates are the pixel coordinates of the corresponding position of the reference point on the other image, which is the reference point on either the first image or the third image.

[0021] Preferably, the operation event includes a click event;

[0022] The step of determining the selected first target region in the first image based on the location information of the detected operation event includes:

[0023] The first target region is determined based on the click positions of at least three click events; wherein the boundary of the first target region passes through the click positions of at least three click events.

[0024] or,

[0025] Using the click location of the click event as the center of the preset graphic, the coverage area of ​​the preset graphic is determined as the first target area.

[0026] Preferably, the operation event includes a line drawing event;

[0027] The step of determining the selected first target region in the first image based on the location information of the detected operation event includes:

[0028] Based on the trajectory information of the line drawing event, the first target area is determined;

[0029] The boundary of the first target area coincides with the trajectory information.

[0030] Preferably, after detecting a preset end operation, the process of determining the selected first target region in the first image is performed.

[0031] Preferably, the step of finding a second target region in the second image corresponding to the location of the first target region based on the first location calibration information includes:

[0032] Based on the first location calibration information, the corresponding pixel points in the second image of each pixel point in the first target region are determined to form the second target region;

[0033] The step of finding a third target region in the third image corresponding to the location of the first target region based on the second location calibration information includes:

[0034] Based on the second location calibration information, the corresponding pixel points of each pixel point in the first target region are determined in the third image to form the third target region.

[0035] Preferably, displaying the image within the second target region of the second image includes:

[0036] Display the image within the second target area at the location of the first target area; or...

[0037] The image within the second target area is displayed at a preset position on the image presentation interface; or,

[0038] The image within the second target area is displayed in a separate image presentation interface, distinct from the aforementioned image presentation interface.

[0039] The display of the image within the third target region in the third image includes:

[0040] Display the image within the third target area at the location of the first target area; or...

[0041] The image within the third target area is displayed at a preset position on the image presentation interface; or,

[0042] The image within the third target area is displayed in a separate image presentation interface, distinct from the image presentation interface described above.

[0043] Preferably, displaying the image within the second target area at the location of the first target area includes:

[0044] A new layer is added at the location of the first target area to display the image in the second target area; or, at the location of the first target area, the values ​​of the corresponding pixels in the first target area are replaced with the values ​​of the pixels in the second target area.

[0045] Displaying the image within the third target area at the location of the first target area includes:

[0046] A new layer is added at the location of the first target area to display the image within the third target area; or, at the location of the first target area, the values ​​of the corresponding pixels in the first target area are replaced with the values ​​of the pixels in the third target area.

[0047] Preferably, the other graphical interface is created via a pop-up window.

[0048] An image display processing apparatus includes: an acquisition unit, an operation event detection unit, a target region delineation unit, and a display unit;

[0049] The acquisition unit is used to acquire thermal imaging images and visible light images obtained by shooting the same scene;

[0050] The operation event detection unit is used to detect operation events on the currently displayed first image in the image presentation interface; wherein the first image is the thermal imaging image, the visible light image, or a dual-light fusion image generated by superimposing the thermal imaging image and the visible light image;

[0051] The target region delineation unit is configured to determine a selected first target region in the first image based on the location information of the detected operation event; and to search for a second target region in the second image corresponding to the location of the first target region based on the first positional calibration information between the first image and the second image; wherein the second image is another image besides the first image among the thermal imaging image, the visible light image, and the dual-light fusion image, and the first positional calibration information includes auxiliary information that ensures that pixels at the same position in the first image and the second image overlap;

[0052] The display unit is used to display the image within the second target area of ​​the second image.

[0053] Preferably, after determining the first target region, the target region delineation unit is further configured to, based on the first position calibration information including auxiliary information position calibration information that ensures the overlap of pixels at corresponding positions in the first image and the second image, search for a third target region in the third image that corresponds to the first target region; wherein, the third image is another image besides the first image and the second image among the thermal imaging image, the visible light image, and the dual-light fusion image, and the second position calibration information includes auxiliary information that ensures the overlap of pixels at corresponding positions in the first image and the third image;

[0054] The display unit is further configured to display an image within the third target area of ​​the third image.

[0055] Preferably, the first position calibration information includes the coordinates of the first reference point, the image size of the image where the first reference point is located, and the relative angle between the first image and the second image when the corresponding pixels at the same position in the first image and the second image are guaranteed to overlap.

[0056] Wherein, the coordinates of the first reference point are the pixel coordinates of the corresponding position of the reference point on the other image, which is either the first image or the second image;

[0057] The second position calibration information includes the coordinates of the second reference point, the image size of the image where the second reference point is located, and the relative angle between the first image and the third image when the pixels at the same position in the first image and the third image are guaranteed to overlap;

[0058] Wherein, the second reference point coordinates are the pixel coordinates of the corresponding position of the reference point on the other image, which is the reference point on either the first image or the third image.

[0059] Preferably, the operation event includes a click event;

[0060] In the operation event detection unit, based on the location information of the detected operation event, the selected first target region in the first image is determined, including:

[0061] The first target region is determined based on the click positions of at least three click events; wherein the boundary of the first target region passes through the click positions of at least three click events.

[0062] or,

[0063] Using the click location of the click event as the center of the preset graphic, the coverage area of ​​the preset graphic is determined as the first target area.

[0064] Better place,

[0065] The operation events include line drawing events;

[0066] In the operation event detection unit, determining the selected first target region in the first image based on the location information of the detected operation event includes:

[0067] Based on the trajectory information of the line drawing event, the first target area is determined;

[0068] The boundary of the first target area coincides with the trajectory information.

[0069] In the operation event detection unit, after a preset end operation is detected, the process of determining the selected first target region in the first image is executed.

[0070] Preferably, in the target region delineation unit, the step of searching for a second target region in the second image corresponding to the location of the first target region based on the first location calibration information includes:

[0071] Based on the first location calibration information, the corresponding pixel points in the second image of each pixel point in the first target region are determined to form the second target region;

[0072] In the target region delineation unit, the step of searching for a third target region in the third image corresponding to the location of the first target region based on the second location calibration information includes:

[0073] Based on the second location calibration information, the corresponding pixel points of each pixel point in the first target region are determined in the third image to form the third target region.

[0074] Preferably, in the display unit, displaying the image within the second target area of ​​the second image includes:

[0075] Display the image within the second target area at the location of the first target area; or...

[0076] The image within the second target area is displayed at a preset position on the image presentation interface; or,

[0077] The image within the second target area is displayed in a separate image presentation interface, distinct from the aforementioned image presentation interface.

[0078] The step of displaying the image within the third target area of ​​the third image in the display unit includes:

[0079] Display the image within the third target area at the location of the first target area; or...

[0080] The image within the third target area is displayed at a preset position on the image presentation interface; or,

[0081] The image within the third target area is displayed in a separate image presentation interface, distinct from the image presentation interface described above.

[0082] Preferably, in the display unit, displaying the image within the second target area at the location of the first target area includes:

[0083] Add a new layer at the location of the first target area to display the image within the second target area; or,

[0084] At the location of the first target region, the values ​​of the corresponding pixels in the first target region are replaced by the values ​​of the pixels in the second target region.

[0085] In the display unit, displaying the image within the third target area at the location of the first target area includes:

[0086] A new layer is added at the location of the first target area to display the image within the third target area; or, at the location of the first target area, the values ​​of the corresponding pixels in the first target area are replaced with the values ​​of the pixels in the third target area.

[0087] Preferably, in the display unit, the other image interface is created via a pop-up window.

[0088] This application also provides an electronic device, which includes at least a computer-readable storage medium and a processor;

[0089] The processor is configured to read the executable instructions from the computer-readable storage medium and execute the instructions to implement the image display method described in any of the preceding claims.

[0090] As can be seen from the above technical solution, this application acquires a thermal imaging image and a visible light image obtained by capturing the same scene; detects operation events on the currently displayed first image in the image display interface; wherein the first image is a thermal imaging image, a visible light image, or a dual-light fusion image generated by superimposing a thermal imaging image and a visible light image; based on the position information of the detected operation event, determines a selected first target area in the first image; based on the first position calibration information, searches for a second target area in the second image corresponding to the position of the first target area; wherein the second image is another image besides the first image among the thermal imaging image, the visible light image, and the dual-light fusion image; and displays the image within the second target area in the second image. Through this method, on the one hand, position calibration information is used to mark auxiliary information to ensure overlap between corresponding positions in the first and second images; on the other hand, the detection of operation events conveniently determines the first target area of ​​interest in the first image, and then, based on the calibration information, searches for a second target area in the second image corresponding to the position of the first target area. In this way, the location area corresponding to the part of interest is automatically found on another image and displayed, improving the user experience. Attached Figure Description

[0091] Figure 1 This is a schematic diagram illustrating the basic flow of the image display method in this application;

[0092] Figure 2 This is a schematic diagram illustrating the specific process of the image display method in the embodiments of this application;

[0093] Figure 3 Example diagram of location tagging information;

[0094] Figure 4 A schematic diagram illustrating the activation of the image fusion function;

[0095] Figure 5 This is an example of selecting a first target region in an embodiment of this application. Figure 1 ;

[0096] Figure 6 This is an example of selecting a first target region in an embodiment of this application. Figure 2 ;

[0097] Figure 7 This is an example of selecting a first target region in an embodiment of this application. Figure 3 ;

[0098] Figure 8 This is a schematic diagram of the basic structure of the image display device in this application;

[0099] Figure 9 This is a schematic diagram of the basic structure of the electronic device in this application. Detailed Implementation

[0100] To make the objectives, technical means, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings.

[0101] In this application, in any one of thermal imaging images, visible light images, and dual-light fusion images, a first target region on the displayed image is determined by detecting operation events. Then, based on position calibration information, a second target region on another image corresponding to the position of the first target region is determined and displayed. In this way, it is possible to conveniently switch between displaying the part of interest in thermal imaging images, visible light images, and dual-light fusion images, thereby improving the user experience.

[0102] Figure 1 This is a schematic diagram illustrating the basic flow of the image display method in this application. Figure 1 As shown, the method includes:

[0103] Step 101: Acquire thermal imaging images and visible light images of the same scene.

[0104] Thermal imaging cameras can simultaneously generate both thermal and visible light images after capturing images of the same scene. This type of capture typically involves two cameras positioned close to each other, each producing a separate thermal and visible light image. Due to differences in resolution and position between the two cameras, the resulting thermal and visible light images will differ in size and subtle angles. Based on this, initial positional calibration information can be obtained between the thermal and visible light images. This calibration information can include auxiliary information to ensure that corresponding positions in the thermal and visible light images coincide. Using this initial calibration information, angle and position corrections can be made to ensure that pixels at the same position in the thermal and visible light images are aligned. This initial positional calibration information is usually fixed in the camera's information settings at the factory.

[0105] Furthermore, by overlaying the acquired thermal imaging image and visible light image, a two-light fused image can be generated. When overlaying the thermal and visible light images, corresponding pixels at the same location need to be aligned, which can be achieved through initial position calibration information. Since the generated two-light fused image is produced by overlaying the thermal and visible light images, initial position calibration information can also be used to ensure that corresponding pixels in the two-light fused image align with both the thermal and visible light images.

[0106] Step 102: Detect operation events on the currently displayed first image in the image presentation interface.

[0107] In this application, a thermal imaging image can be displayed first, and then a region of interest can be selected on the thermal imaging image to display a visible light image or a dual-light fused image of the corresponding region. Alternatively, a visible light image can be displayed first, and then a region of interest can be selected on the visible light image to display a thermal imaging image or a dual-light fused image of the corresponding region. Or, a dual-light fused image can be displayed first, and then a region of interest can be selected on the dual-light fused image to display a thermal imaging image or a visible light image of the corresponding region. For ease of description, in the thermal imaging image, visible light image, and dual-light fused image, the image first displayed on the image presentation interface is referred to as the first image, and the target image to be displayed is referred to as the second image.

[0108] For example, when focusing on the state of an object under abnormal temperatures, a thermal image can be used as the first image and a visible light image as the second image; when focusing on the temperature information of a specific object, a visible light image can be used as the first image and a thermal image as the second image. In practice, options can be provided in the interface for users to select the first and second images, or the system's default first and second images can be used.

[0109] For the first image displayed, detect operation events related to that first image. These operation events can be user-triggered events used to select a region of interest.

[0110] Step 103: Based on the location information of the detected operation event, determine the selected first target region in the first image.

[0111] Each operation event corresponds to specific location information. Based on this location information, a region marked by the location information can be determined. Since the operation event is performed on the first image, mapping the region marked by the location information onto the first image yields a region in the first image, called the first target region. This first target region is the region selected by the operation event. For example, a selection operation can be performed on the first image (e.g., a thermal imaging image) using a device such as a mouse. After detecting the selection operation, the selected first target region is determined on the first image based on the position of the selection operation.

[0112] Step 104: Based on the first location calibration information, find the second target region in the second image that corresponds to the location of the first target region.

[0113] As described in step 101, by using the initial position calibration information, the pixels corresponding to the same position in the thermal imaging image, visible light image, and dual-light fusion image can be ensured to overlap through angle and position correction. In this application, referring to the meaning of the initial position calibration information, position calibration information between the first image and the second image is introduced, referred to as the first position calibration information, to ensure that pixels corresponding to the same position in the first image and the second image overlap. Specifically, the first position calibration information can be obtained from the initial position calibration information, or it can be obtained in advance by comparing the first image and the second image of the same scene. For example, the first image (e.g., a thermal imaging image) and the second image (e.g., a visible light image or a dual-light fusion image) taken in the same scene can be manually dragged in advance to make the pixels corresponding to the same position in the two images overlap, and the position calibration information between the two images after dragging is recorded as the first position calibration information.

[0114] Based on this first positional calibration information, for each pixel in the first target region, pixels corresponding to the same positions are identified in the second image to form the second target region. This ensures that the pixels in the second target region correspond to the same positions as the pixels in the first target region, achieving overlap.

[0115] Step 105: Display the image within the second target region of the second image.

[0116] Once the second target area is determined, the image within that area can be displayed and provided to the user.

[0117] Through the above steps, on the one hand, the first target region of interest in the first image is determined by detecting the operation event, thus conveniently identifying the part of interest; on the other hand, based on the positional calibration information, a second target region corresponding to the first target region is automatically found in the second image, thus automatically and accurately determining the image region in the other image corresponding to the part of interest. In this way, the part of interest can be conveniently switched between thermal imaging images and visible light images.

[0118] At this point, Figure 1 The basic method flow shown above has ended. Figure 1 Based on the process shown, after step 103, the following may be further included:

[0119] Step 104a: Based on the second position calibration information, find the third target region in the third image that corresponds to the same position as the first target region;

[0120] The third image is another image besides the first and second images among the thermal imaging image, visible light image, and dual-light fusion image. The second positional calibration information includes auxiliary information that ensures the overlap of pixels at corresponding positions in the first and third images. Specifically, the second positional calibration information is the positional calibration information between the first and third images, and its acquisition method can be the same as the aforementioned first positional calibration information, which will not be repeated here. Thus, based on the second positional calibration information, a third target region corresponding to the same position as the first target region can be determined.

[0121] Step 105a: Display the image within the third target region of the third image.

[0122] The processing in steps 104a and 105a can be the same as in steps 104 and 105, respectively, except that the processing is performed on the third image. Through the above processing, the corresponding region of the other image besides the first and second images is searched and displayed. That is, the thermal image, the visible light image, and the dual-light fusion image are all displayed. One of the images is displayed as the original image. After selecting the first target region on this image, the corresponding regions on the other two images are also displayed.

[0123] The specific implementation of the above image display method will be described below through specific embodiments.

[0124] Figure 2 This is a schematic flowchart illustrating the image display method in an embodiment of this application. In this embodiment, a thermal imaging image is used as the first image, and a visible light image is used as the second image for explanation. Figure 2 As shown, the specific method and process include:

[0125] Step 201: Obtain thermal imaging images, visible light images, and first position calibration information obtained by shooting the same scene.

[0126] Thermal and visible light images are obtained by capturing the same scene. Specifically, the pixel values ​​of each pixel in the acquired thermal and visible light images can be stored in a storage unit. Generally, the first position calibration information between the thermal and visible light images can be directly generated based on the initial position calibration information recorded by the camera. The acquired first position calibration information can also be stored in the storage unit and can be associated with the thermal and visible light images. The first position calibration information can be, for example, an image registration relationship or mapping relationship.

[0127] Optionally, the first position calibration information may include the coordinates of a first reference point, the length and width of the image containing the reference point, and the relative angle between the thermal imaging image and the visible light image to ensure that corresponding positions in the thermal imaging image and the visible light image coincide. The coordinates of the first reference point are the coordinates of the pixel at the same position on the other image (hereinafter referred to as the first auxiliary reference point) corresponding to the reference point on either the thermal imaging image or the visible light image. Typically, a boundary pixel in the smaller image of the thermal imaging image or a preset pixel can be used as the first reference point.

[0128] For example, since thermal imaging images are typically smaller than visible light images, pixel A at the top left corner of the thermal imaging image can be used as the first reference point. The coordinates of this first reference point are then the coordinates of pixel B in the visible light image, which corresponds to pixel A at the same position. Based on this, the first position calibration information can include the coordinates of pixel B, the length and width of the thermal imaging image, and the angle between the thermal imaging image and the visible light image when corresponding pixels at the same position overlap. Figure 3 As shown.

[0129] Step 202: Display the thermal imaging image on the image presentation interface.

[0130] Rendering thermal images on the image rendering interface can be done using existing methods, which will not be elaborated here.

[0131] Step 203: Activate the image fusion display function.

[0132] By activating the preset fusion display function, the processing of thermal imaging images and visible light images in this embodiment is triggered. Specifically, the fusion display function can be activated by setting an option (such as a button) on or outside the image presentation interface, and activating the fusion display function when the selection of the corresponding option is detected; or, the fusion display function can be activated by default when the image presentation interface is opened.

[0133] Below is an example of activating the fusion function via a button. Figure 4 As shown, a "Fusion Magic Mirror" button is provided on the image presentation interface. When the display processing of this application is required, the user can click the "Fusion Magic Mirror" button. After the system detects that the button has been pressed, it enters the display processing of this application and begins to execute subsequent steps.

[0134] The processing order of steps 202 and 203 above can also be reversed, that is, step 203 is executed first, and then step 202 is executed.

[0135] Step 204: Detect operation events on the thermal imaging image in the image presentation interface.

[0136] Operation events can be triggered by the user or automatically by the system. Operation events can include click events or line drawing events, for example, operation events input via devices such as a mouse or stylus.

[0137] Step 205: Based on the location information of the detected operation event, determine the selected first target area in the thermal imaging image.

[0138] Each operational event corresponds to specific location information. This location information allows us to determine the area marked by the operational event, which is then used as the first target region in the thermal imaging image. The first target region is determined using appropriate methods depending on the type of operational event.

[0139] Specifically, when the operation event includes a click event, the target area can be determined based on the click positions of at least three corresponding click events. For example, after detecting each click event, the click position of each click event is determined, and the click positions are connected sequentially to form a closed area. The portion within this closed area is then used as the target area. Figure 5 As shown.

[0140] When the operation event includes a click event, the click location can be used as the center of a preset graphic to determine the area covered by the preset graphic as the first target area. For example, if the preset graphic is a circle with a radius of 5 centimeters, after determining the click location, a circular area with a radius of 5 centimeters is determined with that click location as the center as the first target area. Figure 6As shown.

[0141] When the operation event includes a line-drawing event, the target region can be determined based on the trajectory information of the line-drawing event; wherein the boundary of the target region coincides with the trajectory information. For example, after detecting a line-drawing event, a closed region is formed based on the trajectory of the line-drawing event, and a portion within this closed region is taken as the target region, such as... Figure 7 As shown. The line drawing event can either draw a line exactly according to the input trajectory, or it can draw a line based on input marker points using a preset trajectory. For example, when using a mouse as an input device, the line can be drawn exactly according to the mouse's movement trajectory, or the line drawing trajectory can be set to a straight line, determined by the start and end points of the mouse click. Another example is using a mouse as an input device to determine the line drawing trajectory by performing a selection operation; the shape of the selection can be preset, such as a circle or rectangle.

[0142] Additionally, an end action can be pre-set. Upon detecting a user's end action, processing to determine the target area can be performed. For example, when the operation event includes a mouse click, the end action can be set to a right-click, a left-click, or no mouse click detected within a set time. Based on this type of end action setting, multiple mouse clicks can be detected during operation event detection. When a right-click or other end action is detected, the click positions of the previously detected mouse clicks are connected to form a closed area, which is used as the target area. When the operation event includes a mouse selection / line drawing event, the end action can be set to a mouse release action.

[0143] Step 206: Based on the first position calibration information, find the second target region in the visible light image that corresponds to the same position as the first target region.

[0144] In the process of finding the second target region, the first positional calibration information can be used to find the corresponding pixels in the second image for each pixel in the first target region, thus forming the second target region.

[0145] When the first reference point is located in the thermal imaging image, optionally, for any pixel X in the first target region, the relative positional relationship T between the pixel X and the first reference point (e.g., the aforementioned pixel A) is determined; then, based on the positional calibration information, in the visible light image, a first auxiliary reference point (e.g., the aforementioned pixel B) corresponding to the first reference point is determined, and a pixel Y having a positional relationship T with the first auxiliary reference point is found as the pixel corresponding to the same position as pixel X.

[0146] When the first reference point is located in the visible light image, optionally, in the thermal imaging image, a first auxiliary reference point (e.g., the aforementioned pixel point B) is determined at the same position as the reference point. For any pixel point X in the first target region, the relative positional relationship T between the pixel point X and the first auxiliary reference point (e.g., the aforementioned pixel point B) is determined. Then, based on the positional calibration information, in the visible light image, a pixel point Y that has a positional relationship T with the first reference point (e.g., the aforementioned pixel point A) is found as the pixel point at the same position as pixel point X.

[0147] The above description uses a thermal imaging image as the first image and a visible light image as the second image as an example in this embodiment. For cases where a visible light image is used as the first image and a thermal imaging image as the second image, or other cases, the corresponding method will be used. In summary, the process of determining corresponding pixels at the same location can be performed as follows:

[0148] 1. When the first reference point is located in the first image, optionally, for any pixel X in the first target area, the relative positional relationship T between the pixel X and the first reference point is determined; then, based on the positional calibration information, in the second image, a first auxiliary reference point corresponding to the first reference point is determined, and a pixel Y having a positional relationship T with the first auxiliary reference point is found as the pixel corresponding to the same position as pixel X.

[0149] 2. When the first reference point is located in the second image, optionally, in the first image, a first auxiliary reference point is determined to have the same position as the first reference point. For any pixel X in the first target area, the relative positional relationship T between the pixel X and the first auxiliary reference point is determined. Then, based on the first positional calibration information, in the second image, a pixel Y that has a positional relationship T with the first reference point is found as the pixel point with the same position as the pixel X.

[0150] This embodiment only discusses the case of finding and displaying a third target region corresponding to the same position as the first target region on the second image. In practice, for cases where it is necessary to find and display a third target region corresponding to the same position as the first target region on the third image, the second position calibration information can be similar to the first position calibration information, using image registration relationships or mapping relationships, etc. Specifically, it can include: the coordinates of the second reference point, the image size of the image where the second reference point is located, and the relative angle between the first and third images to ensure that the pixels at the same position in the first and third images are coincident; wherein, the coordinates of the second reference point are the pixel coordinates of the corresponding position of the reference point on either the first or third image in the other image. Specifically, the process of determining the corresponding pixel at the same position based on the second position calibration information is the same as the aforementioned process of determining the corresponding pixel at the same position based on the first position calibration information, the difference being that the second reference point is used to replace the first reference point, and the second auxiliary reference point is used to replace the first auxiliary reference point, wherein, the coordinates of the second reference point are the pixel coordinates of the corresponding position of the reference point on either the first or third image in the other image (i.e., the second auxiliary reference point).

[0151] The second position calibration information can also be obtained from the initial position calibration information, or it can be set by manually dragging.

[0152] Using the above method, for each pixel in the first target region, corresponding pixels in the second image at the same position are determined. These determined pixels form the second target region, which is the image region in the second image corresponding to the first target region at the same position. This method of determining the second target region in the second image can automatically discover the corresponding region while ensuring the accuracy of the corresponding region.

[0153] Step 207: Display the image within the second target region of the second image.

[0154] The second target region is cropped from the saved second image and displayed.

[0155] When displaying the image of the second target area, the following method can be used:

[0156] 1. An image of the second target area can be displayed at the location of the first area.

[0157] Optionally, a new layer can be added at the location of the first target area to display the image in the second target area; the layer size can be the same as the size of the second target area, covering the first target area, and rendering the image of the second target area on the newly added layer;

[0158] Alternatively, the values ​​of the corresponding pixels in the first region can be replaced by the values ​​of the pixels in the second target region at the location of the first region; that is, the image of the second target region is displayed on the layer where the thermal imaging image is located, and the image of the second target region is rendered on the first target region.

[0159] 2. Display the image within the second target area at a preset position on the image display interface;

[0160] Optionally, an area can be set aside at a preset position on the thermal imaging image display interface to display the image of the second target area.

[0161] 3. Display the image within the second target area in a separate image presentation interface, distinct from the main image presentation interface.

[0162] An image display interface other than the thermal imaging image display interface can be used to display the image within the second target area. Optionally, a new window can pop up, rendering the image within the second target area on the new window. The pop-up window can partially or completely cover the original image display interface, or it can be located outside the original image display interface.

[0163] In addition, when displaying the image within the second target area, it can be displayed at its original size, or it can be enlarged or reduced according to a set ratio. When the image within the second target area is displayed in a magnified or reduced manner, the display area occupied by the second target area can be adaptively increased or decreased when displaying the image within the second target area at the location of the first target area.

[0164] Meanwhile, the aforementioned image display processing can be implemented on various electronic devices, such as computers and mobile phones. Input devices like mice and fingers are used to input operation events to determine the first target area. As input operation events are updated (e.g., mouse movement), the system tracks and detects these events in real time, determining the latest first target area and the corresponding second target area based on the updated events, and then updating and displaying the image within the latest second target area in real time. When there is a need to retract the display, a pre-set retraction operation can be executed, in which case the first image will be displayed on the image presentation interface.

[0165] At this point, Figure 2The illustrated method flow ends here. Through the processing of this embodiment, the part of interest can be selected by operating on the first image, and the corresponding image region in the second and / or third image can be automatically found and displayed. This achieves automatic searching and improves the accuracy of the corresponding location region. It also allows for convenient switching between displaying the part of interest in thermal imaging, visible light images, and dual-light fusion images, and enables quick location of the position of interest by using different viewing angles.

[0166] The following is an example of image display processing implemented using this application through an interactive scheme, still using a thermal imaging image as the first image and a visible light image as the second image as an example:

[0167] 1. Users view thermal imaging images;

[0168] 2. The user clicks and selects the "Fusion Magic Mirror" option;

[0169] 3. Users can move the "fusion magic mirror" on the thermal imaging image using a mouse, finger, or other means;

[0170] 4. When the "Fusion Magic Mirror" is moved to a certain part, the corresponding part displays a visible light image of the same location.

[0171] Through the above interactions, users can easily find the visible light image of the area they are interested in, allowing them to flexibly interpret an image from different perspectives.

[0172] This application also provides an image display processing apparatus, which can be used to implement the image display processing method in this application. Figure 8 This is a schematic diagram of the basic structure of the image display processing device. (Example) Figure 8 As shown, the device includes: an acquisition unit, an operation event detection unit, a target area delineation unit, and a display unit;

[0173] The acquisition unit is used to acquire thermal imaging images and visible light images obtained by shooting the same scene;

[0174] An operation event detection unit is used to detect operation events on the currently displayed first image in the image presentation interface; wherein the first image is a thermal imaging image, a visible light image, or a dual-light fusion image generated by superimposing a thermal imaging image and a visible light image;

[0175] The target region delineation unit is configured to determine a selected first target region in the first image based on the location information of the detected operation event; and to search for a second target region in the second image corresponding to the location of the first target region based on the first positional calibration information between the first image and the second image; wherein the second image is another image other than the first image among the thermal imaging image, the visible light image, and the dual-light fusion image, and the first positional calibration information includes auxiliary information that ensures that pixels at the same position in the first image and the second image overlap;

[0176] The display unit is used to display the image within the second target area of ​​the second image.

[0177] Optionally, after determining the first target region in the target region delineation unit, a third target region corresponding to the same position as the first target region can be further found in the third image based on the second position calibration information; and the image within the third target region in the third image can be displayed. The third image is another image among the thermal imaging image, visible light image, and dual-light fusion image, excluding the first and second images. The second position calibration information includes auxiliary information that ensures the overlap of pixels at corresponding positions in the first and third images.

[0178] Optionally, the first position calibration information includes the coordinates of the first reference point, the image size of the image where the first reference point is located, and the relative angle between the first image and the second image when the corresponding pixels in the first image and the second image are guaranteed to overlap; wherein, the coordinates of the first reference point are the pixel coordinates of the corresponding position of the reference point on either the first image or the second image on the other image;

[0179] The second position calibration information includes the coordinates of the second reference point, the image size of the image where the second reference point is located, and the relative angle between the first image and the third image when the corresponding pixels in the first image and the third image are guaranteed to overlap.

[0180] The second reference point coordinates are the pixel coordinates of the corresponding position of the reference point on one of the first and third images on the other image.

[0181] Optionally, the operation events include click events;

[0182] In the operation event detection unit, based on the location information of the detected operation event, a first target region in the first image is determined, which may specifically include:

[0183] The first target area is determined based on the click positions of at least three click events; wherein the boundary of the first target area passes through the click positions of at least three click events.

[0184] or,

[0185] The click location of the click event is used as the center of the preset graphic, and the coverage area of ​​the preset graphic is determined as the first target area.

[0186] Optionally, the operation events include line drawing events;

[0187] In the operation event detection unit, based on the location information of the detected operation event, a first target region in the first image is determined, which may specifically include:

[0188] Based on the trajectory information of the line drawing event, the first target area is determined;

[0189] The boundary of the first target area coincides with the trajectory information.

[0190] In the operation event detection unit, after a preset end operation is detected, the process of determining the first target region selected in the first image is performed.

[0191] Optionally, in the target region delineation unit, based on the first location calibration information, a second target region corresponding to the location of the first target region is searched in the second image, which may specifically include:

[0192] Based on the first position calibration information, the corresponding pixel points in the second image of each pixel point in the first target region are determined to form the second target region.

[0193] In the target region delineation unit, based on the second position calibration information, a third target region corresponding to the position of the first target region is found in the third image. Specifically, this may include: based on the second position calibration information, determining the pixel points at the corresponding positions of each pixel point in the first target region in the third image to form the third target region.

[0194] Optionally, the display unit displays an image within the second target area of ​​the second image, which may specifically include:

[0195] Display the image within the second target area at the location of the first area; or,

[0196] Display the image within the second target area at a preset position on the image display interface; or,

[0197] The image within the second target area is displayed in a separate image presentation interface, distinct from the main image presentation interface.

[0198] The display unit displays an image within the third target area of ​​the third image, which may specifically include:

[0199] Display the image within the third target area at the location of the first target area; or,

[0200] Display the image within the third target area at a preset location on the image display interface; or,

[0201] The image within the third target area is displayed in a separate image presentation interface, distinct from the main image presentation interface.

[0202] Optionally, in the display unit, an image within the second target area is displayed at the location of the first target area. Specific processing may include:

[0203] Add a new layer at the location of the first target area to display the image within the second target area; or,

[0204] At the location of the first target area, the values ​​of the corresponding pixels in the first target area are replaced with the values ​​of the pixels in the second target area.

[0205] In the display unit, an image within the third target area is displayed at the location of the first target area. Specific processing may include:

[0206] Add a new layer at the location of the first target area to display the image within the third target area; or,

[0207] At the location of the first target region, the values ​​of the corresponding pixels in the first target region are replaced with the values ​​of the pixels in the third target region.

[0208] Alternatively, in the display unit, another graphical interface can be created via a pop-up window.

[0209] Figure 9 An electronic device is also provided for this application. For example... Figure 9 As shown, it illustrates a structural schematic diagram of the electronic device involved in the embodiments of this application, specifically:

[0210] The electronic device may include a processor 901 with one or more processing cores, a memory 902 with one or more computer-readable storage media, and a computer program stored in the memory and executable on the processor. When the program in the memory 902 is executed, the above-described image display processing method can be implemented.

[0211] Specifically, in practical applications, this electronic device may also include components such as a power supply 903 and an input / output unit 904. Those skilled in the art will understand that... Figure 9 The structure of the electronic device shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0212] The processor 901 is the control center of the electronic device. It connects various parts of the electronic device through various interfaces and lines. By running or executing software programs and / or modules stored in the memory 902, and calling data stored in the memory 902, it performs various functions of the server and processes data, thereby monitoring the electronic device as a whole.

[0213] Memory 902 can be used to store software programs and modules, i.e., the aforementioned computer-readable storage medium. Processor 901 executes various functional applications and data processing by running the software programs and modules stored in memory 902. Memory 902 may primarily include a program storage area and a data storage area, wherein the program storage area may store the operating system, application programs required for at least one function, etc.; the data storage area may store data created according to the use of the server, etc. In addition, memory 902 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory 902 may also include a memory controller to provide processor 901 with access to memory 902.

[0214] The electronic device also includes a power supply 903 that supplies power to the various components. This power supply can be logically connected to the processor 901 via a power management system, enabling functions such as charging, discharging, and power consumption management. The power supply 903 may also include one or more DC or AC power supplies, a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator, or any other components.

[0215] The electronic device may also include an input / output unit 904, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control. The input unit output 904 can also be used to display information input by the user or information provided to the user, as well as various graphical user interfaces, which can be composed of graphics, text, icons, video, and any combination thereof.

[0216] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for displaying and processing an image, characterized in that, include: Acquire thermal and visible light images of the same scene. Detect user-triggered operation events in the image presentation interface for the currently displayed first image; wherein the first image is the thermal imaging image, the visible light image, or a dual-light fusion image generated by superimposing the thermal imaging image and the visible light image; Based on the location information of the detected operation event, a first target region is determined in the first image; Based on the first positional calibration information between the first image and the second image, a second target region corresponding to the same position as the first target region is found in the second image; wherein, the second image is another image other than the first image among the thermal imaging image, the visible light image and the dual-light fusion image, and the first positional calibration information includes auxiliary information to ensure that the pixels corresponding to the same position in the first image and the second image coincide. Display the image within the second target region of the second image; The step of displaying the image within the second target region of the second image includes: Display the image within the second target area at the location of the first target area; or... The image within the second target area is displayed at a preset position on the image presentation interface; or, The image within the second target area is displayed in a separate image presentation interface, distinct from the aforementioned image presentation interface.

2. The method according to claim 1, characterized in that, After determining the first target region, the method further includes: Based on the second positional calibration information between the first image and the third image, a third target region corresponding to the same position as the first target region is found in the third image; wherein, the third image is another image other than the first image and the second image among the thermal imaging image, the visible light image and the dual-light fusion image, and the second positional calibration information includes auxiliary information to ensure that the pixels corresponding to the same position in the first image and the third image are coincident; Display the image within the third target region of the third image.

3. The method according to claim 1, characterized in that, The first position calibration information includes the coordinates of the first reference point, the image size of the image where the first reference point is located, and the relative angle between the first image and the second image when the corresponding pixels in the first image and the second image are guaranteed to overlap; Wherein, the coordinates of the first reference point are the pixel coordinates of the corresponding position of the first reference point on the other image, which is either the first image or the second image.

4. The method according to claim 2, characterized in that, The second position calibration information includes the coordinates of the second reference point, the image size of the image where the second reference point is located, and the relative angle between the first image and the third image when the pixels at the same position in the first image and the third image are guaranteed to overlap; Wherein, the second reference point coordinates are the pixel coordinates of the corresponding position of the reference point on the other image, which is the reference point on either the first image or the third image.

5. The method according to claim 1, characterized in that, The operation events include click events; The step of determining the selected first target region in the first image based on the location information of the detected operation event includes: The first target region is determined based on the click positions of at least three click events; wherein the boundary of the first target region passes through the click positions of at least three click events. or, Using the click location of the click event as the center of the preset graphic, the coverage area of ​​the preset graphic is determined as the first target area.

6. The method according to claim 1, characterized in that, The operation events include line drawing events; The step of determining the selected first target region in the first image based on the location information of the detected operation event includes: Based on the trajectory information of the line drawing event, the first target area is determined; The boundary of the first target area coincides with the trajectory information.

7. The method according to claim 5, characterized in that, After detecting the preset end operation, the process of determining the selected first target region in the first image is performed.

8. The method according to claim 1, characterized in that, The step of finding a second target region in the second image corresponding to the location of the first target region based on the first location calibration information includes: Based on the first location calibration information, the corresponding pixel points in the second image of each pixel point in the first target region are determined to form the second target region.

9. The method according to claim 2, characterized in that, The step of finding a third target region in the third image corresponding to the location of the first target region based on the second location calibration information includes: Based on the second location calibration information, the corresponding pixel points of each pixel point in the first target region are determined in the third image to form the third target region.

10. The method according to claim 2, characterized in that, The display of the image within the third target region in the third image includes: Display the image within the third target area at the location of the first target area; or... The image within the third target area is displayed at a preset position on the image presentation interface; or, The image within the third target area is displayed in a separate image presentation interface, distinct from the image presentation interface described above.

11. The method according to claim 10, characterized in that, Displaying the image within the second target area at the location of the first target area includes: A new layer is added at the location of the first target area to display the image in the second target area; or, at the location of the first target area, the values ​​of the corresponding pixels in the first target area are replaced with the values ​​of the pixels in the second target area. Displaying the image within the third target area at the location of the first target area includes: A new layer is added at the location of the first target area to display the image within the third target area; or, at the location of the first target area, the values ​​of the corresponding pixels in the first target area are replaced with the values ​​of the pixels in the third target area.

12. The method according to claim 10, characterized in that, The other image presentation interface is created via a pop-up window.

13. An image display processing apparatus, characterized in that, include: The system includes an acquisition unit, an operation event detection unit, a target area delineation unit, and a display unit. The acquisition unit is used to acquire thermal imaging images and visible light images obtained by shooting the same scene; The operation event detection unit is used to detect operation events triggered by the user in the image presentation interface for the currently displayed first image; wherein, the first image is the thermal imaging image, the visible light image, or a dual-light fusion image generated by superimposing the thermal imaging image and the visible light image; The target region delineation unit is configured to determine a selected first target region in the first image based on the location information of the detected operation event; and to search for a second target region in the second image corresponding to the location of the first target region based on the first positional calibration information between the first image and the second image; wherein the second image is another image besides the first image among the thermal imaging image, the visible light image, and the dual-light fusion image, and the first positional calibration information includes auxiliary information that ensures that pixels at the same position in the first image and the second image overlap; The display unit is used to display the image within the second target area in the second image; The step of displaying the image within the second target region of the second image includes: Display the image within the second target area at the location of the first target area; or... The image within the second target area is displayed at a preset position on the image presentation interface; or, The image within the second target area is displayed in a separate image presentation interface, distinct from the aforementioned image presentation interface.

14. The apparatus according to claim 13, characterized in that, After determining the first target region, the target region delineation unit is further used to search for a third target region in the third image that corresponds to the same position as the first target region based on the second position calibration information between the first image and the third image; wherein, the third image is another image other than the first image and the second image among the thermal imaging image, the visible light image and the dual-light fusion image, and the second position calibration information includes auxiliary information to ensure that the pixels corresponding to the same position in the first image and the third image are coincident; The display unit is further configured to display an image within the third target area of ​​the third image; If the operation event includes a click event, then: In the operation event detection unit, based on the location information of the detected operation event, the selected first target region in the first image is determined, including: The first target area is determined based on the click positions of at least three click events; wherein the boundary of the first target area passes through the click positions of at least three click events; or, the coverage area of ​​the preset graphic is determined as the first target area based on the click positions of the click events as the center of the preset graphic. If the operation event includes a line drawing event, then: In the operation event detection unit, determining the selected first target region in the first image based on the location information of the detected operation event includes: Based on the trajectory information of the line drawing event, the first target area is determined; Wherein, the boundary of the first target area coincides with the trajectory information; In the operation event detection unit, after a preset end operation is detected, the process of determining the selected first target region in the first image is executed; In the target region delineation unit, the step of finding a second target region in the second image corresponding to the location of the first target region based on the first location calibration information includes: Based on the first location calibration information, the corresponding pixel points in the second image of each pixel point in the first target region are determined to form the second target region; In the target region delineation unit, the step of searching for a third target region in the third image corresponding to the location of the first target region based on the second location calibration information includes: Based on the second position calibration information, the pixel points in the first target region corresponding to the positions in the third image are determined to form the third target region; The step of displaying the image within the third target area of ​​the third image in the display unit includes: Display the image within the third target area at the location of the first target area; or... The image within the third target area is displayed at a preset position on the image presentation interface; or, The image within the third target area is displayed in a separate image presentation interface, distinct from the aforementioned image presentation interface. In the display unit, displaying the image within the second target area at the location of the first target area includes: A new layer is added at the location of the first target area to display the image in the second target area; or, at the location of the first target area, the values ​​of the corresponding pixels in the first target area are replaced with the values ​​of the pixels in the second target area. In the display unit, displaying the image within the third target area at the location of the first target area includes: A new layer is added at the location of the first target area to display the image within the third target area; or, at the location of the first target area, the values ​​of the corresponding pixels in the first target area are replaced with the values ​​of the pixels in the third target area.

15. An electronic device, characterized in that, The electronic device includes at least a computer-readable storage medium and a processor; The processor is configured to read executable instructions from the computer-readable storage medium and execute the instructions to implement the image display processing method according to any one of claims 1 to 12.

Images