Image display method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an image display method and device, electronic equipment and a storage medium, and is applied to the technical field of image processing. The method comprises the following steps: acquiring a first image and a second image, and acquiring a thermal imaging window position in the second image; copying the Y data part of the first image into the second image according to the thermal imaging window position when the thermal imaging window position shows that the copied starting pixel rows and / or starting pixel columns are odd numbers; aligning the thermal imaging window position according to a preset alignment rule to obtain a UV alignment position; according to the UV alignment position, copying the UV data part of the first image into the second image to obtain a third image, and displaying the third image; the initial point of the thermal imaging image can be the position of the odd point coordinate of the second image, so that the display position precision of the thermal imaging image is improved, the aiming error is reduced, and the aiming precision is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image display method, device, electronic device and storage medium. Background Art

[0002] Compared with the previous situation of directly observing the target through the aiming device, in the relevant technology, the aiming product can display the image collected by the sensor on the aiming screen to facilitate the user to retrieve, identify and lock the target. In order to improve the accuracy of aiming, at least two different sensors can be provided in the aiming product to collect images, and the images collected by each sensor are displayed on the aiming screen. In one example, in the CLIPON mode of the aiming product, there is a CLIPON window in the aiming screen for superimposing an image with richer content and more details on the original image. The window can be moved according to the needs of the user, so that the user can obtain more and clearer target information.

[0003] In the related art, the CLIPOON window can only be moved with an even number of pixels, for example, the CLIPOON window is moved 2, 4, 6 or 8 pixels each time, that is, the starting position of the CLIPOON window image can only be a pixel point with even coordinates, and the position of odd pixels cannot be used as the starting position of the CLIPON window image overlay, resulting in the position of the CLIPON window image displayed on the screen is not accurate enough. When aiming at the target, there will be an error of 1 pixel, which affects the aiming accuracy. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide an image display method, device, electronic device and storage medium to improve the display position accuracy of the window image. The specific technical solution is as follows:

[0005] In a first aspect of an embodiment of the present application, there is provided an image display method, the method comprising:

[0006] Acquire a first image and a second image, and acquire a thermal imaging window position in the second image; wherein the field of view of the first image is within the field of view of the second image, and the first image and the second image are images in YUV format;

[0007] When the thermal imaging window position indicates that the starting pixel row and / or starting pixel column of the copy is an odd number, copying the Y data portion of the first image to the second image according to the thermal imaging window position;

[0008] Aligning the thermal imaging window position according to a preset alignment rule to obtain a UV alignment position;

[0009] According to the UV alignment position, the UV data portion of the first image is copied to the second image to obtain a third image, and the third image is displayed.

[0010] In a possible implementation manner, when the thermal imaging window position indicates that the starting pixel row and / or starting pixel column of the copy is an odd number, copying the Y data portion of the first image to the second image according to the thermal imaging window position includes:

[0011] When the starting pixel row and / or starting pixel column of the thermal imaging window position indicating the copy is an odd number, obtaining the starting pixel coordinate (a, b) of the thermal imaging window position, wherein a and b are both integers, and at least one of a and b is an odd number;

[0012] The data at position Y(i, j) in the first image is copied to position Y(i+a, j+b) in the second image, where Y(i, j) represents the position of the i-th row and j-th column in the Y data portion of the first image.

[0013] In a possible implementation manner, aligning the thermal imaging window position according to a preset alignment rule to obtain a UV alignment position includes:

[0014] The thermal imaging window position is aligned in a leftward and upward alignment manner to obtain a UV alignment position.

[0015] In a possible implementation manner, aligning the thermal imaging window position in a leftward and upward alignment manner to obtain a UV alignment position includes:

[0016] When the thermal imaging window position indicates that the starting pixel row of the copy is an odd number and the starting pixel column is an even number, the thermal imaging window position is translated upward by one pixel to obtain a UV alignment position;

[0017] When the thermal imaging window position indicates that the starting pixel row of the copy is an even number and the starting pixel column is an odd number, the thermal imaging window position is shifted to the left by one pixel to obtain a UV alignment position;

[0018] When the thermal imaging window position indicates that the starting pixel row of the copy is an odd number and the starting pixel column is an odd number, the thermal imaging window position is translated one pixel to the left and one pixel upward to obtain a UV alignment position.

[0019] In a possible implementation, acquiring the first image and the second image includes:

[0020] Acquire a thermal imaging image collected by the first sensor and a target resolution, wherein the target resolution is the same as the resolution of the thermal imaging window;

[0021] Adjusting the resolution of the thermal imaging image to the target resolution to obtain a first image;

[0022] A second image captured by a second sensor is acquired, wherein a field of view of the first sensor is within a field of view of the second sensor.

[0023] According to a second aspect of the embodiments of the present application, an image display device is provided, the device comprising an image preprocessing module, an image processing module and a display module.

[0024] The image processing module includes an image first copy submodule, an image alignment submodule and an image second copy submodule;

[0025] The image preprocessing module is used to obtain the first image and the second image, and obtain the position of the thermal imaging window in the second image; wherein the field of view of the first image is within the field of view of the second image, and the first image and the second image are images in YUV format;

[0026] The image first copy submodule is used to copy the Y data portion of the first image to the second image according to the thermal imaging window position when the thermal imaging window position indicates that the starting pixel row and / or starting pixel column of the copy is an odd number;

[0027] The image alignment submodule is used to align the thermal imaging window position according to a preset alignment rule to obtain a UV alignment position;

[0028] The image second copy submodule is used to copy the UV data portion of the first image to the second image according to the UV alignment position to obtain a third image;

[0029] The display module is used to display the third image.

[0030] In a possible implementation, the image first copy submodule includes:

[0031] a starting pixel acquisition unit, specifically used to acquire the starting pixel coordinates (a, b) of the thermal imaging window position when the starting pixel row and / or starting pixel column of the thermal imaging window position indicating the copy is an odd number, wherein a and b are both integers, and at least one of a and b is an odd number;

[0032] The image copying unit is specifically used to copy the data at the Y(i, j) position in the first image to the Y(i+a, j+b) position in the second image, where Y(i, j) represents the position of the i-th row and j-th column in the Y data part of the first image.

[0033] In a possible implementation, the image alignment submodule includes:

[0034] The image alignment unit is specifically used to align the thermal imaging window position by aligning leftward and upward to obtain a UV alignment position.

[0035] In a possible implementation, the image alignment unit includes:

[0036] The image first alignment subunit is specifically used to translate the thermal imaging window position upward by one pixel to obtain a UV alignment position when the thermal imaging window position indicates that the starting pixel row of the copy is an odd number and the starting pixel column is an even number;

[0037] The second image alignment subunit is specifically used to shift the thermal imaging window position to the left by one pixel to obtain a UV alignment position when the thermal imaging window position indicates that the starting pixel row of the copy is an even number and the starting pixel column is an odd number;

[0038] The third image alignment subunit is specifically used to translate the thermal imaging window position one pixel to the left and one pixel upward to obtain a UV alignment position when the thermal imaging window position indicates that the starting pixel row of the copy is an odd number and the starting pixel column is an odd number.

[0039] In a possible implementation, the image preprocessing module includes:

[0040] The thermal imaging image acquisition submodule is specifically used to acquire the thermal imaging image collected by the first sensor and the target resolution, wherein the size of the target resolution is the same as the resolution size of the thermal imaging window;

[0041] A resolution adjustment submodule, specifically used to adjust the resolution of the thermal imaging image to the target resolution to obtain a first image;

[0042] The second image acquisition submodule is specifically used to acquire a second image collected by a second sensor, wherein the field of view of the first sensor is within the field of view of the second sensor.

[0043] In another aspect of the embodiment of the present application, an electronic device is further provided, including:

[0044] Memory, used to store computer programs;

[0045] The processor is used to implement the image display method described in any of the embodiments of the present application when executing the program stored in the memory.

[0046] In another aspect of the present application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any of the above-mentioned image display methods is implemented.

[0047] An embodiment of the present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the above-mentioned image display methods.

[0048] Beneficial effects of the embodiments of the present application:

[0049] An image display method, device, electronic device and storage medium provided in the embodiments of the present application can copy the Y data portion of the first image to the second image when the thermal imaging window position indicates that the starting pixel row and / or starting pixel column of the copy is an odd number, and then UV align the position of the thermal imaging window so that the starting point of the thermal imaging image can be the position of the odd-numbered point coordinates of the second image, thereby improving the display position accuracy of the first image (window image), reducing the error during aiming, and improving the aiming accuracy.

[0050] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0052] Figure 1a A schematic diagram of CLIPON mode usage of a sighting product in the prior art;

[0053] Figure 1b It is a schematic diagram of YUV encoding of an image in the prior art;

[0054] Figure 2 A flow chart of an image display method provided in an embodiment of the present application;

[0055] Figure 3 A schematic diagram of Y data copy provided in an embodiment of the present application;

[0056] Figure 4 A first schematic diagram of UV data alignment provided in an embodiment of the present application;

[0057] Figure 5 A second schematic diagram of UV data alignment provided in an embodiment of the present application;

[0058] Figure 6 A third schematic diagram of UV data alignment provided in an embodiment of the present application;

[0059] Figure 7 A fourth schematic diagram of UV data alignment provided in an embodiment of the present application;

[0060] Figure 8 A fifth schematic diagram of UV data alignment provided in an embodiment of the present application;

[0061] Fig. 9 A sixth schematic diagram of UV data alignment provided in an embodiment of the present application;

[0062] Fig.10 Another flow chart of the image display method provided in the embodiment of the present application;

[0063] Fig.11 A schematic diagram of the structure of an image display device provided in an embodiment of the present application;

[0064] Fig.12 A schematic diagram of the structure of an image display device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0066] In sighting products, there is a use mode called CLIPON mode, which is mainly used to overlay images with richer content, more details or higher clarity on the original image, which can more effectively retrieve, identify and lock targets. When using this mode, a smaller display window will appear on the display screen, and the user can retrieve, identify and lock targets by moving the window. Figure 1a As shown, the user visible area is the movable display window in CLIPON mode, and the OLED screen area is the display screen of the sight product, which displays the original image.

[0067] In the related art, the image with more details is usually superimposed on the original image by YUV (YcbCr, color encoding method) encoding the original image and the image with more details, and replacing the image data of the original image with the image data of the image with more details within the coordinate range corresponding to the CLIPON window. Taking the YUV420 format as an example, the image after YUV encoding is as follows Figure 1bAs shown, each "2X2" square pixel shares a set of UV data. For example, the four pixels with coordinates (0, 0), (0, 1), (1, 0) and (1, 1) share the data set U00 and V00. Therefore, when the CLIPON window moves, it can only move with "2X2" square pixels as the minimum unit. Therefore, the CLIPON window only supports display at the starting coordinates of even pixels, and each movement must be 2 pixels or a multiple of 2 pixels in the X or Y direction.

[0068] Since the CLIPON window only supports display at even-numbered starting coordinates after calibration, a 1-pixel error will be introduced when facing a scene aligned with an odd-numbered starting point, causing the position of the image with more details displayed on the screen to be inaccurate, thereby affecting the user's accuracy when aiming at the target. In an example, at a distance of 100 meters, when using a sight product in the prior art, a 1-pixel error will result in a 1-cm error in the landing point of the projectile.

[0069] In order to solve at least one of the above problems, a first aspect of an embodiment of the present application provides an image display method, the method comprising: Figure 2 Steps shown:

[0070] Step S201: Acquire a first image and a second image, and acquire a thermal imaging window position in the second image.

[0071] The field of view of the first image is within the field of view of the second image, and both the first image and the second image are images in YUV format. The first image and the second image may be images with different field of view for the same scene, and the two may be acquired by different sensors. The field of view of the second image is larger than that of the first image, and the details of the first image are richer than those of the second image. The field of view of the first image is within the field of view of the second image, and the position of the thermal imaging window in the second image indicates the position of the field of view of the first image in the second image.

[0072] The first image and the second image can both be thermal images. In some scenarios, the second image can also be a visible light image, all of which are within the protection scope of this application. In one example, a white light sight and a night vision device are installed on a terminal device (such as a sighting device). The white light sight is used for aiming, and the night vision device is used to obtain thermal images. In this case, the second image is a thermal image with a large field of view, which can be a grayscale image after grayscale processing, and the first image is a thermal image with rich details in a small field of view.

[0073] YUV encoding is performed on the acquired first image and second image, and each pixel in the image is represented by Y, U, and V data, wherein the Y data represents the brightness information of the pixel, and the U data and the V data represent the chromaticity information of the pixel, wherein the chromaticity information includes hue and saturation. In an embodiment of the present application, the YUV sampling format and storage format of the first image and the second image are the same, for example, both are sampled based on YUV4:4:4, YUV4:2:2 or YUV4:2:0 format, and are stored in a plane format or a packed format. In an example, if the first image and the second image are both in NV12 format, then the first image and the second image are both sampled in YUV4:2:0 format and stored in a plane format.

[0074] The thermal imaging window is used to display the first image, and its position can be calibrated at the factory or adjusted by the user during actual use, such as Figure 1a The user visible area in the thermal imaging window in the embodiment of the present application. In practical applications, the method of the embodiment of the present application can be implemented by a terminal device. Specifically, the terminal device can be a device with a sight, for example, a sighting product, a surveying and mapping tool, and other devices that need to obtain thermal imaging images for observation.

[0075] Step S202: when the thermal imaging window position indicates that the starting pixel row and / or starting pixel column of the copy is an odd number, the Y data portion of the first image is copied to the second image according to the thermal imaging window position.

[0076] In an example, Figure 3 As shown, the Y data of the first image and the second image are encoded starting from Y00. When the thermal imaging window position indicates that the starting pixel row of the copy is the row where Y10, Y30, Y50, Y70, or Y90 is located, and / or the thermal imaging window position indicates that the starting pixel column of the copy is the column where Y01, Y03, Y05, Y07, or Y09 is located, the Y data portion of the first image is copied to the corresponding coordinate range in the second image with the starting coordinate in the second image as the starting point. For example, when the starting pixel row and starting pixel column of the copy are both even numbers, the copying of the Y data is the same as the related art, and the Y data in the first image is copied to the second image in an even-aligned manner; if the starting pixel row and / or starting pixel column of the copy are odd numbers, it is not necessary to align the Y data to an even number, and the Y data of the first image is directly copied to the second image according to the starting coordinate position.

[0077] Among them, the starting coordinates of the thermal imaging window are affected by factors such as the structure of the terminal equipment, material processing and installation. The starting coordinates can be calibrated through production line production, or the user can manually move the thermal imaging window position to select the calibrated starting coordinates when using the terminal equipment.

[0078] like Figure 3 As shown, taking "Y33" in the second image as the starting coordinate of the thermal imaging window as an example, the coordinate is located in both odd rows and odd columns. When the Y data of the first image data is copied to the second image, the Y data of the "Y00" coordinate in the first image is used to replace the Y data of the "Y33" coordinate in the second image; the Y data of the "Y01" coordinate in the first image is used to replace the Y data of the "Y34" coordinate in the second image; and so on, until the Y data in the coordinate ranges of "Y33-Y36, Y43-Y46, Y53-Y56 and Y63-Y66" of the second image is replaced by the Y data in the coordinate range of "Y00-Y33" in the first image.

[0079] Step S203: aligning the thermal imaging window position according to a preset alignment rule to obtain a UV alignment position.

[0080] The preset alignment rules are used to align the UV data in the horizontal and vertical directions of the pixels. For example, when the starting pixel row of the copy is an odd number, it is translated one pixel up or down in the row direction of the pixels; when the starting pixel column of the copy is an odd number, it is translated one pixel left or right in the column direction of the pixels. When the starting pixel row of the copy is an even number, the thermal imaging window position does not need to be moved in the row direction of the pixels; when the starting pixel column of the copy is an even number, the thermal imaging window position does not need to be moved in the column direction of the pixels. In actual scenarios, the preset alignment rules can be determined according to the YUV format of the image. Still taking the example that the first image and the second image are both in NV12 format, the Y data of each "2X2" small square shares a set of UV data. For example, in Figure 3 In the first image shown, "Y00", "Y01", "Y10" and "Y11" share the set of UV data "U00V00", and "Y02", "Y03", "Y12" and "Y13" share the set of UV data "U01V01".

[0081] When aligning the thermal imaging window position, the UV can be aligned according to the corresponding relationship between the UV data and the Y data. Since the Y data of each "2X2" small square shares a set of UV data, when the thermal imaging window position represents that the starting pixel row and / or starting pixel column of the copy is an odd number, the thermal imaging window position can be translated up, down, left or right by one pixel to perform UV alignment when it satisfies 2 alignment, and obtain the UV alignment position.

[0082] Step S204: according to the UV alignment position, partially copy the UV data of the first image to the second image to obtain a third image, and display the third image.

[0083] Similarly, after obtaining the UV alignment position, the UV data of the second image at the position is replaced with the UV data of the first image to achieve the copying of the UV data. In the embodiment of the present application, step S204 can be implemented by different device modules, for example, the image processing module copies the UV data, and the display module displays the third image. When displaying the third image, the SDK (Software Development Kit) bottom layer interface can be called to send the third image to the display module, and the display module displays the third image on the terminal screen.

[0084] By applying the method of the embodiment of the present application, when the thermal imaging window position indicates that the starting pixel row and / or starting pixel column of the copy is an odd number, the Y data portion of the first image can be copied to the second image, and then the position of the thermal imaging window is UV aligned, so that the starting point of the thermal imaging image can be the position of the odd-numbered point coordinates of the second image, thereby improving the display position accuracy of the thermal imaging image, reducing the error during aiming, and improving the aiming accuracy.

[0085] In a possible implementation, step S202 can implement the copying of Y data by the following steps:

[0086] Step 1: when the starting pixel row and / or starting pixel column of the thermal imaging window position indicating the copy is an odd number, obtaining the starting pixel coordinate (a, b) of the thermal imaging window position, wherein a and b are both integers, and at least one of a and b is an odd number;

[0087] Step 2: Copy the data at position Y(i, j) in the first image to position Y(i+a, j+b) in the second image, where Y(i, j) represents the position at row i and column j in the Y data portion of the first image.

[0088] In one example, Figure 3 Taking the starting coordinate of the thermal imaging image shown as Y33 of the second image as an example, a and b are both 3. Starting from Y00 of the first image, the data at the Y00 position of the first image is copied to the Y33 position of the second image, and the data at the Y01 position of the first image is copied to the Y34 position of the second image, and so on, until the data at the Y33 position of the first image is copied to the Y66 position of the second image.

[0089] By using the method of the embodiment of the present application, the copy range of the Y data of the first image in the second image can be calculated through the coordinate position of the Y data in the first image and the second image, so that the Y data of the first image can be copied to the corresponding position of the second image to achieve the copy of the Y data. And at least one of a and b is an odd number, which can ensure that the starting pixel coordinates of the thermal imaging image are odd coordinates, so that the Y data is copied from the odd coordinate position, which improves the display position accuracy of the thermal imaging window, reduces the error during aiming, and improves the aiming accuracy.

[0090] In a possible implementation, aligning the thermal imaging window position according to a preset alignment rule to obtain a UV alignment position includes:

[0091] The thermal imaging window position is aligned by aligning to the left and upward to obtain the UV alignment position.

[0092] In practical applications, when the thermal imaging window is located at an odd coordinate, it is shifted leftward and upward by one pixel to satisfy the 2-alignment of the Y data, and then UV alignment is performed to obtain the UV alignment position. In a possible implementation, aligning the thermal imaging window position by aligning leftward and upward may include the following situations:

[0093] Case 1: When the thermal imaging window position indicates that the starting pixel row of the copy is an odd number and the starting pixel column is an even number, the thermal imaging window position is shifted upward by one pixel to obtain the UV alignment position.

[0094] Take the starting point coordinate as Y32 of the second image as an example, Figure 4 As shown, in the second image, the Y data range corresponding to the thermal imaging window position is a 4X4 square range starting from Y32. When aligning, first translate the square range upward by one pixel to become a 4X4 square range starting from Y22, and perform UV alignment on the translated thermal imaging window position. In the second image, the UV data corresponding to Y22 is U11V11, and the UV alignment position is the position range starting from U11V11.

[0095] Case 2: When the thermal imaging window position indicates that the starting pixel row of the copy is an even number and the starting pixel column is an odd number, the thermal imaging window position is shifted one pixel to the left to obtain the UV alignment position.

[0096] Take the starting point coordinate of the second image as an example, Figure 5As shown, in the second image, the Y data range corresponding to the thermal imaging window position is a 4X4 square range starting from Y23. When aligning, first translate the square range to the left by one pixel to become a 4X4 square range starting from Y22, and perform UV alignment on the translated thermal imaging window position. In the second image, the UV data corresponding to Y22 is U11V11, and the UV alignment position is the position range starting from U11V11.

[0097] Case 3: When the thermal imaging window position indicates that the starting pixel row of the copy is an odd number and the starting pixel column is an odd number, the thermal imaging window position is shifted one pixel to the left and one pixel downward to obtain the UV alignment position.

[0098] Take the starting point coordinate of the second image as an example, Figure 6 As shown, in the second image, the Y data range corresponding to the thermal imaging window position is a 4X4 square range starting from Y33. When aligning, the square range is first translated to the left by one pixel, and then translated upward by one pixel to become a 4X4 square range starting from Y22, and UV alignment is performed on the translated thermal imaging window position. In the second image, the UV data corresponding to Y22 is U11V11, and the UV alignment position is the position range starting from U11V11.

[0099] In practical applications, the thermal imaging window position can also be aligned rightward and / or downward to obtain the UV alignment position. In practical applications, UV alignment can be performed in a variety of ways, and a solution with small image error and good display effect is selected as the UV alignment solution.

[0100] In a possible implementation, UV alignment may also include the following situations:

[0101] Case 4: When the thermal imaging window position indicates that the starting pixel row of the copy is an odd number and the starting pixel column is an even number, the thermal imaging window position is shifted downward by one pixel to obtain the UV alignment position.

[0102] Take the starting point coordinate as Y32 of the second image as an example, Figure 7 As shown, in the second image, the Y data range corresponding to the thermal imaging window position is a 4X4 square range starting from Y32. When aligning, first translate the square range downward by one pixel to become a 4X4 square range starting from Y42, and perform UV alignment on the translated thermal imaging window position. In the second image, the UV data corresponding to Y42 is U21V21, and the UV alignment position is the position range starting from U21V21.

[0103] Case 5: When the thermal imaging window position indicates that the starting pixel row of the copy is an even number and the starting pixel column is an odd number, the thermal imaging window position is shifted one pixel to the right to obtain the UV alignment position.

[0104] Take the starting point coordinate of the second image as an example, Figure 8 As shown, in the second image, the Y data range corresponding to the thermal imaging window position is a 4X4 square range starting from Y23. When aligning, first translate the square range to the right by one pixel to become a 4X4 square range starting from Y24, and perform UV alignment on the translated thermal imaging window position. In the second image, the UV data corresponding to Y24 is U12V12, and the UV alignment position is the position range starting from U12V12.

[0105] Case 6: When the thermal imaging window position indicates that the starting pixel row of the copy is an odd number and the starting pixel column is an odd number, the thermal imaging window position is shifted one pixel to the right and one pixel downward to obtain the UV alignment position.

[0106] Take the starting point coordinate of the second image as an example, Fig. 9 As shown, in the second image, the Y data range corresponding to the thermal imaging window position is a 4X4 square range starting from Y33. When aligning, the square range is first translated to the right by one pixel, and then translated downward by one pixel to become a 4X4 square range starting from Y44, and UV alignment is performed on the translated thermal imaging window position. In the second image, the UV data corresponding to Y44 is U22V22, and the UV alignment position is the position range starting from U22V22.

[0107] By applying the method of the embodiment of the present application, the thermal imaging window position can be UV aligned to the left and / or upward, so that the copied Y data can obtain corresponding UV data, reducing image errors.

[0108] In a possible implementation, the acquired movement image may be preprocessed to acquire the first image and the second image. For example, the first image and the second image may be acquired by the following steps:

[0109] Step 1: Obtain a thermal imaging image and a target resolution collected by a first sensor.

[0110] Step 2: Adjust the resolution of the thermal imaging image to the target resolution to obtain a first image.

[0111] Step 3: Acquire a second image captured by a second sensor.

[0112] The target resolution is the same as the resolution of the thermal imaging window. The field of view of the first sensor is within the field of view of the second sensor. In practical applications, the thermal imaging image is displayed in the thermal imaging window, and the second image is displayed on the screen of the terminal device. In this case, the resolution of the thermal imaging image needs to be consistent with the target resolution of the thermal imaging window, and the resolution of the second image needs to be adjusted to be consistent with the resolution of the device screen.

[0113] In one example, the method of the embodiment of the present application can be Fig.10 The process shown is used for image display. First, a thermal imaging image collected by the first sensor is obtained. For example, if the resolution of the thermal imaging image is 640X512 and the resolution of the thermal imaging window is 1180X994, image pre-processing is required to adjust the resolution of the collected thermal imaging image to 1180X994 to obtain the first image. In practical applications, this step can be performed by scaling the resolution of the thermal imaging image to the same resolution as the thermal imaging window through the VPU (Video Processing Unit) of the terminal device.

[0114] After obtaining the scaled thermal imaging image and the image collected by the second sensor, Fig.10 The VGS (Video Graphics Sub-System) of the terminal device shown performs YUV encoding on the scaled thermal imaging image and the image collected by the second sensor, thereby obtaining a first image and a second image. The resolution of the second image is consistent with the resolution of the screen, which is 1920X1080.

[0115] After obtaining the first image and the second image, Fig.10 The Clipon processing module in the terminal device shown uses odd coordinates as the starting pixel coordinates, copies the first image to the second image to obtain a third image, and then sends the third image for display processing, and the display module displays the third image on the screen.

[0116] By applying the method of the embodiment of the present application, the resolution of the thermal imaging image can be adjusted in advance so that the resolution of the thermal imaging image is consistent with that of the thermal imaging window, thereby facilitating the subsequent copying and display of the image.

[0117] In a second aspect of the embodiments of the present application, an image display device is provided, such as Fig.11 As shown, the device includes an image preprocessing module 111, an image processing module 112 and a display module 113.

[0118] The image processing module includes an image first copy submodule 1121, an image alignment submodule 1122 and an image second copy submodule 1123;

[0119] An image preprocessing module, used to obtain a first image and a second image, and to obtain a thermal imaging window position in the second image; wherein the field of view of the first image is within the field of view of the second image, and the first image and the second image are images in YUV format;

[0120] The image first copy submodule is used to copy the Y data portion of the first image to the second image according to the thermal imaging window position when the thermal imaging window position indicates that the starting pixel row and / or starting pixel column of the copy is an odd number;

[0121] The image alignment submodule is used to align the thermal imaging window position according to a preset alignment rule to obtain a UV alignment position;

[0122] The image second copy submodule is used to copy the UV data portion of the first image to the second image to obtain a third image according to the UV alignment position;

[0123] The display module is used to display the third image.

[0124] The device using the embodiment of the present application can copy the Y data portion of the first image to the second image when the thermal imaging window position indicates that the starting pixel row and / or starting pixel column of the copy is an odd number, and then perform UV alignment on the position of the thermal imaging image, so that the starting point of the thermal imaging image can be the position of the odd-numbered point coordinates of the second image, thereby improving the display position accuracy of the thermal imaging window, reducing the error during aiming, and improving the aiming accuracy.

[0125] In a possible implementation, the image first copy submodule includes:

[0126] A starting pixel acquisition unit, specifically used to acquire the starting pixel coordinates (a, b) of the thermal imaging window position when the starting pixel row and / or starting pixel column representing the copy of the thermal imaging window position is an odd number, wherein a and b are both integers, and at least one of a and b is an odd number;

[0127] The image copying unit is specifically used to copy the data at the Y(i, j) position in the first image to the Y(i+a, j+b) position in the second image, where Y(i, j) represents the position of the i-th row and j-th column in the Y data part of the first image.

[0128] By using the device of the embodiment of the present application, the copy range of the Y data of the first image in the second image can be calculated by the coordinate position of the Y data in the first image and the second image, so that the Y data of the first image can be copied to the corresponding position of the second image to achieve the copy of the Y data. And at least one of a and b is an odd number, which can ensure that the starting pixel coordinates of the thermal imaging image are odd coordinates, so that the Y data is copied from the odd coordinate position, thereby improving the display position accuracy of the thermal imaging window, reducing the error during aiming, and improving the aiming accuracy.

[0129] In a possible implementation, the image alignment submodule includes:

[0130] The image alignment unit is specifically used to align the thermal imaging window position by aligning to the left and upward to obtain a UV alignment position.

[0131] In a possible implementation, the image alignment unit includes:

[0132] The first image alignment subunit is specifically used to translate the thermal imaging window position upward by one pixel to obtain a UV alignment position when the thermal imaging window position indicates that the starting pixel row of the copy is an odd number and the starting pixel column is an even number;

[0133] The second image alignment subunit is specifically used to shift the thermal imaging window position to the left by one pixel to obtain a UV alignment position when the thermal imaging window position indicates that the starting pixel row of the copy is an even number and the starting pixel column is an odd number;

[0134] The third image alignment subunit is specifically used to translate the thermal imaging window position one pixel to the left and one pixel upward to obtain the UV alignment position when the thermal imaging window position represents that the starting pixel row of the copy is an odd number and the starting pixel column is an odd number.

[0135] By using the device of the embodiment of the present application, the thermal imaging window position can be UV aligned to the left and / or upward, so that the copied Y data can obtain corresponding UV data, reducing image errors.

[0136] In a possible implementation, the image preprocessing module includes:

[0137] The thermal imaging image acquisition submodule is specifically used to acquire the thermal imaging image collected by the first sensor and the target resolution, wherein the target resolution is the same as the resolution of the thermal imaging window;

[0138] A resolution adjustment submodule, specifically used to adjust the resolution of the thermal imaging image to a target resolution to obtain a first image;

[0139] The second image acquisition submodule is specifically used to acquire a second image collected by a second sensor, wherein the field of view of the first sensor is within the field of view of the second sensor.

[0140] By using the device of the embodiment of the present application, the resolution of the thermal imaging image can be adjusted in advance so that the resolution of the thermal imaging image is consistent with that of the thermal imaging window, thereby facilitating subsequent copying and display of the image.

[0141] The present application also provides an electronic device, such as Fig.12 As shown, including:

[0142] Memory 1201, used for storing computer programs;

[0143] The processor 1202 is used to implement the following steps when executing the program stored in the memory 1201:

[0144] Acquire a first image and a second image, and acquire a thermal imaging window position in the second image; wherein the field of view of the first image is within the field of view of the second image, and the first image and the second image are images in YUV format;

[0145] When the thermal imaging window position indicates that the starting pixel row and / or starting pixel column of the copy is an odd number, copying the Y data portion of the first image to the second image according to the thermal imaging window position;

[0146] Align the thermal imaging window position according to the preset alignment rules to obtain the UV alignment position;

[0147] According to the UV alignment position, the UV data portion of the first image is copied to the second image to obtain a third image, and the third image is displayed.

[0148] In addition, the electronic device may further include a communication bus and / or a communication interface, and the processor 1202, the communication interface, and the memory 1201 communicate with each other via the communication bus. In some possible embodiments, the electronic device may be a sighting device, or a measuring device for surveying, mapping, or observation, etc. In addition to the memory and the processor, its structure may further include a display screen, a first sighting module, and a second sighting module, wherein the display screen is used for image display, the first sighting module is used for acquiring a first image, and the second sighting module is used for acquiring a second image.

[0149] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0150] The communication interface is used for communication between the above electronic device and other devices.

[0151] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0152] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0153] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned image display methods are implemented.

[0154] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any image display method in the above embodiments.

[0155] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0156] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0157] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, and computer-readable storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0158] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. An image display method, characterized in that: The method comprises: Acquire a first image and a second image, and acquire a thermal imaging window position in the second image; wherein the field of view of the first image is within the field of view of the second image, and the first image and the second image are images in YUV format; When the thermal imaging window position indicates that the starting pixel row and / or starting pixel column of the copy is an odd number, copying the Y data portion of the first image to the second image according to the thermal imaging window position; Aligning the thermal imaging window position according to a preset alignment rule to obtain a UV alignment position; According to the UV alignment position, the UV data portion of the first image is copied to the second image to obtain a third image, and the third image is displayed.

2. The method according to claim 1, characterized in that When the starting pixel row and / or starting pixel column indicating the copy at the thermal imaging window position is an odd number, copying the Y data portion of the first image to the second image according to the thermal imaging window position comprises: When the starting pixel row and / or starting pixel column of the thermal imaging window position indicating the copy is an odd number, obtaining the starting pixel coordinate (a, b) of the thermal imaging window position, wherein a and b are both integers, and at least one of a and b is an odd number; The data at position Y(i, j) in the first image is copied to position Y(i+a, j+b) in the second image, where Y(i, j) represents the position of the i-th row and j-th column in the Y data portion of the first image.

3. The method according to claim 1, characterized in that: The step of aligning the thermal imaging window position according to a preset alignment rule to obtain a UV alignment position includes: The thermal imaging window position is aligned in a leftward and upward alignment manner to obtain a UV alignment position.

4. The method according to claim 3, characterized in that The step of aligning the thermal imaging window position in a leftward and upward alignment manner to obtain a UV alignment position includes: When the thermal imaging window position indicates that the starting pixel row of the copy is an odd number and the starting pixel column is an even number, the thermal imaging window position is translated upward by one pixel to obtain a UV alignment position; When the thermal imaging window position indicates that the starting pixel row of the copy is an even number and the starting pixel column is an odd number, the thermal imaging window position is shifted to the left by one pixel to obtain a UV alignment position; When the thermal imaging window position indicates that the starting pixel row of the copy is an odd number and the starting pixel column is an odd number, the thermal imaging window position is translated one pixel to the left and one pixel upward to obtain a UV alignment position.

5. The method according to claim 1, characterized in that The acquiring of the first image and the second image comprises: Acquire a thermal imaging image collected by the first sensor and a target resolution, wherein the target resolution is the same as the resolution of the thermal imaging window; Adjusting the resolution of the thermal imaging image to the target resolution to obtain a first image; A second image captured by a second sensor is acquired, wherein a field of view of the first sensor is within a field of view of the second sensor.

6. An image display device, characterized in that: The device comprises an image preprocessing module, an image processing module and a display module. The image processing module includes an image first copy submodule, an image alignment submodule and an image second copy submodule; The image preprocessing module is used to obtain the first image and the second image, and obtain the position of the thermal imaging window in the second image; wherein the field of view of the first image is within the field of view of the second image, and the first image and the second image are images in YUV format; The image first copy submodule is used to copy the Y data portion of the first image to the second image according to the thermal imaging window position when the thermal imaging window position indicates that the starting pixel row and / or starting pixel column of the copy is an odd number; The image alignment submodule is used to align the thermal imaging window position according to a preset alignment rule to obtain a UV alignment position; The image second copy submodule is used to copy the UV data portion of the first image to the second image according to the UV alignment position to obtain a third image; The display module is used to display the third image.

7. The device according to claim 6, characterized in that The image alignment submodule comprises: The image alignment unit is specifically used to align the thermal imaging window position by aligning leftward and upward to obtain a UV alignment position.

8. The device according to claim 7, characterized in that The image alignment unit comprises: The first image alignment subunit is specifically used to translate the thermal imaging window position upward by one pixel to obtain a UV alignment position when the thermal imaging window position indicates that the starting pixel row of the copy is an odd number and the starting pixel column is an even number; The second image alignment subunit is specifically used to shift the thermal imaging window position to the left by one pixel to obtain a UV alignment position when the thermal imaging window position indicates that the starting pixel row of the copy is an even number and the starting pixel column is an odd number; The third image alignment subunit is specifically used to translate the thermal imaging window position one pixel to the left and one pixel upward to obtain a UV alignment position when the thermal imaging window position indicates that the starting pixel row of the copy is an odd number and the starting pixel column is an odd number.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-5 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images