Image registration and fusion method, device and computer storage medium
By cropping and offset cropping of visible and infrared light images, image registration fusion is achieved, solving the problems of large delay and high resource consumption in the prior art, improving the efficiency and quality of image processing, and suitable for high-scene applications. .
Patent Information
- Application Number
- CN202111271584.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-29
- Publication Date
- 2024-12-20
- Estimated Expiration
- 2041-10-29
AI Technical Summary
The existing image registration and fusion technology has problems such as large delay, high resource consumption and black edges, which is difficult to meet the needs of high-scenario applications.
By acquiring visible and infrared images, cropping and offset cropping are performed separately, small images of preset sizes are obtained, and image registration and fusion amplification are performed, which avoids the delay of DDR cache and RAM resource consumption.
The image fusion without resolution limit is realized, the black edge problem is solved, and the RAM resource consumption is reduced, and the efficiency and quality of image processing is improved.
Smart Images
Figure CN114219840B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing, and in particular to an image registration and fusion method, device and computer storage medium. Background Art
[0002] Image registration is to perform spatial transformation on two or more images acquired by different cameras or under different conditions (illumination, camera position, angle, etc.) so that each image can be geometrically matched. The purpose of image registration is to remove the geometric inconsistency between the image to be registered and the reference image, including translation, rotation and deformation, in preparation for further image processing.
[0003] In the common existing technologies of image registration and fusion, registration adopts block processing and uses DDR for image caching. Due to the inconsistent read and write speeds of DDR, the ISP image processing clock, DDR read and write clock, and fusion processing output clock may all be different. In order to avoid the appearance of faults across frames, all areas involving DDR cache are required to cache at least one frame of data before starting the next step of processing, which will result in a large delay defect. The delay defect is not conducive to the application of higher scenarios and does not have the competitiveness of the product.
[0004] Registration uses stream processing. When there is no DDR cache, the main disadvantage is that more system RAM resources are sacrificed to fill the image edges, and there is also a black edge problem. When using level 1 DDR cache, the problem of excessive RAM resources is solved, but the edge filling effect is not ideal. Summary of the invention
[0005] The main technical problem solved by the present application is to propose an image registration and fusion method, device and computer storage medium, which can well balance the delay, resource consumption and black edge problems in the image processing process.
[0006] In order to solve the above technical problems, a technical solution adopted in the present application is: obtaining a visible light image and an infrared light image of the captured picture; cropping the visible light image to obtain a visible light thumbnail of a preset size; offset cropping the infrared light image to obtain an infrared thumbnail of a preset size; the infrared light thumbnail and the visible light thumbnail are aligned; and the visible light thumbnail and the transformed infrared thumbnail are fused and enlarged to obtain a fused and enlarged image.
[0007] In order to solve the above-mentioned technical problems, another technical solution adopted in the present application is: to provide an image registration and fusion device, including a processor, a memory, an acquisition circuit and a display screen, the acquisition circuit, the memory and the display screen are respectively coupled to the processor, the acquisition circuit is used to connect to a device with an image sensor, the memory stores a computer program, the display screen is used to display the registered and fused image, and the processor is used to execute the computer program to implement the image registration and fusion method as mentioned above.
[0008] In order to solve the above technical problems, another technical solution adopted by the present application is: providing a computer storage medium storing a computer program, wherein the computer program can be executed by a processor to implement the above image registration and fusion method.
[0009] The beneficial effects of the present application are as follows: different from the prior art, the present application realizes a usage scenario without resolution limitation by acquiring large visible light images and infrared light images, crops visible light images into small visible light images of a preset size, and offset crops infrared light images into small infrared light images of the same preset size, thereby solving the problem of black edges occurring during image registration and fusion. Furthermore, there is no need to consume RAM resources to fill in the edges of the image, thereby solving the problem of RAM resource consumption. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:
[0011] Figure 1 It is a schematic flowchart of an embodiment of the image registration and fusion method of the present application;
[0012] Figure 2 is a schematic diagram of an original image and a small image in an example of the image registration and fusion method of the present application;
[0013] Figure 3 This is a flow chart of the image registration and fusion method embodiment of the present application;
[0014] Figure 4 It is a schematic structural block diagram of the first embodiment of the image registration and fusion device of the present application;
[0015] Figure 5 It is a schematic block diagram of the structure of a device with storage function of the present application. DETAILED DESCRIPTION
[0016] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0017] See also Figure 1 , Figure 1FIG. 1 is a flow chart of an embodiment of the image registration and fusion method of the present application. The embodiment includes the following steps.
[0018] S101: Acquire a visible light image and an infrared light image of a captured image.
[0019] The visible light image of the captured image can be obtained by a device with at least a visible light camera sensor, or a device with both a visible light camera sensor and an infrared light sensor, such as a digital camera, a mobile phone, a tablet, a portable device, a computer, or a spherical camera. The infrared image of the captured image can be obtained by a device with at least an infrared light camera sensor, such as an infrared camera, a thermal imager, or a spherical camera.
[0020] In step S101 , a visible light image and an infrared light image of the same original size are acquired.
[0021] Specifically, the resolutions of different types of sensors may be inconsistent. When acquiring visible light images and infrared images, the camera sensor outputs the image with the starting coordinates as the starting point. When the acquisition device acquires visible light images and infrared images, it acquires the effective center map of the visible light images and infrared images, and acquires effective visible light images and infrared images of the same size. By acquiring visible light images and infrared images of the same original size, the problem of inconsistent resolutions of the acquired visible light images and infrared images caused by different resolutions of the camera sensors can be eliminated.
[0022] S102: Crop the visible light image to obtain a visible light thumbnail of a preset size.
[0023] A visible light image of an original size is obtained, and the visible light image is cropped according to a preset size from the center position of the original visible light image to obtain a visible light thumbnail of a preset size.
[0024] After obtaining the visible light thumbnail, preliminary image processing is required for the visible light thumbnail, such as 3A, bad pixel correction, denoising, strong light suppression, backlight compensation, color enhancement and other algorithm processing.
[0025] S103: offset cropping the infrared light image to obtain an infrared light thumbnail of a preset size; the infrared light thumbnail and the visible light thumbnail meet registration requirements.
[0026] Obtain an infrared image of original size, crop the infrared image according to a preset size, and obtain an infrared thumbnail. When cropping the infrared image according to a preset size, the infrared thumbnail of preset size can be cropped from the center of the infrared image, or it can be cropped from other areas offset from the center. Before registering the infrared thumbnail with the visible light thumbnail, it is also necessary to perform preliminary image processing on the infrared thumbnail, and send the infrared thumbnail and the visible light thumbnail that have undergone preliminary image processing to the registration module for image registration to meet the registration requirements.
[0027] Among them, during image registration, the infrared light thumbnail that has undergone preliminary image processing is calculated with the tform matrix calculated from the infrared light thumbnail and the visible light thumbnail, and the registration coordinates are calculated. Then, the pixel values are interpolated to obtain the registered infrared light thumbnail, that is, the infrared light thumbnail and the visible light thumbnail satisfy the registration.
[0028] S104: The visible light thumbnail and the transformed infrared light thumbnail are fused and enlarged to obtain a fused and enlarged image.
[0029] During image fusion, the obtained visible light thumbnail of preset size and the infrared light thumbnail of preset size after transformation are fused to obtain a fused thumbnail of preset size. However, the thumbnail cannot be used as the output target image, and the fused thumbnail of preset size needs to be enlarged to obtain a fused enlarged image.
[0030] Furthermore, the visible light thumbnail and the transformed infrared light thumbnail are fused and enlarged to obtain a fused and enlarged image of the original size.
[0031] The original-size visible light image is cropped to obtain a visible light thumbnail of a preset size, and the original infrared light image is offset and cropped to obtain an infrared thumbnail of a preset size that satisfies the registration with the visible light thumbnail. That is, the visible light thumbnail of the preset size and the infrared thumbnail of the preset size after the transformation are fused. The fused image is still a thumbnail of the preset size. Therefore, before outputting the fused image, it is necessary to enlarge the fused thumbnail to its original size.
[0032] Based on the above embodiments, the present application makes further improvements in each step in order to optimize the computational efficiency in image processing.
[0033] When the visible light image is cropped to obtain a visible light thumbnail of a preset size, the method further includes cropping the visible light image based on the cropping coordinates to obtain a visible light thumbnail of the preset size.
[0034] The cropping coordinates refer to taking any point on the visible light image as the cropping coordinates of the visible light thumbnail, and cropping the visible light thumbnail of a preset size with the cropping coordinates as the starting point to obtain the visible light thumbnail of the preset size.
[0035] When the infrared light image is offset and cropped to obtain an infrared light thumbnail of a preset size, it also includes cropping the infrared light image based on the offset coordinates to obtain an infrared light thumbnail of a preset size; the offset coordinates are based on the preset parameters of the intercepted coordinates offset so that the visible light thumbnail and the infrared light thumbnail meet the registration.
[0036] The interception coordinate refers to taking any coordinate in the infrared image as the interception coordinate of the infrared thumbnail, and using the interception coordinate as the starting point or center to crop the infrared thumbnail of a preset size. The offset coordinate refers to the coordinate offset after the interception coordinate is offset by the preset parameter. The preset parameter refers to a parameter set when offsetting the interception coordinate. The range of the parameter can be plus or minus 30 as the maximum and minimum values, or other ranges of values. For example, when the preset parameter is (-30, +30), the preset parameters such as (0, 0), (2, 3), (-2, 3), (2, -3), (-2, -3) can be taken. When cropping an infrared image, use the offset coordinate as the starting point to crop the infrared thumbnail of a preset size.
[0037] The infrared light image is offset and cropped to obtain an infrared light thumbnail of a preset size; in another embodiment in which the infrared light thumbnail and the visible light thumbnail are aligned, the visible light thumbnail and the infrared light thumbnail are aligned and compared to obtain offset parameters of the infrared light image; and the infrared light image is offset and cropped according to the offset parameters.
[0038] Specifically, the acquired infrared light thumbnail and visible light thumbnail are sent to the online registration device, the offset parameters of the infrared light thumbnail are calculated, and the interception coordinates are offset according to the offset parameters to obtain new offset coordinates, and the infrared light thumbnail of the preset size is cropped with the new offset coordinates as the starting point to obtain the infrared light thumbnail of the new preset size. The initial preset parameters can also be set in advance, and the registration comparison is performed after the transformation. Here, the initial preset parameters can be set based on experience. If the registration comparison result is suitable, no further adjustment is required, which can improve the efficiency of the transformation processing. If the registration comparison result is not suitable, the preset parameters are adjusted according to the registration comparison result.
[0039] In order to better obtain a visible light thumbnail that meets the requirements, the visible light thumbnail and the original visible light image are further limited, and half of the difference between the preset size and the original size is greater than or equal to the maximum value of the preset parameter.
[0040] Half of the difference between the preset size of the visible light thumbnail and the original size of the visible light image is greater than or equal to the maximum value of the preset parameter, which means that the center of the visible light thumbnail coincides with the center of the original visible light image. In the original visible light image, the center point is taken as the origin, and the midlines of the left, right, upper and lower images of the original image about the origin are drawn through the origin. The areas of the original image that are proportional to the periphery of the above midlines and symmetrical relative to the midlines are obtained through the left and right midlines and the upper and lower midlines, respectively. The intersection of the areas obtained according to the two midlines is the preset size. Among them, the area of any side symmetrical to the left and right midlines, the difference between which and half of the difference between the area of any side of the left and right midlines in the original image is greater than or equal to the maximum value of the preset parameter; similarly, the area of any side symmetrical to the upper and lower midlines, the difference between which and half of the difference between the area of any side of the upper and lower midlines in the original image is greater than or equal to the maximum value of the preset parameter.
[0041] Furthermore, the preset parameters include a width offset parameter and a height offset parameter; the visible light image is cropped to obtain a visible light thumbnail of a preset size, including: the center position of the visible light thumbnail is consistent with the center position of the visible light image; the width spacing between the visible light thumbnail and the visible light image is greater than or equal to the maximum value of the width offset parameter, and the height spacing is greater than or equal to the maximum value of the height offset parameter.
[0042] Generally, the size of an image is standard in width and height, and the preset parameters include a width offset parameter and a height offset parameter. The width offset parameter can correspond to the X axis in the coordinate axis, and the height offset parameter can correspond to the Y axis in the coordinate axis. When cropping a visible light thumbnail from a visible light image, the center position of the visible light thumbnail is the same as the center position of the visible light image, and the cropping coordinates are used as the starting point. The size of the cropped visible light thumbnail is greater than or equal to the width offset parameter, and the height spacing is greater than or equal to the height offset parameter.
[0043] For example, if the size of the original image is W*H and the preset size is w*h, the steps for determining the preset size are as follows:
[0044] 1. Determine the maximum width offset parameter as deltax, the maximum height offset parameter as deltay, and obtain the preset size range based on the maximum offset parameter.
[0045] Then the width distance between the small image with size w*h and the image with size W*H is greater than or equal to the maximum value of the width offset parameter, which can be expressed by the formula:
[0046] (Ww) / 2>=Max(deltax);
[0047] The height spacing is greater than or equal to the maximum height offset parameter, which can be expressed as:
[0048] (Hh) / 2>=Max(deltay);
[0049] According to the above two formulas, a range of the width w and height h of the preset size can be obtained.
[0050] 2. Obtain coefficient K based on w and h.
[0051] The coefficient K is a coefficient for obtaining a small image in equal proportion when cropping a valid small image from a valid image. The coefficient is the ratio of the width offset parameter of the original image to the width offset parameter of the preset image, which can be expressed as:
[0052] K>=W / w;
[0053] The height offset parameter of the original image is greater than the height offset parameter of the preset image, which can be expressed by the formula:
[0054] K>=H / h;
[0055] The above two formulas show that the cropping of small images of preset sizes is proportional cropping.
[0056] 3. Determine the final preset size based on the coefficient K.
[0057] Step 2 shows that the coefficient K is also a valid range. Therefore, according to the effective original size and the coefficient K selected in the valid range, the final preset size can be determined, which is expressed as:
[0058] w=W / K;
[0059] h = H / K;
[0060] See also Figure 2 , take the size of the original visible light image as W*H, and the preset size of the visible light thumbnail is, the center of the visible light thumbnail of size w*h coincides with the center of the visible light image of size W*H. For the infrared thumbnail of size w*h, take the offset parameter (deltax, deltay), take any interception coordinate (x, y) in the original infrared image of size W*H, and offset the interception coordinate to obtain the offset coordinate (x-deltax, y-deltay), and use the offset coordinate (x-deltax, y-deltay) as the starting point to crop the infrared thumbnail of size w*h in the infrared image of size W*H. Because the offset parameter satisfies the relationship (Ww) / 2>=deltax, (Hh) / 2>=deltay, the offset infrared thumbnail will not exceed the range of the original infrared image, that is, the image will not have the black edge problem caused by the offset parameter exceeding the image support range of the original size. Furthermore, there is no need to consume RAM resources to solve the black edge problem, thereby reducing the consumption of RAM resources.
[0061] In another embodiment of the present application, the image registration and fusion method also includes determining whether the display quality of the fused and enlarged image meets preset requirements; if not, re-executing the following steps: transforming the infrared light thumbnail so that the infrared light thumbnail and the visible light thumbnail meet the registration; fusing and enlarging the visible light thumbnail and the transformed infrared light thumbnail to obtain a fused and enlarged image.
[0062] Specifically, after the images are registered, fused and enlarged, the quality of the fused and enlarged image is judged to determine whether the image meets the expectations of image processing. If not, new preset parameters can be obtained again, that is, the visible light thumbnail and the infrared thumbnail are re-registered and fused to obtain new preset parameters, and the infrared image is offset and cropped according to the new preset parameters to obtain a new infrared thumbnail, and then the visible light thumbnail and the new infrared thumbnail are fused and enlarged to obtain a fused and enlarged image.
[0063] By adjusting the preset parameters for obtaining the infrared light thumbnail, the image can be registered, fused, and magnified to achieve the best expected value.
[0064] In a specific embodiment of the present application, the original visible light image collected is 3840x2160, and the starting coordinates are (12, 21); the original infrared light image is 3840x2160, and the starting coordinates are (12, 21). Assume that the maximum values of the preset parameters deltax and deltay are plus or minus 30.
[0065] 1. Crop the original visible light image into a small visible light image of preset size:
[0066] 1-1. Preset size constraints for visible light thumbnails: According to the maximum values of the preset parameters and the relationship between the preset size and the original size, (Ww) / 2>=Max(deltax), (Hh) / 2>=Max(deltay), substitute the original size of the visible light image W=3840, H=2160, and obtain the preset size w<=3780, h<=2100.
[0067] 1-2. Calculate the cropping ratio: Since the cropping ratio is equal, W / K <= 3780, H / K <= 2100 are required. Substitute the effective resolution W = 3840, H = 2160, and get K> = 1.02857. Set the K parameter to 1.03.
[0068] 1-3. Calculate the preset size of the visible light thumbnail: Substitute the above cropping parameter K into the visible light thumbnail to obtain w=3840 / 1.03=3728 and h=2160 / 1.03=2097.
[0069] 1-4. Clipping coordinates: (Ww) / 2=(3840-3728) / 2=56, (Hh) / 2=(2160-2097) / 2=31 (round down).
[0070] 1-5. Cropping: According to the above-mentioned cropping coordinates, a small visible light image center image with a size of w=3728, h=2097 is cropped from the visible light collection center image W*H.
[0071] 2. Crop the original infrared image into a small infrared image of preset size:
[0072] 2-1. The preset size of the infrared light thumbnail is the same as that of the visible light thumbnail, w=3728, h=2097.
[0073] 2-2. Offset parameters: The default initial values of deltax and deltay are both 0.
[0074] 2-3. Infrared light thumbnail offset coordinates: The interception coordinates are the same as the visible light thumbnail cropping coordinates by default (56, 31). Since the offset parameters are all 0, the offset coordinates are (56, 31).
[0075] 2-4. Cropping: In the 3840x2160 infrared image, take the cropping coordinates as the starting point and crop a small infrared image with a size of w=3728, h=2097.
[0076] The visible light image center image and the infrared light image are fused, and the fused image size is a 3728x 2097 image. The fused image is enlarged to a size of 3840x2160, and it is determined whether the enlarged fused image meets the expected processing. If not, new offset parameters are recalculated.
[0077] 2-5. Online registration: The visible light thumbnail center image and the infrared thumbnail are registered online to calculate new offset parameters, such as deltax=-20, deltay=10.
[0078] 2-6. Update the offset coordinates of the infrared light thumbnail: When the offset parameters change, the offset coordinates of the infrared light thumbnail also change. The new offset coordinates are 56-(-20)=76, 31-10=20, that is, (76, 20).
[0079] 2-7. Update the infrared light thumbnail: In the original infrared light image, use the new offset coordinates as the starting point and crop the infrared light thumbnail with a size of 3728 x 2097.
[0080] The new infrared small image is registered, and after the registration is satisfied, it is fused with the central image of the visible light small image to obtain a fused small image with a size of 3728x 2097. The fused small image is then enlarged to obtain an enlarged fused image with a size of 3840x2160.
[0081] In one example of the present application, the image registration and fusion method is implemented in FPGA based on the flow mode.
[0082] See also Figure 3 The image registration and fusion method uses stream processing in FPGA. After the visible light image and infrared light image are processed by ISP, they are directly registered. There is no need for DDR cache to do block processing. After the registration is completed, the visible image and infrared light image are directly fused. Through the process of stream processing, the delay of the first-level DDR cache can be avoided, which can adapt to the scenes with high requirements for image delay and improve product competitiveness.
[0083] The image registration and fusion device mentioned above can refer to the image registration and fusion device (not marked) provided in the present application as follows.
[0084] See also Figure 4 The image registration and fusion device described in the electronic device embodiment of the present application includes: a processor 10, an acquisition circuit 20, a memory 30 and a display screen 40. The acquisition circuit 20, the memory 30 and the display screen 40 are respectively coupled to the processor 10, the acquisition circuit 20 is used to connect to a device with an image sensor, the memory 30 stores a computer program, the display screen 40 is used to display the registered and fused image, and the processor 10 is used to execute the computer program to implement the image registration and fusion method described in the above-mentioned image registration and fusion processing method embodiment of the present application.
[0085] The acquisition circuit 20 is used to collect image information with the external camera sensor through the image registration, fusion and transposition of the present embodiment. The image registration, fusion and transposition can obtain visible light images and infrared light images through the acquisition circuit 20. The display screen is used to display the image after image registration, fusion and enlargement. The memory 30 is used to store program data, which can be RAM, ROM, or other types of storage devices.
[0086] The processor 10 is used to control the operation of image registration, fusion and transposition. The processor 10 may be an integrated circuit chip with signal processing capabilities. The processor 10 may also be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array (FPGA) or other programmable logic devices. The general-purpose processor may also be a microprocessor or the processor 10 may also be any conventional processor, etc.
[0087] In the several embodiments provided in the present application, it should be understood that the disclosed image registration fusion transposition and image registration fusion methods can be implemented in other ways. For example, the above-described image registration fusion device embodiments are merely schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling shown or discussed can be an indirect coupling through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0088] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0089] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0090] See also Figure 5 , if the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium 30. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions (computer programs) to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each embodiment of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks or optical disks, and electronic devices such as computers, mobile phones, laptops, tablet computers, cameras, etc. having the above-mentioned storage media.
[0091] The description of the execution process of the computer program in the computer-readable storage medium can refer to the description in the above-mentioned embodiment of the image registration and fusion method of the present application, which will not be repeated here.
[0092] The above description is only a preferred embodiment of the present application and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are also included in the patent protection scope of the present application.
Claims
1. An image registration and fusion method, characterized in that: The image registration and fusion method comprises: Obtaining a visible light image and an infrared light image of the same original size of the captured image; Cropping the visible light image to obtain a visible light thumbnail of a preset size; Offset and crop the infrared light image to obtain an infrared light thumbnail of the preset size; the infrared light thumbnail is registered with the visible light thumbnail; The visible light thumbnail and the transformed infrared light thumbnail are fused and amplified to obtain a fused and amplified image; Wherein, half of the difference between the preset size and the original size is greater than or equal to the maximum value of the preset parameter; The step of cropping the visible light image to obtain a small visible light image of a preset size includes: Cropping the visible light image based on the cropping coordinates to obtain the visible light thumbnail of the preset size, wherein the cropping coordinates refer to coordinates of taking any point on the visible light image as a starting point to crop the visible light thumbnail of the preset size; The step of offsetting and cropping the infrared light image to obtain the infrared light thumbnail of the preset size includes: The infrared light image is cropped based on the offset coordinates to obtain the infrared light thumbnail of the preset size; the offset coordinates are based on the preset parameters of the intercepted coordinates offset so that the visible light thumbnail and the infrared light thumbnail meet the registration, wherein the intercepted coordinates refer to any coordinate in the infrared light image as the starting point or center to crop the infrared light thumbnail of the preset size, and the offset coordinates refer to the coordinate offset after the intercepted coordinates are offset by the preset parameters.
2. The image registration and fusion method according to claim 1, characterized in that: The preset parameters include a width offset parameter and a height offset parameter; Cropping the visible light image to obtain a visible light thumbnail of a preset size includes: The center position of the visible light thumbnail is consistent with the center position of the visible light image; The width spacing between the visible light thumbnail and the visible light image is greater than or equal to the maximum value of the width offset parameter, and the height spacing between the visible light thumbnail and the visible light image is greater than or equal to the maximum value of the height offset parameter.
3. The image registration and fusion method according to claim 1, characterized in that: The step of fusing and amplifying the visible light thumbnail and the transformed infrared light thumbnail to obtain a fused and amplified image includes: The visible light thumbnail and the transformed infrared light thumbnail are fused and enlarged to obtain a fused and enlarged image of the original size.
4. The image registration and fusion method according to claim 1, characterized in that: The infrared light image is offset and cropped to obtain an infrared light thumbnail of a preset size; and the infrared light thumbnail is aligned with the visible light thumbnail, including: Compare and align the visible light image with the infrared light image to obtain an offset parameter of the infrared light image; The infrared light image is offset and cropped according to the offset parameter.
5. The image registration and fusion method according to claim 1, characterized in that: The image registration and fusion method also includes: Determining whether the display quality of the fused and enlarged image meets preset requirements; If not satisfied, repeat the following steps: The infrared light thumbnail is transformed so that the infrared light thumbnail and the visible light thumbnail meet registration requirements; The visible light thumbnail and the transformed infrared light thumbnail are fused and enlarged to obtain a fused and enlarged image.
6. The image registration and fusion method according to claim 1, characterized in that: The image registration and fusion method is implemented in FPGA based on the flow mode.
7. An image registration and fusion device, characterized in that: It includes a processor, a memory, an acquisition circuit and a display screen, wherein the acquisition circuit, the memory and the display screen are respectively coupled to the processor, the acquisition circuit is used to connect to a device with an image sensor, the memory stores a computer program, the display screen is used to display the image after registration and fusion, and the processor is used to execute the computer program to implement the registration and fusion method according to any one of claims 1 to 6.
8. A computer storage medium, characterized in that: A computer program is stored, and the computer program can be executed by a processor to implement the image registration and fusion method as described in any one of claims 1 to 6.