Image fusion method and device

Abstract

The invention relates to the technical field of monitoring, and provides an image fusion method and device. The method comprises the steps that a first image and a second image to be fused are acquired, and the first image and the second image are shot by a first device and a second device respectively; mapping the coordinate of each point location in the first image to a coordinate system of first equipment to obtain a first set; mapping the coordinate of each point location in the second image to a coordinate system of second equipment to obtain a second set; respectively determining feature vectors of all point locations in the first set and the second set; on the basis of the feature vectors of all the point locations in the first set and the second set, the same point locations in the first set and the second set are judged, and a judgment result is obtained; and fusing the first image and the second image based on the judgment result. By adopting the technical means, the problems of inaccurate matching, easy distortion, obvious splicing seams and the like in image merging in the prior art are solved.

Description

Technical Field

[0001] The present application relates to the field of monitoring technology, and in particular to an image fusion method and device. Background Art

[0002] In the fields of panoramic photography, medical imaging, remote sensing, and surveillance, it is often necessary to merge multiple images into a wide-angle image. Currently, multiple images are directly stitched into one image through image stitching. This method has the problem of inaccurate matching, and the stitched image is prone to distortion and may have obvious stitching seams. Summary of the invention

[0003] In view of this, the embodiments of the present application provide an image fusion method, device, electronic device and computer-readable storage medium to solve the problems of inaccurate matching, easy distortion and obvious seams in image merging in the prior art.

[0004] According to a first aspect of an embodiment of the present application, an image fusion method is provided, including: obtaining a first image and a second image to be fused, wherein the first image and the second image are taken by a first device and a second device respectively; mapping the coordinates of each point in the first image to the coordinate system of the first device to obtain a first set; mapping the coordinates of each point in the second image to the coordinate system of the second device to obtain a second set; respectively determining a feature vector of each point in the first set and the second set; based on the feature vectors of each point in the first set and the second set, judging the same points in the first set and the second set to obtain a judgment result; and fusing the first image and the second image based on the judgment result.

[0005] According to a second aspect of an embodiment of the present application, an image fusion device is provided, including: an acquisition module, configured to acquire a first image and a second image to be fused, wherein the first image and the second image are taken by a first device and a second device respectively; a first mapping module, configured to map the coordinates of each point in the first image to the coordinate system of the first device to obtain a first set; a second mapping module, configured to map the coordinates of each point in the second image to the coordinate system of the second device to obtain a second set; a first determination module, configured to determine the feature vectors of each point in the first set and the second set respectively; a second determination module, configured to judge the same points in the first set and the second set based on the feature vectors of each point in the first set and the second set to obtain a judgment result; and a fusion module, configured to fuse the first image and the second image based on the judgment result.

[0006] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0007] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0008] Compared with the prior art, the embodiments of the present application have the following beneficial effects: obtaining the first image and the second image to be fused, wherein the first image and the second image are respectively taken by the first device and the second device; mapping the coordinates of each point in the first image to the coordinate system of the first device to obtain the first set; mapping the coordinates of each point in the second image to the coordinate system of the second device to obtain the second set; determining the feature vectors of each point in the first set and the second set respectively; judging the same points in the first set and the second set based on the feature vectors of each point in the first set and the second set to obtain the judgment result; and fusing the first image and the second image based on the judgment result. The above technical means can solve the problems of inaccurate matching, easy distortion and obvious seams in image merging in the prior art, thereby improving the matching accuracy between merged images, reducing the distortion of the merged image, and weakening the seams in the merged image. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0010] Figure 1 It is a flowchart of an image fusion method provided in an embodiment of the present application;

[0011] Figure 2 is a flowchart of another image fusion method provided in an embodiment of the present application;

[0012] Figure 3 is a structural schematic diagram of an image fusion device provided in an embodiment of the present application;

[0013] Figure 4 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0014] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0015] An image fusion method and device according to an embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0016] Figure 1 It is a flowchart of an image fusion method provided in an embodiment of the present application. Figure 1 The image fusion method can be executed by a computer or a server, or by software on the computer or the server. Figure 1 As shown, the image fusion method includes:

[0017] S101, obtaining a first image and a second image to be fused, wherein the first image and the second image are respectively captured by a first device and a second device;

[0018] S102, mapping the coordinates of each point in the first image to a coordinate system of the first device to obtain a first set;

[0019] S103, mapping the coordinates of each point in the second image to the coordinate system of the second device to obtain a second set;

[0020] S104, determining the feature vector of each point in the first set and the second set respectively;

[0021] S105, based on the feature vectors of each point in the first set and the second set, determine the same points in the first set and the second set to obtain a determination result;

[0022] S106: Fusion the first image and the second image based on the determination result.

[0023] The first device and the second device can be devices such as cameras. The coordinates of each point in the first image and the coordinates of each point in the second image are the coordinates of each point in the target environment, and the target environment refers to the actual environment. For example, if the image to be fused is an image of the target place (target environment), and a coordinate system of the target place is established, then the coordinates of each point in the first image are the coordinates of each point in the target place. The coordinate systems in this application can all be plane rectangular coordinate systems. The first image is taken by the first device, and the second image is taken by the second device. In order to fuse the first image and the second image, it is necessary to map the coordinates of each point in the first image to the coordinate system of the first device, and map the coordinates of each point in the second image to the coordinate system of the second device. The first set contains the coordinates of each point in the first image under the coordinate system of the first device, and the second set contains the coordinates of each point in the second image under the coordinate system of the second device.

[0024] Determine the feature vector of each point in the first set according to the coordinates of each point in the first set. Determine the feature vector of each point in the second set according to the coordinates of each point in the second set. Determine the same points in the first set and the second set based on the feature vectors of each point in the first set and the second set. The judgment result is about the same points in the first set and the second set. Fuse the first image and the second image based on the judgment result. For example, according to the same points, enlarge or reduce the first image and the second image to the same scale; then overlap the same points in the first image and the second image together to stitch the two images.

[0025] According to the technical solution provided in the embodiment of the present application, the first image and the second image to be fused are obtained, wherein the first image and the second image are respectively taken by the first device and the second device; the coordinates of each point in the first image are mapped to the coordinate system of the first device to obtain the first set; the coordinates of each point in the second image are mapped to the coordinate system of the second device to obtain the second set; the feature vectors of each point in the first set and the second set are determined respectively; based on the feature vectors of each point in the first set and the second set, the same points in the first set and the second set are judged to obtain the judgment result; the first image and the second image are fused based on the judgment result. The above technical means can be adopted to solve the problems of inaccurate matching, easy distortion and obvious seams in image merging in the prior art, thereby improving the matching accuracy between merged images, reducing the distortion of the merged image, and weakening the seams in the merged image.

[0026] In an optional embodiment, the first image and the second image are both taken by the same device, taking the first device as an example. The coordinates of each point in the first image are mapped to the coordinate system of the first device to obtain a first set; the coordinates of each point in the second image are mapped to the coordinate system of the first device to obtain a second set; the feature vectors of each point in the first set and the second set are determined respectively; based on the feature vectors of each point in the first set and the second set, the same points in the first set and the second set are judged to obtain a judgment result; the first image and the second image are fused based on the judgment result.

[0027] In an optional embodiment, multiple images can be fused: the coordinates of each point in each image are mapped to the coordinate system of the device that took the image to obtain a coordinate set corresponding to each image; the feature vector of each point in each coordinate set is determined; based on the feature vector of each point in each coordinate set, the same points in any two coordinate sets are judged to obtain a judgment result; and all images are fused based on the judgment result.

[0028] Furthermore, the coordinates of each point in the first image are mapped to the coordinate system of the first device to obtain a first set, including: determining the first intersection point and the second intersection point corresponding to each point in the first image, and determining the coordinates of each point in the target environment and its corresponding first intersection point and second intersection point; determining the pixel length and pixel width of the image taken by the first device; calculating the coordinates of the point in the coordinate system of the first device based on the pixel length, pixel width, and the coordinates of each point and its corresponding first intersection point and second intersection point; and constructing the first set using the coordinates of all points in the first image in the coordinate system of the first device.

[0029] The length and width of the first image can be used as the pixel length and pixel width of the image captured by the first device. According to the pixel length, pixel width, and the coordinates of the first intersection and the second intersection corresponding to each point, the coordinates of the point are mapped to the coordinate system of the first device.

[0030] Furthermore, determining the first intersection point and the second intersection point corresponding to each point in the first image includes: determining the center pointing line and two boundary lines of the first device on the first image; for any point in the first image: drawing an auxiliary line perpendicular to the center pointing line through the point; determining the first intersection point of the auxiliary line with the center pointing line; determining the second intersection point of the auxiliary line with a closer one of the two boundary lines.

[0031] The center pointing line of the first device is the direction facing the first device, which is the exact center of the monitoring angle of view of the first device. The two boundary lines of the first device are the boundary lines of its monitoring angle of view. For the sake of understanding, it can be simply considered that the first device can only monitor the range within the two boundary lines. Beyond the range of the two boundary lines, the first device cannot monitor. The center pointing line and the two boundary lines of the first device are determined by the model and installation position of the device. Draw an auxiliary line perpendicular to the center pointing line through a point. The intersection of the auxiliary line and the center pointing line is taken as the first intersection, and the intersection of the auxiliary line and a boundary line closer to the auxiliary line is taken as the second intersection. The determined first intersection and second intersection are taken as the first intersection and second intersection corresponding to the point.

[0032] Furthermore, for any point in the first image, the coordinates of the point in the coordinate system of the first device are calculated according to the following formula, including:

[0033] (x 1 ,y 1 )=l 1

[0034] (x 2 ,y 2 )=l 2

[0035]

[0036] Among them, (x,y), (x 1 ,y 1 )、(x 2 ,y 2 ) are the point and its corresponding first intersection point l 1 and the second intersection point l 2 The coordinates in the target environment, w is the pixel length, h is the pixel width, It is the coordinate of the point in the coordinate system of the first device.

[0037] In an optional embodiment, the coordinates of each point in the second image are mapped to the coordinate system of the second device to obtain a second set, including: determining the first intersection point and the second intersection point corresponding to each point in the second image, and determining the coordinates of each point in the target environment and its corresponding first intersection point and second intersection point; determining the pixel length and pixel width of the image captured by the second device; calculating the coordinates of the point in the coordinate system of the second device based on the pixel length, pixel width, and the coordinates of each point and its corresponding first intersection point and second intersection point; and constructing the second set using the coordinates of all points in the second image in the coordinate system of the second device.

[0038] In an optional embodiment, determining the first intersection point and the second intersection point corresponding to each point in the second image includes: determining the center pointing line and two boundary lines of the second device on the second image; for any point in the second image: drawing an auxiliary line perpendicular to the center pointing line through the point; determining a first intersection point of the auxiliary line with the center pointing line; determining a second intersection point of the auxiliary line with a closer one of the two boundary lines.

[0039] Furthermore, determining the characteristic vector of each point in the first set includes: determining the horizontal coordinate value, the absolute value of the horizontal coordinate, the vertical coordinate value and the absolute value of the vertical coordinate of each point according to the coordinates of each point in the first set; and constructing the characteristic vector of each point using the horizontal coordinate value, the absolute value of the horizontal coordinate, the vertical coordinate value and the absolute value of the vertical coordinate of each point.

[0040] The coordinates of a point can be positive or negative, so the horizontal coordinate of a point can be determined by the horizontal coordinate value and the absolute value of the horizontal coordinate value as the absolute value of the horizontal coordinate. The vertical coordinate of a point can be determined by the vertical coordinate value and the absolute value of the vertical coordinate value as the absolute value of the vertical coordinate. The horizontal coordinate value, the absolute value of the horizontal coordinate, the vertical coordinate value and the absolute value of the vertical coordinate of a point are taken as a numerical set as the feature vector of the point.

[0041] Determining the feature vector of each point in the second set is similar to this embodiment.

[0042] Furthermore, determining the characteristic vector of each point in the first set includes: for any point in the first set: determining multiple points within a preset range of the point on the first image; determining the horizontal coordinate value, horizontal coordinate absolute value, vertical coordinate value and vertical coordinate absolute value of the point and the multiple points corresponding to it according to the coordinates of the point and the multiple points corresponding to it in the first set; respectively calculating the sum of the horizontal coordinate value, horizontal coordinate absolute value, vertical coordinate value and vertical coordinate absolute value of the point and the multiple points corresponding to it, and obtaining the horizontal coordinate value sum, horizontal coordinate absolute value sum, vertical coordinate value sum and vertical coordinate absolute value sum corresponding to the point; using the horizontal coordinate value sum, horizontal coordinate absolute value sum, vertical coordinate value sum and vertical coordinate absolute value sum corresponding to the point, constructing the characteristic vector of the point.

[0043] The abscissa values ​​of the point and its corresponding multiple points are summed as the abscissa value sum corresponding to the point. The abscissa absolute value sum, ordinate value sum and ordinate absolute value sum corresponding to the point are similar to the abscissa value sum corresponding to the point. The abscissa value sum, abscissa absolute value sum, ordinate value sum and ordinate absolute value sum corresponding to a point are taken as a numerical set as the feature vector of the point.

[0044] Determining the feature vector of each point in the second set is similar to this embodiment.

[0045] Furthermore, based on the feature vectors of each point in the first set and the second set, the same points in the first set and the second set are judged to obtain a judgment result, including: for any point in the first set and any point in the second set: calculating the variance value between the feature vectors of the two points; when the variance value is less than a preset threshold, determining that the two points are the same points.

[0046] The feature vector of a point is a set of four values, so the variance between the feature vectors of two points can be calculated. When the variance is less than the preset threshold, the two points are determined to be the same; otherwise, the two points are determined to be not the same.

[0047] Figure 2 is a flow chart of another image fusion method provided in an embodiment of the present application, such as Figure 2 As shown:

[0048] S201, determining a central pointing line and two boundary lines of a first device on a first image;

[0049] S202, for any point in the first image: draw an auxiliary line perpendicular to the central pointing line through the point, determine a first intersection point between the auxiliary line and the central pointing line, and determine a second intersection point between the auxiliary line and a more adjacent boundary line of the two boundary lines;

[0050] S203, determining the coordinates of each point in the target environment and the first intersection point and the second intersection point corresponding thereto;

[0051] S204, determining the pixel length and pixel width of the image captured by the first device;

[0052] S205, calculating the coordinates of the point in the coordinate system of the first device according to the pixel length, the pixel width, the coordinates of each point and the first intersection and the second intersection corresponding to the point;

[0053] S206: Construct a first set using the coordinates of all points in the first image in the coordinate system of the first device.

[0054] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0055] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0056] Figure 3 Schematic diagram of an image fusion device provided in an embodiment of the present application. Figure 3 As shown, the image fusion device comprises:

[0057] An acquisition module 301 is configured to acquire a first image and a second image to be fused, wherein the first image and the second image are respectively captured by a first device and a second device;

[0058] A first mapping module 302 is configured to map the coordinates of each point in the first image to a coordinate system of the first device to obtain a first set;

[0059] The second mapping module 303 is configured to map the coordinates of each point in the second image to the coordinate system of the second device to obtain a second set;

[0060] A first determination module 304 is configured to determine the feature vectors of each point in the first set and the second set respectively;

[0061] The second determination module 305 is configured to determine the same points in the first set and the second set based on the feature vectors of the points in the first set and the second set, and obtain a determination result;

[0062] The fusion module 306 is configured to fuse the first image and the second image based on the determination result.

[0063] According to the technical solution provided in the embodiment of the present application, the first image and the second image to be fused are obtained, wherein the first image and the second image are respectively taken by the first device and the second device; the coordinates of each point in the first image are mapped to the coordinate system of the first device to obtain the first set; the coordinates of each point in the second image are mapped to the coordinate system of the second device to obtain the second set; the feature vectors of each point in the first set and the second set are determined respectively; based on the feature vectors of each point in the first set and the second set, the same points in the first set and the second set are judged to obtain the judgment result; the first image and the second image are fused based on the judgment result. The above technical means can be adopted to solve the problems of inaccurate matching, easy distortion and obvious seams in image merging in the prior art, thereby improving the matching accuracy between merged images, reducing the distortion of the merged image, and weakening the seams in the merged image.

[0064] In some embodiments, the fusion module 306 is configured to map the coordinates of each point in the first image to the coordinate system of the first device to obtain a first set; map the coordinates of each point in the second image to the coordinate system of the first device to obtain a second set; determine the feature vectors of each point in the first set and the second set respectively; based on the feature vectors of each point in the first set and the second set, judge the same points in the first set and the second set to obtain a judgment result; and fuse the first image and the second image based on the judgment result.

[0065] In some embodiments, the fusion module 306 is configured to map the coordinates of each point in each image to the coordinate system of the device that took the image to obtain a coordinate set corresponding to each image; determine the feature vector of each point in each coordinate set; based on the feature vector of each point in each coordinate set, judge the same points in any two coordinate sets to obtain a judgment result; and fuse all images based on the judgment result.

[0066] In some embodiments, the first mapping module 302 is configured to determine the first intersection point and the second intersection point corresponding to each point in the first image, and determine the coordinates of each point in the target environment and its corresponding first intersection point and second intersection point; determine the pixel length and pixel width of the image captured by the first device; calculate the coordinates of the point in the coordinate system of the first device based on the pixel length, pixel width, and the coordinates of each point and its corresponding first intersection point and second intersection point; and construct a first set using the coordinates of all points in the first image in the coordinate system of the first device.

[0067] In some embodiments, the first mapping module 302 is configured to determine the center pointing line and two boundary lines of the first device on the first image; for any point in the first image: draw an auxiliary line perpendicular to the center pointing line through the point; determine a first intersection of the auxiliary line and the center pointing line; determine a second intersection of the auxiliary line and a closer one of the two boundary lines.

[0068] In some embodiments, the first mapping module 302 is configured to calculate the coordinates of the point in the coordinate system of the first device according to the following formula, including:

[0069] (x 1 ,y 1 )=l 1

[0070] (x 2 ,y 2 )=l 2

[0071]

[0072] Among them, (x,y), (x 1 ,y 1 )、(x 2 ,y 2 ) are the point and its corresponding first intersection point l 1 and the second intersection point l 2 The coordinates in the target environment, w is the pixel length, h is the pixel width,

[0073] (xx 1 ×w,yy1 ×h)

[0074] x 2 -x 1 y 2 -y 1 It is the coordinate of the point in the coordinate system of the first device.

[0075] In some embodiments, the second mapping module 303 is configured to determine the first intersection and the second intersection corresponding to each point in the second image, and determine the coordinates of each point in the target environment and its corresponding first intersection and second intersection; determine the pixel length and pixel width of the image captured by the second device; calculate the coordinates of the point in the coordinate system of the second device based on the pixel length, pixel width, and the coordinates of each point and its corresponding first intersection and second intersection; and construct a second set using the coordinates of all points in the second image in the coordinate system of the second device.

[0076] In some embodiments, the second mapping module 303 is configured to determine the center pointing line and two boundary lines of the second device on the second image; for any point in the second image: draw an auxiliary line perpendicular to the center pointing line through the point; determine a first intersection of the auxiliary line and the center pointing line; determine a second intersection of the auxiliary line and a closer one of the two boundary lines.

[0077] In some embodiments, the first determination module 304 is configured to determine the horizontal coordinate value, the absolute value of the horizontal coordinate, the vertical coordinate value and the absolute value of the vertical coordinate of each point according to the coordinates of each point in the first set; and construct a feature vector of each point using the horizontal coordinate value, the absolute value of the horizontal coordinate, the vertical coordinate value and the absolute value of the vertical coordinate of each point.

[0078] In some embodiments, the first determination module 304 is configured to: for any point in the first set: determine multiple points within a preset range of the point on the first image; determine the horizontal coordinate value, horizontal coordinate absolute value, vertical coordinate value and vertical coordinate absolute value of the point and the multiple points corresponding to it according to the coordinates of the point and the multiple points corresponding to it in the first set; respectively calculate the sum of the horizontal coordinate value, horizontal coordinate absolute value, vertical coordinate value and vertical coordinate absolute value of the point and the multiple points corresponding to it, and obtain the horizontal coordinate value sum, horizontal coordinate absolute value sum, vertical coordinate value sum and vertical coordinate absolute value sum corresponding to the point; use the horizontal coordinate value sum, horizontal coordinate absolute value sum, vertical coordinate value sum and vertical coordinate absolute value sum corresponding to the point to construct a feature vector of the point.

[0079] In some embodiments, the first determination module 304 is configured to: for any point in the first set and any point in the second set: calculate the variance value between the feature vectors of the two points; when the variance value is less than a preset threshold, determine that the two points are the same point.

[0080] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0081] Figure 4 Schematic diagram of an electronic device 4 provided in an embodiment of the present application. Figure 4 As shown, the electronic device 4 of this embodiment includes: a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. When the processor 401 executes the computer program 403, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 401 executes the computer program 403, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0082] The electronic device 4 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 4 may include, but is not limited to, a processor 401 and a memory 402. Those skilled in the art will appreciate that Figure 4 The electronic device 4 is merely an example and does not limit the electronic device 4 , and may include more or less components than those shown in the figure, or different components.

[0083] The processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0084] The memory 402 may be an internal storage unit of the electronic device 4, for example, a hard disk or memory of the electronic device 4. The memory 402 may also be an external storage device of the electronic device 4, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 4. The memory 402 may also include both an internal storage unit and an external storage device of the electronic device 4. The memory 402 is used to store computer programs and other programs and data required by the electronic device.

[0085] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0086] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0087] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An image fusion method, characterized in that: include: Acquire a first image and a second image to be fused, wherein the first image and the second image are respectively captured by a first device and a second device; Mapping the coordinates of each point in the first image to the coordinate system of the first device to obtain a first set; Mapping the coordinates of each point in the second image to the coordinate system of the second device to obtain a second set; Determine the feature vector of each point in the first set and the second set respectively; Based on the feature vectors of each point in the first set and the second set, determine the same points in the first set and the second set to obtain a determination result; The first image and the second image are fused based on the judgment result.

2. The method according to claim 1, characterized in that: Mapping the coordinates of each point in the first image to the coordinate system of the first device to obtain a first set includes: Determine a first intersection point and a second intersection point corresponding to each point in the first image, and determine coordinates of each point in the target environment and the first intersection point and the second intersection point corresponding to each point; Determining a pixel length and a pixel width of an image captured by the first device; Calculate the coordinates of the point in the coordinate system of the first device according to the pixel length, the pixel width, the coordinates of each point and the first intersection point and the second intersection point corresponding to the point; A first set is constructed using the coordinates of all points in the first image in the coordinate system of the first device.

3. The method according to claim 1, characterized in that: Determining a first intersection point and a second intersection point corresponding to each point in the first image includes: Determine a central pointing line and two boundary lines of the first device on the first image; For any point in the first image: Draw an auxiliary line through the point perpendicular to the central pointing line; Determine a first intersection point between the auxiliary line and the center pointing line; A second intersection point between the auxiliary line and a more adjacent one of the two boundary lines is determined.

4. The method according to claim 2, characterized in that: For any point in the first image, the coordinates of the point in the coordinate system of the first device are calculated according to the following formula, including: (x1,y1)=l1 (x2,y2)=l2 Wherein, (x, y), (x1, y1), (x2, y2) are the coordinates of the point and its corresponding first intersection point l1 and second intersection point l2 in the target environment, w is the pixel length, h is the pixel width, is the coordinate of the point in the coordinate system of the first device.

5. The method according to claim 1, characterized in that Determining the feature vector of each point in the first set includes: According to the coordinates of each point in the first set, determine the abscissa value, abscissa absolute value, ordinate value and ordinate absolute value of each point; The feature vector of each point is constructed using the horizontal coordinate value, horizontal coordinate absolute value, vertical coordinate value and vertical coordinate absolute value of each point.

6. The method according to claim 1, characterized in that Determining the feature vector of each point in the first set includes: For any point in the first set: Determine a plurality of points within a preset range of the point on the first image; Determine, according to the coordinates of the point and the multiple points corresponding to it in the first set, a horizontal coordinate value, a horizontal coordinate absolute value, a vertical coordinate value and a vertical coordinate absolute value of the point and the multiple points corresponding to it; Calculate the abscissa value, abscissa absolute value, ordinate value and sum of ordinate absolute value of the point and its corresponding multiple points respectively, and obtain the abscissa value sum, abscissa absolute value sum, ordinate value sum and ordinate absolute value sum corresponding to the point; The feature vector of the point is constructed using the sum of the horizontal coordinate values, the sum of the absolute values ​​of the horizontal coordinates, the sum of the vertical coordinate values ​​and the sum of the absolute values ​​of the vertical coordinates corresponding to the point.

7. The method according to claim 1, characterized in that Based on the feature vectors of each point in the first set and the second set, determining the same points in the first set and the second set to obtain a determination result includes: For any point in the first set and any point in the second set: Calculate the variance between the eigenvectors of two points; When the variance value is less than a preset threshold, it is determined that the two points are the same point.

8. An image fusion device, characterized in that: include: An acquisition module is configured to acquire a first image and a second image to be fused, wherein the first image and the second image are respectively captured by a first device and a second device; A first mapping module is configured to map the coordinates of each point in the first image to a coordinate system of the first device to obtain a first set; A second mapping module is configured to map the coordinates of each point in the second image to a coordinate system of the second device to obtain a second set; A first determination module is configured to determine a feature vector of each point in the first set and the second set respectively; A second determination module is configured to determine the same points in the first set and the second set based on the feature vectors of each point in the first set and the second set, and obtain a determination result; A fusion module is configured to fuse the first image and the second image based on the judgment result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images