Image fusion method, target feature recognition method, device and electronic equipment

Abstract

The present application relates to an image fusion method, a target feature recognition method, a device and an electronic device. The image fusion method includes: acquiring a visible light image and an infrared image; performing target recognition on the visible light image to determine at least one first area; performing target recognition on the infrared image to determine at least one second area; performing regional synchronization on the first area of ​​the visible light image and the second area of ​​the infrared image to determine the target area and the non-target area; performing regional fusion on the visible light image and the infrared image, and in the regional fusion process, the target area is fused based on a first fusion method, and the non-target area is fused based on a second fusion method to obtain a fused image. Through the present application, not only the fusion image quality of the target area is improved, but also the color feature loss of the non-target area can be avoided, and the target feature recognition rate of the subsequent fused image is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image fusion method, a target feature recognition method, a device and an electronic device. Background Art

[0002] Image fusion is the process of fusing two or more images from the same scene but with different properties into an image with more complete and comprehensive information for subsequent image analysis, target recognition and tracking.

[0003] As the security market continues to expand, facing different targets, user needs, diverse scenes and the increased adaptability of intelligent capture targets, the industry mainly adjusts the image fusion effect by corresponding different image parameters and program logic for different scenes. However, the existing fusion algorithm is not targeted, and the regional division is not accurate enough, which easily leads to the loss of local color of the target after image fusion, resulting in subsequent intelligent recognition errors of target features and reduced recognition rate. Summary of the invention

[0004] The embodiments of the present application provide an image fusion method, a target feature recognition method, a device and an electronic device to at least solve the problem of low recognition rate after image fusion in the related art.

[0005] In a first aspect, an embodiment of the present application provides an image fusion method, comprising:

[0006] Acquire visible light images and infrared images;

[0007] Performing target recognition on the visible light image to determine at least one first area;

[0008] Performing target recognition on the infrared image to determine at least one second area;

[0009] Performing regional synchronization on the first region of the visible light image and the second region of the infrared image to determine a target region and a non-target region;

[0010] The visible light image and the infrared image are regionally fused. During the regional fusion process, the target area is fused based on a first fusion method, and the non-target area is fused based on a second fusion method to obtain a fused image.

[0011] In some embodiments, acquiring the infrared image includes:

[0012] Identify the target type corresponding to the photographed target;

[0013] Based on the target type and a preset priori shooting index, determining an infrared fill light band corresponding to the shooting target;

[0014] An infrared image corresponding to the photographed target is acquired based on the infrared fill light band.

[0015] In some embodiments, performing regional synchronization on the first area of ​​the visible light image and the second area of ​​the infrared image to determine the target area and the non-target area includes:

[0016] Performing data frame synchronization processing on the visible light image and the infrared image;

[0017] Based on the coordinate information of the first area of ​​the visible light image and the second area of ​​the infrared image after the data frame synchronization processing, the first area and the second area are mapped to each other to obtain a target area;

[0018] A non-target area is determined according to the target area.

[0019] In some embodiments, a fusion ratio corresponding to the first fusion mode is smaller than a fusion ratio corresponding to the second fusion mode, and the fusion ratio is a pixel value fusion ratio of a visible light image and an infrared image.

[0020] In some embodiments, fusing the target region based on the first fusion mode during the region fusion process includes:

[0021] Obtaining the target type corresponding to the target area;

[0022] Based on the target type and a preset priori fusion index, determining a first fusion ratio corresponding to the target area;

[0023] The target area is fused based on the first fusion ratio.

[0024] In some embodiments, fusing the non-target area based on the second fusion mode during the area fusion process includes:

[0025] Determining a second fusion ratio corresponding to the non-target area;

[0026] Non-target area fusion is performed based on the second fusion ratio.

[0027] In a second aspect, an embodiment of the present application provides a target feature recognition method, comprising:

[0028] Perform image fusion based on the image fusion method described in the first aspect to determine a fused image;

[0029] Target feature recognition is performed based on the fused image.

[0030] In a third aspect, an embodiment of the present application provides an image fusion device, including:

[0031] An image acquisition unit, used for acquiring visible light images and infrared images;

[0032] A first region determining unit, configured to perform target recognition on the visible light image to determine at least one first region;

[0033] A second region determination unit, configured to perform target recognition on the infrared image and determine at least one second region;

[0034] A region synchronization unit, used for performing region synchronization on the first region of the visible light image and the second region of the infrared image to determine a target region and a non-target region;

[0035] The image fusion unit is used to perform regional fusion on the visible light image and the infrared image. During the regional fusion process, the target area is fused based on a first fusion method, and the non-target area is fused based on a second fusion method to obtain a fused image.

[0036] In a fourth aspect, an embodiment of the present application provides a target feature recognition device, including:

[0037] A fused image determining unit, configured to perform image fusion based on the image fusion method described in the first aspect to determine a fused image;

[0038] The target feature recognition unit is used to perform target feature recognition based on the fused image.

[0039] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the image fusion method described in the first aspect or the target feature recognition method described in the second aspect is implemented.

[0040] Compared with the related art, the image fusion method, target feature recognition method, device and electronic device provided in the embodiments of the present application determine at least one first area by performing target recognition on the visible light image; determine at least one second area by performing target recognition on the infrared image; perform regional synchronization on the first area of ​​the visible light image and the second area of ​​the infrared image to determine the target area and the non-target area; perform regional fusion on the visible light image and the infrared image, and in the regional fusion process, fuse the target area based on the first fusion method, and fuse the non-target area based on the second fusion method to obtain a fused image, thereby taking intelligent target recognition as the starting point and cleverly utilizing the regional characteristics obtained by target recognition to perform regional division, which not only improves the fused image quality of the target area, but also avoids the loss of color features in the non-target area, thereby improving the target feature recognition rate of subsequent fused images.

[0041] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0043] Figure 1 is a flowchart of an image fusion method in one of the embodiments of the present application;

[0044] Figure 2 is a schematic diagram of a process for obtaining an infrared image in one of the embodiments of the present application;

[0045] Figure 3 is a schematic diagram of a process for determining a target area and a non-target area in one embodiment of the present application;

[0046] Figure 4 is a schematic diagram of a process of fusing the target area based on a first fusion method during the area fusion process in one of the embodiments of the present application;

[0047] Figure 5 is a structural block diagram of an image fusion device in one of the embodiments of the present application;

[0048] Figure 6 This is a structural block diagram of a target feature recognition device in one of the embodiments of the present application;

[0049] Figure 7 It is a schematic diagram of the structure of an electronic device in one of the embodiments of the present application.

[0050] Description of the drawings: 301, image acquisition unit; 302, first area determination unit; 303, second area determination unit; 304, area synchronization unit; 305, image fusion unit; 401, fused image determination unit; 402, target feature recognition unit; 50, bus; 51, processor; 52, memory; 53, communication interface. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0052] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.

[0053] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0054] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0055] Image fusion technology is developed from information fusion and is an advanced image processing technology that integrates multiple source images. Among them, multi-source image fusion is to perform appropriate fusion processing on multiple source images of the same scene or target collected by multiple sensors to obtain a more accurate, comprehensive and reliable image description of the same scene. In general, due to the influence of factors such as lighting, environmental conditions, target state, target position and inherent characteristics of sensors, the image information obtained by different sensors is different, and the image information obtained by the same sensor at different times and locations is also different. Image fusion can make full use of the redundant information and complementary information contained in multiple images to be fused, and synthesize a new image that meets certain needs based on a certain fusion algorithm. The fused image is more in line with the visual characteristics of people or machines, which is conducive to further analysis of the image, detection, recognition or tracking of targets.

[0056] The image fusion method provided in this embodiment can be applied to any fusion process of infrared images and visible light images with the same or different viewing angles, and this application is not limited thereto. The following is an example of the acquisition steps of a single-frame fusion image.

[0057] This embodiment provides an image fusion method, which can be applied to electronic devices. Figure 1 is a flow chart of an image fusion method according to an embodiment of the present application, such as Figure 1 As shown, the process includes the following steps:

[0058] Step S101, acquiring a visible light image and an infrared image.

[0059] In this embodiment, the electronic device may be an image acquisition device; or it may be an image processing device that does not have an image acquisition function but is communicatively connected to the image acquisition device and can receive and process images acquired by the image acquisition device.

[0060] In this embodiment, the image acquisition device includes but is not limited to a camera. Specifically, a visible light image can be acquired by a visible light camera, and an infrared image can be acquired by an infrared camera; or an infrared imager and other devices can be used to acquire infrared images and visible light images at the same time. The method for acquiring infrared images and visible light images in this embodiment can adopt the same method as in the prior art, and will not be described in detail here.

[0061] Step S102: performing target recognition on the visible light image to determine at least one first area.

[0062] Step S103: performing target recognition on the infrared image to determine at least one second area.

[0063] In this embodiment, it is necessary to divide the visible light image and the infrared image into regions, so that different regions of the visible light image and the infrared image can be fused separately during the image fusion process.

[0064] In this embodiment, the first area where the target object is located can be determined by performing target recognition on the visible light image. Optionally, the first area identifies a rectangular area in the visible light image where the target object is located, which can be obtained by calibration using a labeling tool, and the rectangular area covers the target object. More preferably, the first area is obtained by performing edge detection on the identified target object, traversing each pixel on the image, and connecting all detected edge points, which is not limited in this embodiment. Specifically, target recognition based on machine vision, target recognition based on deep learning, etc. can be adopted, and a suitable target detection algorithm can be selected according to actual needs to improve the accuracy and efficiency of target recognition. Intelligent division of the area based on the target recognition result makes the division result of the first area more accurate, and the complete color features and texture features of the target object can be retained to the greatest extent, thereby improving the image fusion effect and reducing the difficulty of feature extraction of the fused image. In some embodiments, when a target object is obtained by identifying the visible light image, a corresponding first area can be determined; in other embodiments, when two or more target objects are obtained by identifying the visible light image, multiple corresponding first areas can be determined.

[0065] Similarly, by performing target recognition on the infrared image, the second area where the target object is located is determined.

[0066] Step S104 , performing regional synchronization on the first region of the visible light image and the second region of the infrared image to determine a target region and a non-target region.

[0067] In this embodiment, the first region and the second region can be matched by registering the visible light image and the infrared image, and the registration method can be a calibration method, a feature point method, etc. After the regions are synchronized, in the visible light image, the target region is the first region, and the non-target region is the other region; in the infrared image, the target region is the second region, and the non-target region is the other region.

[0068] Step S105 , performing regional fusion on the visible light image and the infrared image. In the regional fusion process, the target area is fused based on a first fusion method, and the non-target area is fused based on a second fusion method to obtain a fused image.

[0069] Since infrared images can record the infrared radiation information of the target, they have better detection performance for thermal targets, but they are not sensitive to changes in background brightness and have low imaging resolution, which is not conducive to human eye interpretation. Visible light images have higher resolution and can provide comprehensive edge texture and detail information, but are not sensitive to target information.

[0070] In this embodiment, by adopting different fusion modes for the target area and the non-target area to perform regional fusion, the different working characteristics of visible light and infrared light can be fully utilized to complement each other and improve the image fusion quality of the target area.

[0071] In summary, the image fusion method, target feature recognition method, device and electronic device provided in the embodiments of the present application determine at least one first area by performing target recognition on the visible light image; determine at least one second area by performing target recognition on the infrared image; perform regional synchronization on the first area of ​​the visible light image and the second area of ​​the infrared image to determine the target area and the non-target area; perform regional fusion on the visible light image and the infrared image, and in the regional fusion process, fuse the target area based on the first fusion method, and fuse the non-target area based on the second fusion method to obtain a fused image, thereby taking intelligent target recognition as the starting point and cleverly utilizing the regional characteristics obtained by target recognition to perform regional division, which not only improves the fused image quality of the target area, but also avoids the loss of color features in the non-target area, thereby improving the target feature recognition rate of subsequent fused images.

[0072] The embodiments of the present application are described and illustrated below through preferred embodiments.

[0073] like Figure 2 As shown, based on the above embodiments, in some embodiments, acquiring the infrared image includes:

[0074] Step S1011, identifying the target type corresponding to the photographed target.

[0075] In this embodiment, the shooting targets are different in different shooting scenes. The target types corresponding to the shooting targets include but are not limited to faces, vehicles, license plates, animals, buildings, plants, etc. The target type recognition method can adopt the existing technology, and this application will not elaborate on it here.

[0076] Step S1012: determining an infrared fill light band corresponding to the photographed target based on the target type and a preset priori photographing index.

[0077] In this embodiment, different target types correspond to different material surfaces, and different material surfaces have different transmittances, refractive indices, and dispersions for infrared rays of different wavelengths, and therefore have different infrared reflection effects in different infrared bands. An infrared fill light band that matches different target types can be established through a priori shooting index, so that the infrared fill light band corresponding to the shooting target can be automatically determined based on the target type and the preset priori shooting index. Among them, the priori shooting index can be established based on a large amount of survey data and experimental data. It can be understood that the infrared fill light band that matches different target types can be a wavelength value with the best reflection effect, or it can be a better band interval, and the present application is not limited here.

[0078] For example, in the a-band (750nm), the font and background of the license plate can be clearly distinguished, and in the b-band (850nm), the grayscale of the face is more layered. Correspondingly, the priori shooting index includes: the infrared fill light band corresponding to the target type vehicle is the a-band (750nm); the infrared fill light band corresponding to the target type face is the b-band (850nm).

[0079] Step S1013: acquiring an infrared image corresponding to the photographed target based on the infrared fill light band.

[0080] In this embodiment, after determining the infrared fill light band corresponding to the shooting target according to the target type corresponding to the shooting target, the corresponding infrared fill light band can be adaptively switched to obtain the corresponding infrared image, and the infrared reflection effect of the shooting target in the acquired infrared image is optimal. In some embodiments, dual infrared fill light lamps, variable wavelength infrared lamps or multiple mixed light fill light lamps can be used to switch between different infrared fill light bands.

[0081] like Figure 3 As shown, based on the above embodiments, in some embodiments, performing regional synchronization on the first area of ​​the visible light image and the second area of ​​the infrared image, and determining the target area and the non-target area includes:

[0082] Step S1041, performing data frame synchronization processing on the visible light image and the infrared image. Specifically, the visible light image and the infrared image can be mapped to each other by means of inter-frame dual-path data synchronization, so as to ensure that the visible light image and the infrared image are at the same time before image fusion.

[0083] Step S1042, based on the coordinate information of the first area of ​​the visible light image and the second area of ​​the infrared image obtained after the data frame synchronization processing, the first area and the second area are mapped to each other to obtain a target area.

[0084] Step S1043: determining a non-target area according to the target area.

[0085] In this embodiment, after data frame synchronization processing is performed on the visible light image and the infrared image, the coordinate information of the first area of ​​the visible light image is matched with the coordinate information of the second area of ​​the infrared image, and the matched area is determined as the target area, and the other areas are non-target areas.

[0086] like Figure 4 As shown, based on the above embodiments, in some of the embodiments, fusing the target region based on the first fusion mode during the region fusion process includes:

[0087] Step S1051, obtaining the target type corresponding to the target area. Specifically, target recognition is performed on the target area to obtain the corresponding target type. Target types include but are not limited to faces, vehicles, license plates, animals, buildings, plants, etc.

[0088] Step S1052: determining a first fusion ratio corresponding to the target area based on the target type and a preset priori fusion index.

[0089] In the related art, when images are fused, the entire image uses the same fusion ratio biased towards infrared, but this will cause some color features of the target object in the target area of ​​the fused image to be lost.

[0090] In this embodiment, local area fusion is adopted, and in the area fusion process, the target area is fused based on the first fusion method, and the non-target area is fused based on the second fusion method. The fusion ratio is the pixel value fusion ratio of the visible light image and the infrared image.

[0091] Different fusion effects can be produced for different target types at different fusion ratios. In order to achieve adaptive switching of different fusion ratios under different target types, a fusion ratio matching different target types can be established through a priori fusion index, so that the first fusion ratio corresponding to the target type can be automatically determined according to the target type and the preset priori fusion index. The priori fusion index can be established based on a large amount of survey data and experimental data.

[0092] Step S1053: performing target area fusion based on the first fusion ratio.

[0093] In this embodiment, after the first fusion ratio is determined, local ratio fusion is performed on the target area, that is, the product of the visible light pixel value of the target area and the first fusion ratio is used as the fusion result of the target area.

[0094] Based on the above embodiments, in some embodiments, fusing the non-target area based on the second fusion mode during the area fusion process includes:

[0095] Step S1054: determining a second fusion ratio corresponding to the non-target area.

[0096] Step S1055: performing non-target area fusion based on the second fusion ratio.

[0097] In this embodiment, the fusion ratio corresponding to the first fusion mode is smaller than the fusion ratio corresponding to the second fusion mode. That is, the first fusion ratio biased towards infrared is used in the target area to improve the infrared image fusion ratio of the target area, and the second fusion ratio biased towards visible light is used in the non-target area. Through the above steps, the image fusion quality of the target area can be improved, while the color feature loss of the non-target area can be avoided to the greatest extent.

[0098] Exemplarily, image fusion may be performed as shown in the following formula: Wherein, F is the fusion output, Froi is the first fusion mode corresponding to the target area, Fout is the second fusion mode corresponding to the non-target area, W1 is the first fusion ratio, and W2 is the second fusion ratio.

[0099] F=Froi(W1)+Fout(W2)

[0100] For example, the first fusion ratio is 0.3 / 0.7=3 / 7, that is, in the target area, the visible light image and the infrared image are fused by multiplying the pixel value of the visible light image by 0.3 plus the pixel value of the infrared image by 0.7; the second fusion ratio is 0.6 / 0.4=3 / 2, that is, in the non-target area, the visible light image and the infrared image are fused by multiplying the pixel value of the visible light image by 0.6 plus the pixel value of the infrared image by 0.4.

[0101] This embodiment also provides a target feature recognition method, which is characterized by comprising the following steps:

[0102] Step S201 : performing image fusion based on the image fusion method described in the above embodiment to determine a fused image.

[0103] Step S201, performing target feature recognition based on the fused image.

[0104] In this embodiment, the specific implementation principle and technical effect of the method for determining the fused image are the same as those in the above embodiment, and for the sake of brief description, this embodiment will not be described in detail. Then, a method such as target recognition based on deep learning can be used to perform target feature recognition.

[0105] Through the above steps, the target area and the corresponding target type characteristics obtained by regional division through intelligent target recognition are cleverly utilized to perform multi-band infrared switching and regional fusion to obtain a fused image, thereby improving the target feature recognition rate of the fused image in various target recognition scenarios.

[0106] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0107] This embodiment also provides an image fusion device, which is used to implement the above embodiments and preferred implementations, and will not be repeated here. As used below, the terms "module", "unit", "subunit", etc. can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0108] Figure 5 is a structural block diagram of an image fusion device according to an embodiment of the present application, such as Figure 5 As shown, the device includes: an image acquisition unit 301, a first area determination unit 302, a second area determination unit 303, an area synchronization unit 304 and an image fusion unit 305.

[0109] An image acquisition unit 301 is used to acquire a visible light image and an infrared image;

[0110] A first region determining unit 302 is configured to perform target recognition on the visible light image to determine at least one first region;

[0111] A second region determining unit 303, configured to perform target recognition on the infrared image and determine at least one second region;

[0112] A region synchronization unit 304 is used to perform region synchronization on the first region of the visible light image and the second region of the infrared image to determine a target region and a non-target region;

[0113] The image fusion unit 305 is used to perform regional fusion on the visible light image and the infrared image. During the regional fusion process, the target area is fused based on a first fusion method, and the non-target area is fused based on a second fusion method to obtain a fused image.

[0114] In some of the embodiments, the image acquisition unit 301 includes: a target type recognition module, an infrared fill light band determination module and an infrared image determination module.

[0115] A target type recognition module is used to identify the target type corresponding to the photographed target;

[0116] An infrared fill light band determination module, used to determine the infrared fill light band corresponding to the shooting target based on the target type and a preset priori shooting index;

[0117] The infrared image determination module is used to obtain the infrared image corresponding to the shooting target based on the infrared fill light band.

[0118] In some embodiments, the area synchronization unit 304 includes: a frame synchronization processing module, a target area acquisition module and a non-target area acquisition module.

[0119] A frame synchronization processing module, used for performing data frame synchronization processing on the visible light image and the infrared image;

[0120] A target area acquisition module, used to map the first area and the second area of ​​the visible light image to each other based on the coordinate information of the first area and the second area of ​​the infrared image after data frame synchronization processing, so as to obtain a target area;

[0121] The non-target area acquisition module is used to determine the non-target area according to the target area.

[0122] In some embodiments, a fusion ratio corresponding to the first fusion mode is smaller than a fusion ratio corresponding to the second fusion mode, and the fusion ratio is a pixel value fusion ratio of a visible light image and an infrared image.

[0123] In some of the embodiments, the image fusion unit 305 includes: a target type determination module, a first fusion ratio determination module and a target area fusion module.

[0124] A target type determination module, used to obtain the target type corresponding to the target area;

[0125] A first fusion ratio determination module, used to determine a first fusion ratio corresponding to the target area based on the target type and a preset priori fusion index;

[0126] A target region fusion module is used to perform target region fusion based on the first fusion ratio.

[0127] In some embodiments, the image fusion unit 305 further includes:

[0128] A second fusion ratio determination module, used to determine a second fusion ratio corresponding to the non-target area;

[0129] A non-target area fusion module is used to perform non-target area fusion based on the second fusion ratio.

[0130] Figure 6 is a structural block diagram of a target feature recognition device according to an embodiment of the present application, such as Figure 6 As shown, the device includes: a fused image determination unit 401 and a target feature recognition unit 402.

[0131] A fused image determining unit 401 is used to perform image fusion based on the image fusion method described in the above embodiment to determine a fused image;

[0132] The target feature recognition unit 402 is used to perform target feature recognition based on the fused image.

[0133] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0134] In addition, the image fusion method or target feature recognition method in combination with the embodiments of the present application can be implemented by an electronic device. Figure 7 Schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application.

[0135] The electronic device may include a processor 51 and a memory 52 storing computer program instructions.

[0136] Specifically, the processor 51 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0137] Among them, the memory 52 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 52 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 52 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 52 may be inside or outside the data processing device. In a specific embodiment, the memory 52 is a non-volatile memory. In a specific embodiment, the memory 52 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable ProgrammableRead-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable ProgrammableRead-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0138] The memory 52 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 51 .

[0139] The processor 51 implements any one of the image fusion methods or target feature recognition methods in the above embodiments by reading and executing computer program instructions stored in the memory 52 .

[0140] In some of the embodiments, the electronic device may further include a communication interface 53 and a bus 50. Figure 7 As shown, the processor 51, the memory 52, and the communication interface 53 are connected via a bus 50 and communicate with each other.

[0141] The communication interface 53 is used to implement communication between the modules, devices, units and / or equipment in the embodiment of the present application. The communication interface 53 can also implement data communication with other components such as: external devices, image / data acquisition equipment, databases, external storage, and image / data processing workstations.

[0142] The bus 50 includes hardware, software or both, and couples the components of the electronic device to each other. The bus 50 includes but is not limited to at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, bus 50 may include Accelerated Graphics Port (AGP) or other graphics bus, Extended Industry Standard Architecture (EISA) bus, Front Side Bus (FSB), Hyper Transport (HT) interconnect, Industry Standard Architecture (ISA) bus, InfiniBand interconnect, Low Pin Count (LPC) bus, memory bus, Micro Channel Architecture (MCA) bus, Peripheral Component Interconnect (PCI) bus, PCI-Express (PCI-X) bus, Serial Advanced Technology Attachment (SATA) bus, Video Electronics Standards Association Local Bus (VLB) bus, or other suitable bus or a combination of two or more of these. Where appropriate, bus 50 may include one or more buses. Although embodiments of the present application describe and illustrate a particular bus, the present application contemplates any suitable bus or interconnect.

[0143] The electronic device can execute the image fusion method or target feature recognition method in the embodiment of the present application based on the acquired program instructions.

[0144] In addition, in combination with the image fusion method or target feature recognition method in the above embodiments, the present application embodiment may provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the image fusion methods or target feature recognition methods in the above embodiments is implemented.

[0145] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0146] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. An image fusion method, characterized in that: include: Acquire visible light images and infrared images; Performing target recognition on the visible light image to determine at least one first area; Performing target recognition on the infrared image to determine at least one second area; Performing regional synchronization on the first region of the visible light image and the second region of the infrared image to determine a target region and a non-target region; Performing regional fusion on the visible light image and the infrared image, in the regional fusion process, fusing the target area based on a first fusion method, and fusing the non-target area based on a second fusion method, to obtain a fused image; the fusion ratio corresponding to the first fusion method is smaller than the fusion ratio corresponding to the second fusion method, and the fusion ratio is a pixel value fusion ratio of the visible light image and the infrared image; The fusing of the target region based on the first fusion mode in the region fusion process includes: Obtaining the target type corresponding to the target area; Based on the target type and a preset priori fusion index, determining a first fusion ratio corresponding to the target area; The target area is fused based on the first fusion ratio.

2. The image fusion method according to claim 1, characterized in that: Acquiring infrared images includes: Identify the target type corresponding to the photographed target; Based on the target type and a preset priori shooting index, determining an infrared fill light band corresponding to the shooting target; An infrared image corresponding to the photographed target is acquired based on the infrared fill light band.

3. The image fusion method according to claim 1, characterized in that: Performing regional synchronization on the first region of the visible light image and the second region of the infrared image to determine the target region and the non-target region includes: Performing data frame synchronization processing on the visible light image and the infrared image; Based on the coordinate information of the first area of ​​the visible light image and the second area of ​​the infrared image after the data frame synchronization processing, the first area and the second area are mapped to each other to obtain a target area; A non-target area is determined according to the target area.

4. The image fusion method according to claim 1, characterized in that: The fusing of the non-target area based on the second fusion mode in the area fusion process includes: Determining a second fusion ratio corresponding to the non-target area; Non-target area fusion is performed based on the second fusion ratio.

5. A target feature recognition method, characterized in that: include: Perform image fusion based on the image fusion method according to any one of claims 1 to 4 to determine a fused image; Target feature recognition is performed based on the fused image.

6. An image fusion device, characterized in that: include: An image acquisition unit, used for acquiring visible light images and infrared images; A first region determining unit, configured to perform target recognition on the visible light image to determine at least one first region; A second region determination unit, configured to perform target recognition on the infrared image and determine at least one second region; A region synchronization unit, used for performing region synchronization on the first region of the visible light image and the second region of the infrared image to determine a target region and a non-target region; an image fusion unit, configured to perform regional fusion on the visible light image and the infrared image, wherein during the regional fusion process, the target area is fused based on a first fusion method, and the non-target area is fused based on a second fusion method to obtain a fused image; a fusion ratio corresponding to the first fusion method is smaller than a fusion ratio corresponding to the second fusion method, and the fusion ratio is a pixel value fusion ratio of the visible light image and the infrared image; The image fusion unit includes: a target type determination module, a first fusion ratio determination module and a target area fusion module; The target type determination module is used to obtain the target type corresponding to the target area; The first fusion ratio determination module is used to determine a first fusion ratio corresponding to the target area based on the target type and a preset priori fusion index; The target area fusion module is used to perform target area fusion based on the first fusion ratio.

7. A target feature recognition device, characterized in that: include: a fused image determining unit, configured to perform image fusion based on the image fusion method according to any one of claims 1 to 4, and determine a fused image; The target feature recognition unit is used to perform target feature recognition based on the fused image.

8. An electronic device, comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the image fusion method according to any one of claims 1 to 4 or the target feature recognition method according to claim 5.

Images