Video-level multi-mode camouflage identification system and identification method

Abstract

The invention belongs to the technical field of camouflage recognition, and particularly relates to a video-level multi-mode camouflage recognition system and method, and the method comprises the steps: S1, carrying out the common-view-field imaging of a region containing a camouflage target through an image collection unit, and obtaining a color image and a spectral image; s2, an image preprocessing unit performs color gamut conversion processing on the color image and performs spectral feature extraction processing on the spectral image; s3, a disguise identification unit modifies the color of a disguise target contained in the color image after the color gamut conversion processing into red, and performs pixel-level screening on the spectral image after the spectral feature extraction processing to realize identification of the disguise target; and S4, the display storage unit performs format conversion, display and storage on the recognized camouflage target. The invention solves the technical problems that the application scene of the existing camouflage identification imaging system is limited or the camouflage identification method cannot achieve video-level imaging and is difficult to identify in real time.

Description

Technical Field

[0001] The present invention belongs to the technical field of camouflage recognition, and in particular relates to a video-level multi-modal camouflage recognition system and recognition method. Background Art

[0002] Camouflage clothing and camouflage nets are used to make the color characteristics of the object close to the surrounding environment, so as to achieve the effect of hiding the object. The color of the current camouflaged object is very close to the background, and it is difficult to distinguish it only through color cameras or human visual observation. However, its spectral characteristics are still quite different from the background. The spectral imaging method can be used to identify and screen the camouflaged object.

[0003] The current spectral imaging technology mainly uses gratings as the core spectroscopic element. Its imaging process relies on push-scanning and cannot perform real-time imaging of the target. In order to restore the texture information of the target, the target is required to remain stationary for a period of time, and the pattern texture of the moving target cannot be restored. The existing snapshot imaging solutions on the market perform spectral splitting in a focal plane manner. Although they can quickly and in real time obtain the spectral information of the target by sacrificing spatial resolution to obtain data expansion in the spectral dimension, the lack of a large amount of texture information will also lead to a decrease in recognition accuracy. The existing camouflage recognition imaging system has a small range of recognized materials and limited application scenarios. In addition, the camouflage recognition method is difficult to achieve video-level output, and real-time imaging and real-time recognition analysis are not possible. Summary of the invention

[0004] The present invention combines the advantages of visible light detectors (visible cameras) and spectral imaging detectors (snapshot spectral cameras), adopts a common optical path imaging system, and uses a multimodal image processing algorithm for camouflage identification. By acquiring multimodal images (color images and multispectral images), the camouflaged target is quickly and effectively judged, and the spectral data information is combined with the spatial data information to obtain a wider spectral band, resolution, and rich spectrum information. The image information, spatial information, peaks and troughs in the spectral image, and the reflection intensity of the target to be identified in the corresponding range can be recorded simultaneously within the selected band, so as to realize the identification and classification of the target to be identified.

[0005] In view of this, the present invention aims to provide a video-level multimodal camouflage recognition system and recognition method to solve the shortcomings of the existing camouflage recognition imaging system, which has a small recognition range of materials and limited application scenarios, and the camouflage recognition method is difficult to achieve video-level output and cannot perform real-time imaging and real-time recognition and analysis. The present invention adopts a mosaic-style on-chip integrated coated spectroscopic spectral chip combined with a visible light detector to extract information of the target to be identified. The present invention can realize the recognition and classification of the target to be identified.

[0006] To achieve the above object, the technical solution created by the present invention is implemented as follows: A video-level multimodal camouflage recognition system comprises an image acquisition unit, an image preprocessing unit, a camouflage recognition unit and a display storage unit, wherein the image acquisition unit is used to perform common field imaging on an area containing a camouflaged target to obtain a color image and a spectral image, the image preprocessing unit performs color gamut conversion processing on the color image and performs spectral feature extraction processing on the spectral image, the camouflage recognition unit modifies the color of the camouflaged target contained in the color image after the color gamut conversion processing to red, and performs pixel-level screening on the spectral image after the spectral feature extraction processing to realize the recognition of the camouflaged target, and the display storage unit performs format conversion, display and storage on the recognized camouflaged target.

[0007] Furthermore, the image acquisition unit includes a lens, a spectroscopic module, and a visible light detector and a spectral imaging detector arranged side by side. The lens is aimed at the camouflaged target, and the spectroscopic module divides the light transmitted through the lens into two beams. The two beams of light are respectively incident on the visible light detector and the spectral imaging detector, so that the visible light detector and the spectral imaging detector perform common field imaging on the camouflaged target to obtain a color image and a spectral image.

[0008] Furthermore, the spectroscopic module includes a beam splitter, and a spectral imaging detector collects multi-spectral information of the camouflaged target.

[0009] Furthermore, the image preprocessing unit includes a color gamut conversion module and a spectral feature extraction module, the color gamut conversion module includes a normalization submodule, a hue extraction submodule and an image segmentation submodule, wherein the normalization submodule performs normalization processing on each pixel of the color image, the hue extraction submodule performs hue extraction on each pixel of the color image after normalization processing, and the image segmentation submodule constructs an image segmentation matrix according to the hue extraction result; The spectral feature extraction module includes a reflectance calculation submodule, a spectral feature extraction submodule and a cluster analysis submodule, wherein the reflectance calculation submodule calculates the reflectance of the spectral image to obtain the reflectance image, the spectral feature extraction submodule calculates the reflectance image and extracts the spectral features of the pixel points of the reflectance image, and the cluster analysis submodule classifies the spectral image into three categories based on the calculation results of the spectral feature submodule.

[0010] Furthermore, the camouflage identification unit includes a regional target identification submodule and a pixel type confirmation submodule, wherein the regional target identification submodule identifies the camouflage target based on the three-classification results of the cluster analysis submodule, and uses the identification results to construct a camouflage matrix, and the pixel type confirmation submodule modifies the camouflage target to red based on the camouflage matrix to achieve identification of the camouflage target.

[0011] A video-level multimodal camouflage recognition method is implemented using a video-level multimodal camouflage recognition system, and specifically includes the following steps: S1: Using an image acquisition unit to perform common field imaging on the area containing the camouflaged target to obtain a color image and a spectral image; S2: The image preprocessing unit performs color gamut conversion processing on the color image and performs spectral feature extraction processing on the spectral image; S3: The camouflage recognition unit changes the color of the camouflage target contained in the color image after the color gamut conversion process to red, and performs pixel-level screening on the spectral image after the spectral feature extraction process to realize the recognition of the camouflage target; S4: The display storage unit performs format conversion, display and storage on the identified disguised target.

[0012] Furthermore, step S2 specifically includes the following steps: S21: Normalize all pixels of the color image: ; ; ; in, is the normalized coordinate value of the i-th red pixel, is the normalized coordinate value of the i-th green pixel, is the normalized coordinate value of the i-th blue pixel, is the coordinate value of the i-th red pixel, is the coordinate value of the i-th green pixel, is the coordinate value of the i-th blue pixel; S22: Perform hue conversion on the normalized color image using the following formula: ; Among them, H is the hue of the i-th pixel of the color image, max is the maximum value, min is the maximum value, R is the R channel, and G is the G channel.

[0013] S23: Construct the image segmentation matrix pic_bool. The image segmentation matrix has the same dimension as the R channel image pic_r in the color image. If 125°≥H i ≥115°, the image segmentation matrix is i The pixel value corresponding to the value is 1, otherwise it is 0; S24: Calculate the reflectance of the spectral image to obtain a reflectance image: ; in, is the reflectance image, is the spectral image, is a dark noise image, is a whiteboard image; S25: Construct a spectral feature extraction formula, and use the spectral feature extraction formula to calculate the reflectance image to extract the spectral features of the pixel points of the reflectance image: ; in, is the calculation result of the reflectivity of the i-th spectral band and the j-th spectral band in the reflectivity image, is the reflectance of the i-th spectral band in the reflectance image, is the reflectivity of the jth spectral band in the reflectivity image; S26: randomly select n pixels on the spectral image, and use the n pixels as cluster centers, and assign each pixel of the spectral image to the nearest cluster center by the following formula: ; in, is the root mean square, dist is and The straight-line distance, is the pixel point on the spectral image, is the cluster center, is the first pixel on the spectral image, is the first cluster center, is the second pixel on the spectral image, is the second cluster center, is the nth pixel on the spectral image, is the third cluster center; S27: Calculate the mean value of the pixel values ​​contained in each cluster center, and use the pixel points corresponding to the mean value of the pixel values ​​contained in each cluster center as new cluster centers; S28: replacing the cluster center of step S26 with a new cluster center, and repeating steps S26 to S27 until the pixel position of the cluster center no longer changes or the preset number of iterations is reached; S29: Determine the colored camouflaged target, the background target of the same color as the camouflaged target, and other targets of different colors from the camouflaged target according to the pixel position of the cluster center obtained in step S28, so as to achieve three-classification of the spectral image.

[0014] Further, in step S28, the preset number of iterations is no less than three.

[0015] Furthermore, step S3 specifically includes the following steps: S31: constructing a classification matrix based on the three classification results of the cluster analysis submodule, the classification matrix has the same size as the image segmentation matrix, and the classification matrix is ​​set according to the spatial positions of the pixels of each category; In the classification matrix, the pixel where the camouflage target is located is set to 1, the pixel where the background target is located is set to 0, and the pixel where the other background is located is set to 2; S32: The camouflage target is identified by the following formula, and a camouflage matrix is ​​constructed according to the identification result: ; in, For operation with To identify the results, is the classification matrix, is the image segmentation matrix; S33: setting the value of the R channel of the image pixel value corresponding to the pixel point with a value of 1 in the camouflage matrix to 255, changing the camouflage target to red, and realizing the recognition of the camouflage target.

[0016] Further, in step S32, if the recognition result of the current pixel and the value in the image segmentation matrix are both 1, it means that the current pixel is the pixel where the camouflage target is located, otherwise it is the pixel where the background target or other target is located; The pixel where the camouflaged target is located is set to 1, and the pixel where the background target or other target is located is set to 0, and the camouflage matrix is ​​constructed according to the spatial position of each pixel in the spectral image.

[0017] Compared with the prior art, the invention can achieve the following beneficial effects: (1) The video-level multimodal camouflage recognition system created by the present invention can simultaneously obtain multi-spectral information of the target to be identified by using a dual-camera spectroscopic structure. Compared with the traditional method, the snapshot mosaic camera is used for imaging, and real-time multi-spectral imaging of the target to be identified can be performed without sacrificing time resolution. The image preprocessing unit performs operations such as denoising and enhancement on the collected image information to eliminate interference and enhance image information. The camouflage recognition unit mainly calculates the color difference of the color image through data processing operations to extract the color-similar areas in the image and complete the preliminary screening of the camouflage recognition function; the present invention obtains the image information of each spectral band, and further screens the target color-similar areas extracted from the color image according to the spectral difference, thereby completing the camouflage recognition function of the target to be identified. The storage and display module of the present invention can complete the conversion of image information format, information storage and video-level output functions.

[0018] (2) The present invention creates the video-level multimodal camouflage recognition system, which combines spectral imaging with visible light imaging through a dual-camera structure, and adopts a multimodal image information acquisition method to obtain clear details of the image of the target to be identified while achieving video-level output, making multimodal real-time camouflage recognition based on direct spectral imaging possible. The multimodalities are visible light imaging and spectral imaging. The core recognition method of the present invention is to extract the areas with similar colors in the collected image area through a visible light detector, and after restoring the spectral image using a spectral imaging detector, screen the areas with similar colors extracted by the visible light detector in the spectral dimension, and use the areas with large spectral differences as the camouflage parts recognized by the system.

[0019] (3) The present invention adopts a spectral imaging detector with on-chip integrated spectrometry to replace the traditional spectroscopic systems such as gratings and prisms. The spectral imaging detector has the characteristics of small size, light weight, etc., and is easy to integrate. It solves the problems that the traditional spectroscopic method requires push scanning or switching filters to achieve spectral imaging, and the compressed calculation spectral spectroscopic method requires complex and tedious repetition of the original algorithm, and the reconstruction quality changes with the scene, and it is impossible to perform real-time spectral imaging or real-time reconstruction of the target. The spectral imaging detector can directly perform spectral imaging and can recognize camouflaged targets in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings constituting part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments and descriptions of the present invention are used to explain the present invention and do not constitute an improper limitation on the present invention. In the drawings: Figure 1 A schematic diagram of the structure of a video-level multimodal masquerade recognition system according to an embodiment of the present invention; Figure 2 A flowchart of a video-level multimodal camouflage identification method according to an embodiment of the present invention; Figure 3 A schematic diagram of the acquisition form and sequence of the spectral image described in the embodiment of the present invention; Figure 4 The present invention is a hardware structure diagram of a video-level multimodal camouflage identification system according to an embodiment of the present invention.

[0021] Description of reference numerals: 1. Image acquisition unit; 2. Image preprocessing unit; 3. Camouflage recognition unit; 4. Display storage unit; 5. Visible light detector; 6. Spectral imaging detector; 7. FPGA; 8. Flash; 9. DDR3 memory; 10. ARM. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the invention more clear, the invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described here are only used to explain the invention and do not constitute a limitation of the invention.

[0023] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0024] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0025] In the description of the invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the invention can be understood according to specific circumstances.

[0026] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0027] like Figure 1As shown, the present invention provides a video-level multimodal camouflage recognition system, comprising an image acquisition unit 1, an image preprocessing unit 2, a camouflage recognition unit 3 and a display storage unit 4 connected in sequence, wherein the image acquisition unit 1 is used to perform common field imaging on the area containing the camouflage target to obtain a color image and a spectral image, that is, to realize the extraction of two types of images of the area where the camouflage target (the camouflage target is not red, but generally green) is located; the image preprocessing unit 2 is used to perform image preprocessing before camouflage recognition, specifically: perform color gamut conversion processing on the color image, and perform spectral feature extraction processing on the spectral image, the camouflage recognition unit 3 extracts the approximate color area of ​​the color image by color difference comparison, and changes the color of the camouflage target contained in the color image after the color gamut conversion processing to red, so as to facilitate human naked eye observation, and uses image restoration operation to perform pixel-level screening on the spectral image after the spectral feature extraction processing (three-classification of the spectral image), that is, perform camouflage recognition on the color-similar area of ​​the color image according to the spectral difference, and realize the recognition of the camouflage target, and the display storage unit 4 performs format conversion, display and storage on the recognized camouflage target.

[0028] The image acquisition unit 1 includes a lens, a spectroscopic module, and a visible light detector 5 and a spectral imaging detector 6 arranged side by side, which are sequentially arranged along the transmission direction of the optical path. The lens is aimed at the camouflaged target, and the spectroscopic module divides the light transmitted through the lens into two beams. The two beams of light are incident on the visible light detector 5 and the spectral imaging detector 6 respectively, so that the visible light detector 5 and the spectral imaging detector 6 perform common field imaging on the camouflaged target to obtain a color image and a spectral image. By means of spectroscopic method, a dual camera structure (specifically, a visible light detector 5 and a spectral imaging detector 6 arranged side by side) is used for common field imaging, which can realize the acquisition of multimodal image information (including color images and spectral images), thereby effectively capturing the camouflaged target (dynamic). The visible light detector 5 is used to obtain a high-resolution image with rich detail information, and the spectral imaging detector 6 can use a snapshot spectral imaging detector 6 to obtain a high frame rate image. The use of these two modal detectors for common field work can process the spectral image and the color image in real time to achieve video-level imaging.

[0029] The spectroscopic module includes a beam splitter (including a neutral beam splitter), and the spectral imaging detector 6 uses an on-chip spectroscopic technology based on pixel-level mosaic coating to collect multi-spectral information of the camouflaged target.

[0030] The image preprocessing unit 2 includes a color gamut conversion module and a spectral feature extraction module. The color gamut conversion module includes a normalization submodule, a hue extraction submodule and an image segmentation submodule. The normalization submodule performs normalization processing on each pixel of the color image, the hue extraction submodule performs hue extraction on each pixel of the color image after normalization processing, and the image segmentation submodule constructs an image segmentation matrix according to the hue extraction result.

[0031] The spectral feature extraction module includes a reflectance calculation submodule, a spectral feature extraction submodule and a cluster analysis submodule, wherein the reflectance calculation submodule calculates the reflectance of the spectral image to obtain the reflectance image, the spectral feature extraction submodule calculates the reflectance image and extracts the spectral features of the pixel points of the reflectance image, and the cluster analysis submodule classifies the spectral image into three categories based on the calculation results of the spectral feature submodule.

[0032] The image preprocessing unit 2 performs image preprocessing operations on the color image and the spectral image, and performs color gamut conversion on the color image, including normalization, hue extraction and threshold segmentation processing, in order to separate the area where the camouflage target is located from the background for subsequent identification. The work of extracting spectral features from the spectral image includes calculating the reflectance of each spectral band, extracting features between spectral bands, clustering analysis of spectral features and restoring the resolution of the spectral image. Taking green plants as camouflage targets as an example, the purpose of extracting spectral features from the spectral image is to divide the spectral features into three categories: plants with green color (camouflage targets), background targets with green color, and targets with colors other than green.

[0033] The camouflage identification unit 3 includes a regional target identification submodule and a pixel type confirmation submodule, wherein the regional target identification submodule identifies the camouflage target based on the three-classification results of the cluster analysis submodule, and uses the identification results to construct a camouflage matrix; the pixel type confirmation submodule modifies the camouflage target to red based on the camouflage matrix to realize the identification of the camouflage target.

[0034] The camouflage recognition unit 3 comprehensively analyzes the results of the image preprocessing unit 2, performs pixel-level screening based on the color similarity and spectral difference between the camouflage target and the surrounding environment, and modifies the camouflage target contained in the acquired color image to red to facilitate naked eye observation.

[0035] The processed image information is converted by the display storage unit 4 and then output and displayed at the video level through the network port.

[0036] The present invention can accurately and efficiently extract and analyze information of a camouflaged target, achieve video-level output, and realize identification of the camouflaged target.

[0037] like Figure 2As shown, the video-level multimodal masquerade recognition method provided by the present invention is implemented by using a video-level multimodal masquerade recognition system, and specifically includes the following steps: S1: Using the image acquisition unit 1 to perform common field imaging on the area containing the camouflaged target to obtain a color image and a spectral image; A dual-camera structure is adopted, and the beam splitter is used to split the light, so that the visible light detector 5 and the spectral imaging detector 6 can be imaged simultaneously. The core spectroscopic technology used by the spectral imaging detector 6 is an on-chip spectroscopic technology based on pixel-level mosaic coating. That is, multi-spectral information of the camouflaged target can be obtained in one shot. The image acquisition unit 1 transmits the collected image information to the image preprocessing unit 2 for further processing.

[0038] The spectral imaging detector 6 is a film structure based on the Fabry-Perot cavity. It adopts spectral channel combination etching technology to directly coat the pixel-level mosaic filter film on the surface of the imaging chip, so that the spectral imaging detector 6 has the advantages of small size, high energy utilization and high reliability.

[0039] The visible light detector 5 obtains a high-resolution image of the area where the camouflaged target is located, which can reflect the detailed information of the camouflaged target. The spectral imaging detector 6 realizes snapshot imaging and obtains a high-frame rate image, that is, it obtains multi-spectral information of the camouflaged target in one shot, and quickly obtains the information of the camouflaged target in the spectral dimension, and then uses the two modal images (color image and multispectral image) to achieve real-time exposure and real-time data processing.

[0040] like Figure 3 As shown, taking the nine-channel pixel-level mosaic filter film as an example, the red dotted box is the imaging chip pixel, band1~band9 is the pixel-level spectroscopic film, and the RGB color image obtained by the visible light detector 5 is , the spectral image acquired by the spectral imaging detector 6 is , is the spectral image corresponding to the Nth spectral channel. Here, pic_rgb and pic_light have the same length and width but different dimensions, pic_rgb is three-dimensional and pic_light is one-dimensional.

[0041] S2: the image preprocessing unit 2 performs color gamut conversion processing on the color image and performs spectral feature extraction processing on the spectral image; The key to camouflaging a target is to maintain consistency in color observation. For example, in a jungle scene, the camouflaged personnel are wearing green camouflage uniforms, which means that the scene to be distinguished is a green scene. By selecting the green area of ​​the color image, subsequent spectral resolution is achieved. To identify the green in the color image, the color image needs to be converted from RGB to HSV color gamut. The H hue represents the type or category of color, and the value range is 0° to 360°. The value of standard green is 120°. Considering the influence of color difference and lighting environment conditions, the hue within the range of ±5° of standard green is regarded as green.

[0042] S21: Normalize all pixels of the color image: ; ; ; in, is the normalized coordinate value of the i-th red pixel, is the normalized coordinate value of the i-th green pixel, is the normalized coordinate value of the i-th blue pixel, is the coordinate value of the i-th red pixel, is the coordinate value of the i-th green pixel, is the coordinate value of the i-th blue pixel.

[0043] S22: Perform hue conversion on the normalized color image using the following formula: ; Among them, H is the hue of the i-th pixel of the color image, max is the maximum value, min is the maximum value, R is the R channel, and G is the G channel.

[0044] S23: Construct the image segmentation matrix pic_bool. The image segmentation matrix has the same dimension as the R channel image pic_r in the color image. If 125°≥H i ≥115°, the image segmentation matrix is i The pixel value corresponding to the value is 1, otherwise it is 0; After the above settings, in the image segmentation matrix pic_bool, the pixel value of 1 corresponds to the green color of the color image.

[0045] First, the reflectance of the spectral image is calculated to obtain a dark noise image pic_dark under the same exposure time as the spectral image, and a white board (white board with a reflectance of 100%) image pic_white under the same exposure time as the spectral image, and the reflectance image of the spectral image is obtained.

[0046] S24: Calculate the reflectance of the spectral image to obtain a reflectance image: ; in, is the reflectance image, is the spectral image, is a dark noise image, is a whiteboard image; Get the total reflectance image , the overall reflectance image is decomposed into the reflectance images of each spectral band, and the corresponding spectral bands are , and the order from 1 to n is arranged from small to large, for The reflectance image of the spectral band, for The reflectance image of the spectral band, for Reflectance image of the spectral band.

[0047] S25: Construct a spectral feature extraction formula, and use the spectral feature extraction formula to calculate the reflectance image to extract the spectral features of the pixel points of the reflectance image: ; in, is the calculation result of the reflectivity of the i-th spectral band and the j-th spectral band in the reflectivity image, is the reflectance of the i-th spectral band in the reflectance image, is the reflectance of the jth spectral band in the reflectance image, and , the reflectivity of the pixel .

[0048] For ease of understanding, as an example, three pixels are selected as cluster centers, and the three pixels are c1, c2, and c3. For example, when traversing to the x1 pixel, assuming that there are 4 spectrum segments, the six reflectance values ​​corresponding to the pixel x1 are RL 12 (x1), RL 13 (x1), RL 14 (x1), RL 23 (x1), RL 24 (x1), RL 34 (x1); The six reflectivity values ​​corresponding to the cluster center c1 are RL 12 (c1), RL 13 (c1), RL 14 (c1), RL 23 (c1), RL 24(c1), RL 34 (c1); The six reflectivity values ​​corresponding to the cluster center c2 are RL 12 (c2), RL 13 (c2), RL 14 (c2), RL 23 (c2), RL 24 (c2), RL 34 (c2); The six reflectivity values ​​corresponding to the cluster center c3 are RL 12 (c3), RL 13 (c3), RL 14 (c3), RL 23 (c3), RL 24 (c3), RL 34 (c3); Then according to the above formula dist(x1,c1)=sqrt((RL 12 (x1) -RL 12 (c1) 2 +(RL 13 (x1) -RL 13 (c1) 2 +(RL 14 (x1) -RL 14 (c1) 2 +(RL 23 (x1) -RL 23 (c1) 2 +(RL 24 (x1) -RL 24 (c1) 2 +(RL 34 (x1) -RL 34 (c1) 2 ) dist(x1,c2)=sqrt (Similarly, replace all c1 in the above dist(x1,c1) formula with c2) dist(x1,c3)=sqrt (Similarly, replace all c1 in the above dist(x1,c1) formula with c3) At this time, take the minimum value among dist(x1,c1), dist(x1,c2), and dist(x1,c3). If dist(x1,c2) ​​has the smallest value, we believe that x1 belongs to class c2.

[0049] S26: randomly select n pixels on the spectral image, and use the n pixels as cluster centers, and assign each pixel of the spectral image to the nearest cluster center by the following formula: ; in, is the root mean square, dist is and The straight-line distance, is the pixel point on the spectral image, is the cluster center, is the first pixel on the spectral image, is the first cluster center, is the second pixel on the spectral image, is the second cluster center, is the nth pixel on the spectral image, is the third cluster center.

[0050] S27: Calculate the mean value of the pixel values ​​contained in each cluster center, and use the pixel points corresponding to the mean value of the pixel values ​​contained in each cluster center as new cluster centers; The formula for calculating the mean pixel value contained in each cluster center is: ; in, is the mean value of the pixels contained in each cluster center, and sum is the summation function.

[0051] S28: Replace the cluster center of step S26 with a new cluster center, and repeat steps S26 to S27 until the pixel position of the cluster center no longer changes or reaches a preset number of iterations (the preset number of iterations is no less than three times).

[0052] S29: Determine the colored camouflaged target, the background target of the same color as the camouflaged target, and other targets of different colors from the camouflaged target according to the pixel position of the cluster center obtained in step S28, so as to achieve three-classification of the spectral image.

[0053] S3: the camouflage identification unit 3 changes the color of the camouflage target contained in the color image after the color gamut conversion process to red, and performs pixel-level screening on the spectral image after the spectral feature extraction process to achieve identification of the camouflage target; S31: constructing a classification matrix based on the three classification results of the cluster analysis submodule, the classification matrix has the same size as the image segmentation matrix, and the classification matrix is ​​set according to the spatial positions of the pixels of each category; In the classification matrix, the pixel where the camouflaged target is located is set to 1, the pixel where the background target is located is set to 0, and the pixel where other backgrounds are located is set to 2.

[0054] After obtaining a classification matrix of the same size as the spectral image, the spatial positions of the pixels of each category are marked, and the classification matrix is ​​copied to the same size as the image segmentation matrix.

[0055] S32: The camouflage target is identified by the following formula, and a camouflage matrix is ​​constructed according to the identification result: ; in, For operation with To identify the results.

[0056] If the recognition result of the current pixel and the value in the image segmentation matrix are both 1, it means that the current pixel is the pixel where the camouflage target is located, otherwise it is the pixel where the background target or other target is located; The pixel where the camouflaged target is located is set to 1, and the pixel where the background target or other target is located is set to 0, and the camouflage matrix is ​​constructed according to the spatial position of each pixel in the spectral image.

[0057] To perform an "AND" operation on the classification matrix and the image segmentation matrix, take the camouflage target as an example, the pixel point where jieguo_lv is 1 is the green environment area. The pixel value results of each pixel point are traversed. When the pixel value of the pixel point is jieguo_lv is 0 and pic_bool is 1, it can be judged that this pixel point is a background target. The traversal result is stored in the camouflage matrix. The type of this camouflage matrix is ​​bool type, 1 is the pixel point where the camouflage target is located, and 0 is the pixel point where the background target is located.

[0058] S33: setting the value of the R channel of the image pixel value corresponding to the pixel point with a value of 1 in the camouflage matrix to 255, changing the camouflage target to red, and realizing the recognition of the camouflage target.

[0059] The color image is processed according to the rule that when the pixel point is 1 in the camouflage matrix, the corresponding pixel value of the color image is [255, b, r], that is, the br channel value remains unchanged, and the r channel value is changed to 255. At this time, the camouflaged target is displayed in red; when the pixel point value in the camouflage matrix is ​​0, the pixel value of the color image remains unchanged.

[0060] S4: The display storage unit 4 performs format conversion, display and storage on the identified disguised target.

[0061] The processed image information is converted by the display storage unit 4 and then output and displayed at the video level through the network port. Figure 4As shown, FPGA7 is used to drive the two imaging detectors and configure the various parameters of the detectors to realize the imaging function. At the same time, the format of the collected data is converted, and the collection requirements are controlled through the ARM10 core to complete the integration of the data stream. The results are displayed through the network port for data transmission.

[0062] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the disclosure of the present invention can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document does not limit this.

[0063] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A video-level multimodal camouflage recognition system, characterized by: The invention comprises an image acquisition unit, an image preprocessing unit, a camouflage recognition unit and a display storage unit, wherein the image acquisition unit is used to perform common field imaging on an area containing a camouflage target to obtain a color image and a spectral image, the image preprocessing unit performs color gamut conversion processing on the color image, and performs spectral feature extraction processing on the spectral image, the camouflage recognition unit modifies the color of the camouflage target contained in the color image after the color gamut conversion processing to red, and performs pixel-level screening on the spectral image after the spectral feature extraction processing to realize the recognition of the camouflage target, and the display storage unit performs format conversion, display and storage on the recognized camouflage target.

2. The video-level multimodal masquerade recognition system according to claim 1, characterized in that: The image acquisition unit includes a lens, a spectroscopic module, and also includes a visible light detector and a spectral imaging detector arranged side by side. The lens is aimed at the camouflaged target, and the spectroscopic module divides the light transmitted through the lens into two beams. The two beams of light are respectively incident on the visible light detector and the spectral imaging detector, so that the visible light detector and the spectral imaging detector perform common field imaging on the camouflaged target to obtain a color image and a spectral image.

3. The video-level multimodal masquerade recognition system according to claim 2, characterized in that: The light splitting module includes a beam splitter, and the spectral imaging detector collects multi-spectral information of the camouflaged target.

4. The video-level multimodal masquerade recognition system according to claim 1, characterized in that: The image preprocessing unit includes a color gamut conversion module and a spectral feature extraction module, the color gamut conversion module includes a normalization submodule, a hue extraction submodule and an image segmentation submodule, wherein the normalization submodule performs normalization processing on each pixel of the color image, the hue extraction submodule performs hue extraction on each pixel of the color image after normalization processing, and the image segmentation submodule constructs an image segmentation matrix according to the hue extraction result; The spectral feature extraction module includes a reflectance calculation submodule, a spectral feature extraction submodule and a cluster analysis submodule, wherein the reflectance calculation submodule performs reflectance calculation on the spectral image to obtain the reflectance image, the spectral feature extraction submodule calculates the reflectance image and extracts the spectral features of the pixel points of the reflectance image, and the cluster analysis submodule performs three classifications on the spectral image based on the calculation results of the spectral feature submodule.

5. The video-level multimodal camouflage identification system according to claim 4, characterized in that: The camouflage identification unit includes a regional target identification submodule and a pixel type confirmation submodule, wherein the regional target identification submodule identifies the camouflage target based on the three-classification result of the cluster analysis submodule, and constructs a camouflage matrix using the identification result, and the pixel type confirmation submodule modifies the camouflage target to red based on the camouflage matrix to realize the identification of the camouflage target.

6. A video-level multimodal masquerade identification method, implemented using the video-level multimodal masquerade identification system according to any one of claims 1 to 5, characterized in that: The specific steps include: S1: Using an image acquisition unit to perform common field imaging on the area containing the camouflaged target to obtain a color image and a spectral image; S2: the image preprocessing unit performs color gamut conversion processing on the color image and performs spectral feature extraction processing on the spectral image; S3: The camouflage recognition unit changes the color of the camouflage target contained in the color image after the color gamut conversion process to red, and performs pixel-level screening on the spectral image after the spectral feature extraction process to realize the recognition of the camouflage target; S4: The display storage unit performs format conversion, display and storage on the identified disguised target.

7. The video-level multimodal camouflage identification method according to claim 6, characterized in that: Step S2 specifically includes the following steps: S21: performing normalization processing on all pixels of the color image: ； ； ； in, is the normalized coordinate value of the i-th red pixel, is the normalized coordinate value of the i-th green pixel, is the normalized coordinate value of the i-th blue pixel, is the coordinate value of the i-th red pixel, is the coordinate value of the i-th green pixel, is the coordinate value of the i-th blue pixel; S22: Perform hue conversion on the normalized color image using the following formula: ； Among them, H is the hue of the i-th pixel of the color image, max is the maximum value, min is the maximum value, R is the R channel, and G is the G channel; S23: construct an image segmentation matrix pic_bool, the image segmentation matrix has the same dimension as the R channel image pic_r in the color image, if 125°≥H i ≥115°, the image segmentation matrix is i The pixel value corresponding to the value is 1, otherwise it is 0; S24: Calculate the reflectance of the spectral image to obtain a reflectance image: ； in, is the reflectance image, is the spectral image, is a dark noise image, is a whiteboard image; S25: construct a spectral feature extraction formula, and use the spectral feature extraction formula to calculate the reflectance image to extract the spectral features of the pixel points of the reflectance image: ； in, is the calculation result of the reflectivity of the i-th spectral band and the j-th spectral band in the reflectivity image, is the reflectance of the i-th spectral band in the reflectance image, is the reflectivity of the jth spectral band in the reflectivity image; S26: randomly select n pixels on the spectral image, and use the n pixels as cluster centers, and assign each pixel of the spectral image to the nearest cluster center by the following formula: ； in, is the root mean square, dist is and The straight-line distance, is the pixel point on the spectral image, is the cluster center, is the first pixel on the spectral image, is the first cluster center, is the second pixel on the spectral image, is the second cluster center, is the nth pixel on the spectral image, is the third cluster center; S27: Calculate the mean value of the pixel values ​​contained in each cluster center, and use the pixel points corresponding to the mean value of the pixel values ​​contained in each cluster center as new cluster centers; S28: replacing the cluster center of step S26 with a new cluster center, and repeating steps S26 to S27 until the pixel position of the cluster center no longer changes or the preset number of iterations is reached; S29: Determine the colored camouflaged target, the background target of the same color as the camouflaged target, and other targets of different colors from the camouflaged target according to the pixel position of the cluster center obtained in step S28, so as to achieve three-classification of the spectral image.

8. The video-level multimodal camouflage identification method according to claim 7, characterized in that: In step S28, the preset number of iterations is no less than three.

9. The video-level multimodal masquerade recognition method according to claim 7, characterized in that: The step S3 specifically includes the following steps: S31: constructing a classification matrix based on the three classification results of the cluster analysis submodule, wherein the classification matrix has the same size as the image segmentation matrix and is set according to the spatial positions of the pixels of the respective categories; In the classification matrix, the pixel where the camouflage target is located is set to 1, the pixel where the background target is located is set to 0, and the pixel where other backgrounds are located is set to 2; S32: The camouflage target is identified by the following formula, and a camouflage matrix is ​​constructed according to the identification result: ； in, For operation with To identify the results, is the classification matrix, is the image segmentation matrix; S33: setting the value of the R channel of the image pixel value corresponding to the pixel point with a value of 1 in the camouflage matrix to 255, changing the camouflage target to red, and realizing the recognition of the camouflage target.

10. The video-level multimodal masquerade recognition method according to claim 9, characterized in that: In step S32, if the recognition result of the current pixel and the value in the image segmentation matrix are both 1, it means that the current pixel is the pixel where the camouflage target is located, otherwise it is the pixel where the background target or other target is located; The pixel where the camouflaged target is located is set to 1, and the pixel where the background target or other target is located is set to 0, and the camouflage matrix is ​​constructed according to the spatial position of each pixel in the spectral image.

Images