An active illumination observation system and method based on image quality assessment

Abstract

The present application relates to the field of computer vision and robot, and more particularly to an active illumination observation system and method based on image quality evaluation, a configuration optimization module adjusts the posture of an active illumination module and a camera holder control module to obtain a plurality of input images; an image quality evaluation module performs sharpness evaluation, matching degree evaluation and structural similarity evaluation on the input images, obtains an evaluation score of the input images through weighted calculation and inputs the evaluation score into the configuration optimization module; the configuration optimization module sorts the evaluation scores of the input images provided by the image quality evaluation module, selects the highest evaluation score as the optimal active illumination observation system configuration after screening. The present application quickly obtains a plurality of system configurations with better observation effect in a given interval, and further obtains the optimal configuration of the active illumination observation system through further subdivision search, evaluation, sorting and screening. Through the strategy of combining coarse search and fine positioning, the efficiency, accuracy and robustness of visual perception are ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision and robotics, and in particular to an active illumination observation system and method based on image quality evaluation. BACKGROUND

[0002] With the development of computer vision and robotics technology, more and more robots are used to replace human work in engineering with harsh working environment and safety hazards. The image quality obtained by the camera affects the results of feature detection and target recognition, and also affects the accuracy of robot control. Therefore, in actual work, it is very important to adjust the illumination intensity, angle and camera view angle according to the imaging situation to obtain high-quality observation images.

[0003] In engineering applications, active illumination systems are used to obtain the most suitable lighting effect for the environment. Most of the current active illumination systems are laser active illumination systems, which use laser as the light source and take advantage of its high brightness, monochromaticity and collimation to illuminate dark and weak targets. The camera receives the echo signal of the target to form an image, and realizes tracking, detection and identification of targets in remote, small, dark or adverse weather conditions, and is mostly used in outdoor scenes at night. However, in robot operation, target distance is short, and precision requirement is high, the above-mentioned laser active illumination system does not match the application scene and operating environment, and the imaging quality cannot meet the task requirements.

[0004] The purpose of existing image quality evaluation is to design an algorithm to evaluate the damage degree of the image, i.e. the distortion degree, and to give an evaluation value as consistent as possible with the human eye. Common objective image quality evaluation indicators include error-based evaluation indicators, perception model and image structure information-based evaluation indicators, and machine learning-based evaluation indicators. However, in tasks related to target recognition, feature detection and robot operation, it is not enough to evaluate only the distortion degree of the image. If the clarity, matching degree and structural similarity can be quantified, and a score with a unified quantitative standard is output, the observation image quality can be more comprehensively evaluated to determine whether it meets the task requirements. SUMMARY

[0005] In view of the deficiencies of the prior art, the present application provides an active illumination observation system and method based on image quality evaluation, which is aimed at the field of computer vision and robotics in harsh working environments, and aims to integrate an active illumination module, a camera gimbal control module and an image quality evaluation module to automatically obtain the system configuration of the best observation effect through a configuration optimization module, to obtain high-quality observation images by adjusting the illumination intensity, angle and camera view angle, and to improve the robustness and accuracy of visual perception in target recognition and feature detection tasks.

[0006] In order to solve the above problems, the technical solution of the present application is: the present application provides an active illumination observation system based on image quality evaluation, comprising: an active illumination module, a camera holder control module, a configuration optimization module and an image quality evaluation module;

[0007] The host computer comprises the configuration optimization module and the quality evaluation module, the active illumination module and the camera holder control module are electrically connected with the host computer or wirelessly interact with the configuration optimization module, the configuration optimization module adjusts the attitude of the active illumination module and the camera holder control module to obtain a plurality of input images; the configuration optimization module provides the input images and the reference images for the image quality evaluation module;

[0008] The image quality evaluation module performs weighted calculation on the clarity evaluation, the matching degree evaluation and the structural similarity evaluation of the input images and the reference images to obtain the evaluation score of the input images and inputs the evaluation score into the configuration optimization module;

[0009] The configuration optimization module sorts the input image evaluation scores provided by the image quality evaluation module, and selects the highest evaluation score as the optimal configuration of the active illumination observation system.

[0010] Further, the active illumination module comprises at least one illumination light source and an illumination light source holder, the illumination light source holder has three degrees of freedom of pitch, yaw and brightness, and controls the attitude and brightness of the illumination light source.

[0011] Further, the camera holder control module comprises an observation camera and an observation camera holder, the observation camera holder has two degrees of freedom of pitch and yaw, and is used for controlling and adjusting the attitude of the observation camera to provide the input images for the image quality evaluation module.

[0012] Further, the image quality evaluation module comprises a clarity evaluation unit, a matching degree evaluation unit and a structural similarity evaluation unit.

[0013] The clarity evaluation unit evaluates the clarity score of the input image, the clarity score comprises a clarity estimation based on a local amplitude spectrum and a spatial-based clarity estimation; the clarity score is used for quantifying the perceived sharpness of the input image.

[0014] The matching degree estimation unit evaluates the matching degree score of the input image and the reference image, the matching degree score is the number of matching pairs of feature points of the input image and the feature points of the reference image, and the proportion of the feature points of the reference image;

[0015] The structural similarity evaluation unit evaluates the structural similarity score of the input image and the reference image, the structural similarity score comprises a brightness comparison, a contrast comparison and a structure comparison, and the three elements are weighted and multiplied.

[0016] Further, the image quality evaluation module includes that the input image is down-sampled for several times, an image pyramid is constructed, the sampled images are used to cover the input image with different resolutions, and more details are obtained; the image quality evaluation module respectively evaluates the sharpness score, the matching degree score and the structural similarity score of each sampled image in the image pyramid, and obtains the score of each sampled image by weighted summation calculation, and finally the evaluation score of the input image is obtained by synthesizing the scores of all the sampled images;

[0017] The specific expression formula of the score of the sampled image is as follows:

[0018] a q =ω1s q，1 +ω2s q，2 +ω3s q，3 , q = 0, 1, …, Q;

[0019] Wherein, a q is the score of the sampled image, Q is the number of times of down-sampling the input image, q is the serial number of the sampled image, ω1, ω2, ω3 are weight parameters, and ω1+ω2+ω3=1; s q，1 is the sharpness score, s q，2 is the matching degree score, and s q，3 is the structural similarity score.

[0020] The specific expression formula of the evaluation score of the input image is as follows:

[0021]

[0022] Wherein, A is the evaluation score of the input image, q = 0 represents that the calculation is started from 0 and increased; wherein W q is a weight parameter and indicates that the weight summation of Q+1 sampled images is 1.

[0023] Further, the sharpness evaluation unit includes that the sharpness score of each sampled image is evaluated, and the sharpness score includes a local amplitude spectrum-based sharpness estimation and a space-based sharpness estimation.

[0024] The specific expression formula of the sharpness score is as follows:

[0025] s q，1 =t1(P q ) η +t2(P q ) 1-η ;

[0026] Wherein, η is a weight parameter, 0≤η≤1, s q，1 is the sharpness score of the sampled image, t1(P q) a local amplitude spectrum based sharpness estimation for the sampled image, t2(P a ) a spatial based sharpness estimation.

[0027] Further, the local amplitude spectrum based sharpness estimation is performed by dividing the gray-scaled sampled image into equal image blocks, each of which has a preset pixel width overlap, calculating the amplitude spectrum slope of each image block, performing the local amplitude spectrum based sharpness estimation for each image block, and integrating the local amplitude spectrum based sharpness estimation of each image block using a Hanning window to generate the local amplitude spectrum based sharpness estimation of the sampled image.

[0028] The local amplitude spectrum based sharpness estimation of each image block is specifically formulated as follows:

[0029]

[0030] Wherein, b is the image block of the sampled image divided into m1x m1, -a b is the amplitude spectrum slope of the image block b; e is the base of natural logarithm, τ1 and τ2 are adjustment parameters.

[0031] Further, the spatial based sharpness estimation is performed by dividing the gray-scaled sampled image into equal image blocks, each of which has a preset pixel width overlap, calculating the spatial based sharpness estimation of each image block and combining into the spatial based sharpness estimation corresponding to the whole sampled image.

[0032] The spatial based sharpness estimation of each image block is specifically formulated as follows:

[0033]

[0034] Wherein, c is the image block of the sampled image divided into m2x m2, ξ is the image block c subdivided into small blocks of 2x2, v(ξ) is the contrast of the small block ξ, and ξ∈c represents that the small block ξ belongs to c, represents the maximum contrast of all small blocks ξ in the image block c.

[0035] Further, the matching degree evaluation unit evaluates the matching degree score of each sampled image and the reference image; extracts the feature points of the sampled image and the reference image, performs feature point matching, filters outliers through a random consistency algorithm RANSAC, obtains the number of matching pairs, and obtains the matching degree score s q，2 :

[0036]

[0037] Further, the structural similarity evaluation unit evaluates a structural similarity score of the sample image and the reference image, the structural similarity including a luminance comparison l(P q , I q ), a contrast comparison c(P q , I q ) and a structural comparison s(P q , I q ), and the specific formula is as follows:

[0038] s q，3 = [l(P q , I q )] α · [c(P q , I q )] β · [s(P q , I q )] γ ;

[0039] wherein α, β, Y are weight parameters with values greater than 0; s q，3 is the structural similarity score, P q is the sample image, I q is the reference image, and q is the serial number of the sample image;

[0040] The luminance comparison l(P q , I q ) is calculated based on the image mean value, taking the image mean value as the luminance measurement value, and the specific formula is as follows:

[0041]

[0042] μ P，q is the image mean value of the sample image, μ I，q is the image mean value of the reference image, and λ1 is a constant for avoiding the case that the denominator is 0;

[0043] The contrast comparison c(P q , I q ) is calculated based on the image standard deviation, removing the image mean value and taking the image standard deviation as the contrast measurement value, and the specific formula is as follows:

[0044]

[0045] σ P，q is the image standard deviation of the sample image, σ I，q is the image standard deviation of the reference image, and λ2 is a constant for avoiding the case that the denominator is 0;

[0046] The structural comparison s(P q , I q) image mean value is removed, and the image standard deviation is removed, and the covariance of the sampling image and the reference image is calculated, and the specific formula is as follows:

[0047]

[0048] σ PI，q is the covariance of the sampling image and the reference image; λ3 is a constant, which is used to avoid the case that the denominator is 0.

[0049] The application also provides an active illumination observation method based on image quality evaluation, which is applied to the active illumination observation system and has the following steps.

[0050] Step S1: The configuration optimization module divides the five degrees of freedom θ i，n , θ i，N of the active illumination module and the camera holder control module into M equal parts in the interval [θ i (i=1, 2,..., 5), obtains a plurality of system configurations, and obtains the corresponding input image P l , l=1, 2,..., M 5 of each system configuration. i，n , N represents the adjustment range of the i-th degree of freedom between θ i，N ;

[0051] Step S2: The corresponding input image P l , l=1, 2,..., M 5 of each system configuration is input into the image quality evaluation module, the evaluation score A l of the input image P l , l=1, 2,..., M 5 is obtained.

[0052] Step S3: The configuration optimization module performs high-low sorting according to the evaluation score A l , and obtains the input image P l with the top three evaluation scores A j , j=1, 2, 3 and the corresponding three groups of system configurations.

[0053] Step S4: The three groups of system configurations are finely searched, and the five degrees of freedom of the three groups of system configurations are divided into m equal parts, and the fine system configurations and the corresponding input image P k , k=1, 2,..., m 5 are obtained in each group.

[0054] Step S5: The input image P k , k=1, 2,..., m 5and the reference image input the image quality evaluation module to obtain the input image P k evaluation score A k , k = 1, 2,..., m 5 ;

[0055] Step S6: The configuration optimization module respectively sorts the evaluation scores A k , k = 1, 2,..., m 5 from high to low, and when the difference between the highest evaluation score and the lowest evaluation score is less than a set value, the system configuration corresponding to the highest evaluation score is the optimal configuration of each group; otherwise, continue to subdivide the search until the set condition is met.

[0056] Step S7: Compare the highest evaluation scores of the three groups of optimal configurations, and the system configuration corresponding to the highest evaluation score is the optimal configuration of the active illumination observation system.

[0057] Compared with the prior art, the present application has the following beneficial effects:

[0058] (1) The active illumination module of the present application has three degrees of freedom of pitch, yaw and brightness, and can also increase the number of lighting sources and gimbals according to actual needs. At the same time, through cooperation with the camera gimbal control module, observation pictures with different illumination intensity, angle and observation camera view angle can be obtained, which makes up for the shortcomings of the existing active illumination system that the light source position is fixed and the illumination angle is single.

[0059] (2) The existing image quality evaluation algorithm aims to evaluate the distortion degree of the image and obtain a result consistent with subjective evaluation. However, for images used for target recognition, feature detection and other tasks, it is not enough to only evaluate the distortion degree. The image quality evaluation module of the present application quantifies the clarity, matching degree and structural similarity, outputs scores with unified quantitative standard, and ensures the reliability of the evaluation result.

[0060] (3) The configuration optimization module of the present application combines the functions of the active illumination module, the camera gimbal control module and the image quality evaluation module into one, can quickly obtain multiple system configurations with good observation effect in a given interval, and then through further subdivision search, evaluation, sorting and screening, obtains the optimal configuration of the active illumination observation system. This multi-granularity configuration optimization method combines coarse search and fine positioning strategies to ensure the efficiency, accuracy and robustness of visual perception. BRIEF DESCRIPTION OF DRAWINGS

[0061] Other features, objects and advantages of the present application will become more apparent through reading the following detailed description of the non-limiting embodiments with reference to the accompanying drawings:

[0062] Figure 1 is a structural block diagram of the active illumination observation system of the present application;

[0063] Figure 2 is a working schematic diagram of the image quality evaluation module in the active illumination observation system of the present application;

[0064] Figure 3 is a flow chart of the sharpness evaluation unit in the image quality evaluation module of the present application;

[0065] Figure 4 is a flow chart of the matching degree evaluation unit in the image quality evaluation module of the present application;

[0066] Figure 5 is a flow chart of the structural similarity evaluation unit in the image quality evaluation module of the present application;

[0067] Figure 6 is a step flow chart of the active illumination observation method of the present application. DETAILED DESCRIPTION

[0068] The present application will be described in detail below with specific embodiments. The following embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of changes and improvements can be made. These are within the scope of the present application.

[0069] Example 1

[0070] The present application provides an active illumination observation system based on image quality evaluation, please refer to Figure 1 , Figure 1 is a structural block diagram of the active illumination observation system of the present application, including an active illumination module, a camera holder control module, a configuration optimization module and an image quality evaluation module;

[0071] The host computer includes a configuration optimization module and a quality evaluation module, and the active illumination module and the camera holder control module are respectively electrically connected with the host computer or wirelessly connected with the configuration optimization module for data interaction. The configuration optimization module adjusts the attitude of the active illumination module and the camera holder control module to obtain a plurality of input images; the configuration optimization module provides the input images and the reference images for the image quality evaluation module;

[0072] The image quality evaluation module performs weighted calculation on the sharpness evaluation, the matching degree evaluation and the structural similarity evaluation of the input images and the reference images to obtain the evaluation score of the input images and transmits the evaluation score to the configuration optimization module;

[0073] The configuration optimization module provides an input image evaluation score ranking to the image quality evaluation module, and the highest evaluation score after the selection is obtained as the optimal active illumination observation system configuration.

[0074] Further, the active illumination module includes at least one illumination light source and an illumination light source holder, the illumination light source holder has three degrees of freedom of pitch, yaw and brightness, and controls the attitude and brightness of the illumination light source. On this basis, the number of illumination light sources and holders can be increased according to actual requirements.

[0075] Further, the camera holder control module includes an observation camera and an observation camera holder, the observation camera holder has two degrees of freedom of pitch and yaw, and is used for controlling and adjusting the attitude of the observation camera to provide an input image for the image quality evaluation module.

[0076] Please refer to Figure 2 , Figure 2 It is a working schematic view of the image quality evaluation module in the active illumination observation system.

[0077] Further, the image quality evaluation module includes a sharpness evaluation unit, a matching degree evaluation unit and a structural similarity evaluation unit.

[0078] The sharpness evaluation unit evaluates a sharpness score of the input image, the sharpness score includes a local amplitude spectrum-based sharpness estimation and a spatial-based sharpness estimation; and the sharpness score is used for quantifying the perceived sharpness of the input image.

[0079] The matching degree estimation unit evaluates a matching degree score of the input image and the reference image, the matching degree score is a ratio of the number of matching pairs of feature points of the input image and the reference image to the feature points of the reference image.

[0080] The structural similarity evaluation unit evaluates a structural similarity score of the input image and the reference image, the structural similarity score includes a brightness comparison, a contrast comparison and a structure comparison, and a weighted product of the three elements.

[0081] Further, the image quality evaluation module further includes that the input image is down-sampled several times to construct an image pyramid, the sampled images are used to cover the input image of different resolutions to obtain more details; the image quality evaluation module respectively evaluates the sharpness score, the matching degree score and the structural similarity score of each sampled image in the image pyramid, calculates a score of each sampled image by weighted summation, and finally synthesizes all the sampled image scores to obtain an evaluation score of the input image.

[0082] The specific expression formula of the score of the sampled image is as follows:

[0083] a q = ω1s q，1 + ω2sq，2 + ω3s q，3 , q = 0, 1, …, Q;

[0084] wherein, a q is the score of the sampled image, Q is the number of times of down-sampling the input image, q is the serial number of the sampled image, ω1, ω2, ω3 are weight parameters, satisfying ω1+ ω2+ ω3= 1; s q，1 is the sharpness score, s q，2 is the matching score, s q，3 is the structural similarity score;

[0085] The specific expression formula of the evaluation score of the input image is as follows:

[0086]

[0087] wherein, A is the evaluation score of the input image, q = 0 represents starting from 0 to calculate in increasing order; wherein W q is a weight parameter and indicates that the weight summation of the Q+1 sampled images is 1.

[0088] Further, please refer to Figure 3 , Figure 3 is the flow chart of the sharpness evaluation unit in the image quality evaluation module of the present application. The sharpness evaluation unit is used to evaluate the sharpness score of each sampled image, and the sharpness score includes the sharpness estimation based on the local amplitude spectrum and the sharpness estimation based on the space;

[0089] The specific expression formula of the sharpness score is as follows:

[0090] s q，1 = t1(P q ) η + t2(P q ) 1-η ;

[0091] wherein, η is a weight parameter, 0≤η≤1, s q，1 is the sharpness score of the sampled image, t1(P q ) is the sharpness estimation of the sampled image based on the local amplitude spectrum, and t2(P q ) is the sharpness estimation of the sampled image based on the space.

[0092] Further, based on the local amplitude spectrum-based sharpness estimation, by dividing the gray-scale sampling image into equal image blocks, each image block has a preset pixel width overlap, after calculating the amplitude spectrum slope of each image block, the local amplitude spectrum-based sharpness estimation of each image block is calculated, and the local amplitude spectrum-based sharpness estimation of each image block is integrated using the Hanning window to generate the local amplitude spectrum-based sharpness estimation of the sampling image;

[0093] The specific calculation process is as follows:

[0094] (1) The sampling image is grayed and divided into several image blocks;

[0095] (2) Calculate the amplitude spectrum slope of each image block; z b (f) The amplitude spectrum of the image block b is obtained by two-dimensional discrete Fourier transform for the sum of the amplitude spectrum of all directions y b (f, θ) (f is the radial frequency, and θ is the direction):

[0096] z b (f) = Σ θ |y b (f, θ) |;

[0097] (3) Calculate the local amplitude spectrum-based sharpness estimation of each image block, and the specific formula is as follows: Wherein, b is the sampling image divided into m1×m1 image blocks, -α b is the amplitude spectrum slope of the image block b; e is the base of natural logarithm, τ1 and τ2 are adjustment parameters.

[0098] (4) Combine the sharpness estimation results of all image blocks into the local amplitude spectrum-based sharpness estimation of the whole sampling image.

[0099] Further, based on the local amplitude spectrum-based sharpness estimation, by dividing the gray-scale sampling image into equal image blocks, each image block has a preset pixel width overlap, after calculating the amplitude spectrum slope of each image block, the local amplitude spectrum-based sharpness estimation of each image block is calculated, and the local amplitude spectrum-based sharpness estimation of each image block is integrated using the Hanning window to generate the local amplitude spectrum-based sharpness estimation of the sampling image;

[0100] The specific calculation process is as follows:

[0101] (1) The input gray-scale sampling image is divided into several image blocks;

[0102] (2) Calculate the local amplitude spectrum-based sharpness estimation of each image block, and the specific formula is as follows, Wherein, c is the sampling image divided into m2x m2 image blocks, ξ is the image block c subdivided into small blocks of 2x2, v(ξ) is the contrast of small block ξ, ξ∈c represents that small block ξ belongs to c, The maximum contrast of all small blocks ξ in image block c is represented as:

[0103] (3) The sharpness estimation results of all image blocks are combined into the whole image to obtain the spatial-based sharpness estimation.

[0104] Further, refer to Figure 4 , Figure 4 The flow chart of the matching degree evaluation unit in the image quality evaluation module of the application is shown in the figure; the matching degree evaluation unit evaluates the matching degree score of each sampling image and reference image; the feature points of the sampling image and the reference image are extracted, the feature point matching is performed, the outliers are filtered through the random consistency algorithm RANSAC, the number of matching pairs is obtained, and the matching degree score s is obtained q，2 :

[0105]

[0106] Further, refer to Figure 5 , Figure 5 The flow chart of the structural similarity evaluation unit in the image quality evaluation module of the application is shown in the figure; the structural similarity evaluation unit evaluates the structural similarity score of the sampling image and the reference image, and the structural similarity includes the brightness comparison l(P q , I q ), the contrast comparison c(P q , I q ) and the structure comparison s(P q , I q ), and the specific formula is as follows:

[0107] s q，3 =[l(P q , I q )] α ·[c(P q , I q )] β ·[s(P q , I q )]γ

[0108] Wherein, α, β, γ are weight parameters with values greater than 0; s q，3 is the structural similarity score, P q is the sampling image, I q is the reference image, and q is the serial number of the sampling image;

[0109] The brightness comparison l(P q , I q) is calculated based on the image mean value, taking the image mean value as the value of the luminance measurement, and the specific formula is as follows:

[0110]

[0111] μ P，q is the image mean value of the sampling image, μ I，q is the image mean value of the reference image λ1 is a constant, which is used to avoid the case that the denominator is 0;

[0112] The contrast comparison c(P q , I q ) is calculated based on the image standard deviation, removes the image mean value, separates the luminance, and takes the image standard deviation as the contrast estimate value, and the specific formula is as follows:

[0113]

[0114] σ P，q is the image standard deviation of the sampling image σ I，q is the image standard deviation of the reference image λ2 is a constant, which is used to avoid the case that the denominator is 0;

[0115] The structure comparison s(P q , I q ) removes the image mean value and the image standard deviation, that is, after separating the luminance and contrast, it is calculated through the covariance of the sampling image and the reference image, and the specific formula is as follows:

[0116]

[0117] σ PI，q is the covariance of the sampling image and the reference image, λ3 is a constant, which is used to avoid the case that the denominator is 0.

[0118] Embodiment 2

[0119] The application also provides an active illumination observation method based on image quality evaluation, which is applied to the active illumination observation system, please refer to Figure 6 , Figure 6 is the step flow chart of the active illumination observation method of the application, including the following steps:

[0120] Step S1: The configuration optimization module controls the five degrees of freedom θ i，n , θ i，N ] interval of the active illumination module and the camera holder control module θ i(i = 1, 2, ..., 5) are equally divided into M equal parts to obtain several system configurations, and the input image P corresponding to each system configuration is obtained. l l = 1, 2, ..., M 5 n and N represent the adjustment range of the i-th degree of freedom within θ. i，n to θ i，N between;

[0121] Step S2: Input the corresponding image P for each system configuration l l = 1, 2, ..., M 5 The input image quality assessment module, along with the reference image, yields the input image P. l Assessment score A l l = 1, 2, ..., M 5 ;

[0122] Step S3: Configure the optimization module based on the evaluation score A l Sort the samples from highest to lowest to obtain the evaluation score A. l The top three input images P j j = 1, 2, 3 and the corresponding three sets of system configurations;

[0123] Step S4: Perform a fine-grained search on the three sets of system configurations. Divide the five degrees of freedom of each of the three sets of system configurations into m equal parts, and obtain the fine-grained system configuration and corresponding input image P within each set. k k = 1, 2, ..., m 5 ;

[0124] Step S5: Input image P k k = 1, 2, ..., m 5 The input image quality assessment module, along with the reference image, yields the input image P. k Assessment score A k k = 1, 2, ..., m 5 ;

[0125] Step S6: The configuration optimization module will assign evaluation scores A to the three sets of system configurations respectively. k k = 1, 2, ..., m 5 Sort the data from highest to lowest. If the difference between the highest and lowest evaluation scores is less than a set value, the system configuration corresponding to the highest evaluation score is set as the optimal configuration for each group that has been found. Otherwise, continue to refine the search until the set conditions are met.

[0126] Step S7: Compare the highest evaluation scores under the three optimal configurations, and configure the system corresponding to the highest evaluation score as the optimal configuration of the active illumination observation system.

[0127] Although the present application has been disclosed with reference to the preferred embodiments, it is not intended to limit the present application, and any person skilled in the art can make possible changes and modifications to the technical solutions of the present application using the disclosed methods and technical contents without departing from the spirit and scope of the present application. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present application without departing from the technical solutions of the present application shall fall within the protection scope of the technical solutions of the present application.

Claims

1. An active illumination observation system based on image quality assessment, characterized in that, The application relates to an active illumination system and a method for optimizing the configuration of the active illumination system. The active illumination module, the camera gimbal control module, the configuration optimization module and the image quality evaluation module are included. The host computer includes the configuration optimization module and the quality evaluation module, the active illumination module and the camera gimbal control module are electrically connected with the host computer or wirelessly connected with the configuration optimization module for data interaction, the configuration optimization module adjusts the postures of the active illumination module and the camera gimbal control module to obtain a plurality of input images, and the configuration optimization module provides the input images and reference images for the image quality evaluation module. The image quality evaluation module evaluates the input images and the reference images in terms of sharpness, matching degree and structural similarity, obtains an evaluation score of the input images through weighted calculation and inputs the evaluation score into the configuration optimization module. The image quality evaluation module includes a sharpness evaluation unit, a matching degree evaluation unit and a structural similarity evaluation unit. The sharpness evaluation unit evaluates the sharpness score of the input images, the sharpness score includes local amplitude spectrum-based sharpness estimation and space-based sharpness estimation, and the sharpness score is used for quantifying the perceived sharpness of the input images. The matching degree estimation unit evaluates the matching degree score of the input images and the reference images, the matching degree score is the matching pair number of the feature points of the input images and the feature points of the reference images, and the matching degree score is proportional to the feature points of the reference images. The structural similarity evaluation unit evaluates the structural similarity score of the input images and the reference images, the structural similarity score includes brightness comparison, contrast comparison and structure comparison, and the structural similarity score is the weighted product of the three elements. The configuration optimization module provides the evaluation score of the input images for the image quality evaluation module, and the highest evaluation score obtained after sorting and selecting the evaluation score is regarded as the optimal active illumination observation system configuration.

2. The active illumination viewing system of claim 1, wherein, The active illumination module includes at least one illumination light source and an illumination light source gimbal, the illumination light source gimbal has three degrees of freedom of pitch, yaw and brightness, and the posture and brightness of the illumination light source are controlled.

3. The active illumination viewing system of claim 1, wherein, The camera gimbal control module includes an observation camera and an observation camera gimbal, the observation camera gimbal has two degrees of freedom of pitch and yaw, and is used for controlling and adjusting the posture of the observation camera to provide the input images for the image quality evaluation module.

4. The active illumination viewing system of claim 1, wherein, The image quality evaluation module further includes that the input images are down-sampled for several times to construct an image pyramid, the sampling images are used for covering input images of different resolutions to obtain more details, the sharpness score, the matching degree score and the structural similarity score of each sampling image in the image pyramid are evaluated, the score of each sampling image is calculated through weighted summation, and finally the scores of all the sampling images are integrated to obtain the evaluation score of the input images. The specific expression formula of the score of the sampling image is as follows: ； wherein, is a score of the sampled image, Q is a number of times of down-sampling on the input image, q is a serial number of the sampled image, is a weight parameter, satisfying ; is the sharpness score, is the matching score, is the structural similarity score; The specific expression formula of the evaluation score of the input image is as follows: wherein A is the evaluation score of the input image, q=0 represents a calculation starting from 0; wherein is a weight parameter and , represents that the weight sum of Q+1 sampling images is 1.

5. The active illumination viewing system of claim 4, wherein, The sharpness evaluation unit further comprises evaluating the sharpness score of each of the sample images, the sharpness score comprising the local amplitude spectrum based sharpness estimation and the spatial based sharpness estimation; The sharpness score is specifically expressed as follows: ； wherein η is a weight parameter, , is the sharpness score for the sampled image, is the sharpness estimate for the sampled image based on local amplitude spectrum, is the sharpness estimate for the sampled image based on space.

6. The active illumination viewing system of claim 5, wherein, The local amplitude spectrum based sharpness estimation is performed by dividing the gray-scale sample image into equal image blocks, each of the image blocks having a preset pixel width overlap, calculating the amplitude spectrum slope of each of the image blocks, and then performing the local amplitude spectrum based sharpness estimation of each of the image blocks, and using a Hanning window to integrate the local amplitude spectrum based sharpness estimation of each of the image blocks to generate the local amplitude spectrum based sharpness estimation of the sample image: The local amplitude spectrum based sharpness estimation of each of the image blocks is specifically expressed as follows: ； wherein b is the image block into which the sampling image is equally divided, is the amplitude spectrum slope of the image block b; the is the base of the natural logarithm, the and are adjustment parameters.​ 7. The active illumination viewing system of claim 6, wherein, The spatial based sharpness estimation is performed by dividing the gray-scale sample image into equal image blocks, each of the image blocks having a preset pixel width overlap, calculating the spatial based sharpness estimation of each of the image blocks, and then combining the spatial based sharpness estimation of each of the image blocks into the spatial based sharpness estimation corresponding to the entire sample image; The spatial based sharpness estimation of each of the image blocks is specifically expressed as follows, wherein c is the image block into which the sampled image is equally divided , c is the image block into which the sampled image is equally divided , c is the image block into which the sampled image is equally divided , c is the image block into which the sampled image is equally divided , c is the image block into which the sampled image is equally divided , c is the image block into which the sampled image is equally divided 8. The active illumination viewing system of claim 4, wherein, The matching degree evaluation unit evaluates the matching degree score of each of the sample images and the reference image, extracts feature points of the sample images and the reference image, performs feature point matching, filters outliers through a random consensus algorithm (RANSAC), obtains a number of matching pairs, and obtains the matching degree score : 。 9. The active illumination viewing system of claim 4, wherein, The structural similarity assessment unit assesses the structural similarity score of the sample image and the reference image, the structural similarity including a brightness comparison , a contrast comparison , and a structural comparison , and the specific formula is as follows: ； wherein, are weight parameters with values greater than 0; is the structural similarity score, is a sample image, is a reference image, q is the index of the sample image; The luminance comparison Based on the image mean value, taking the image mean value as the value of the luminance measure, the specific formula is as follows: ； The is the image mean of the sampling image, the is the image mean of the reference image, is a constant to avoid a division by zero; The contrast comparison Based on the image standard deviation, the image mean is removed, and the image standard deviation is taken as the contrast estimation value. The specific formula is as follows: ； The is the image standard deviation of the sampling image, the is the image standard deviation of the reference image, is a constant to avoid a case where the denominator is 0; The structure comparison After removing the image mean and the image standard deviation, the covariance between the sample image and the reference image is calculated, and the specific formula is as follows: ； The is the covariance of the sampled image and the reference image; is a constant to avoid division by zero.

10. An active illumination observation method based on image quality evaluation, applied to the active illumination observation system according to any one of claims 1 to 9, characterized in that, The method comprises the following steps: Step S1: configuring the optimization module to divide the five degrees of freedom of the active illumination module and the camera gimbal control module into M equal parts within the interval , obtaining a plurality of system configurations, and obtaining the input image corresponding to each of the system configurations . , wherein n and N represent the adjustment range of the ith degree of freedom is between and .​ Step S2: inputting the input image corresponding to each of the system configurations to an input image quality evaluation module to obtain evaluation scores of the input image , ;​ Step S3: the configuration optimization module obtains the evaluation score according to the evaluation score The evaluation score is obtained by high-low sorting The top three input images And the corresponding three groups of system configurations Step S4: performing a fine search of the three sets of system configurations, respectively dividing the five degrees of freedom of the three sets of system configurations into equal parts, obtaining a fine system configuration and a corresponding input image in each set ; Step S5: inputting the input image and the reference image into the image quality evaluation module to obtain an evaluation score of the input image , , ;​​ Step S6: the configuration optimization module respectively ranks the evaluation scores of the three groups of system configurations , performs high-low sorting, when the gap between the highest evaluation score and the lowest evaluation score is less than a set value, the system configuration corresponding to the highest evaluation score is the optimal configuration of each group; otherwise, continue to perform subdivision search until the set condition is met; Step S7: comparing the highest evaluation scores under the three groups of optimal configurations, and taking the system configuration corresponding to the highest evaluation score as the optimal configuration of the active illumination observation system.

Images