Method and system for enhancing visual perception of a patrol robot based on dynamic light source compensation

Abstract

The application provides a kind of based on dynamic light source compensation's inspection robot vision perception enhancement method and system, it is related to visual perception technical field.The method comprises: obtaining current detection area image;Determine whether to enhance visual perception;If it is determined to enhance visual perception, obtain the position information of inspection robot, and construct the three-dimensional illumination environment model of underground sewage treatment plant;Obtain the highest reflectivity;Obtain humidity influence factor;Determine dynamic illumination compensation demand coefficient;Determine illumination intensity and color temperature;Obtain the undetected area image of the preset distance at the advancing direction of inspection robot;Determine the illumination angle;According to the illumination intensity, the color temperature and the illumination angle, using the controllable light compensation device of inspection robot is dynamically light source compensation.According to the application, illumination angle, intensity and color temperature can be dynamically adjusted according to the real-time position of inspection robot and visual perception demand, and the visual perception enhancement of shadow area and light reflection area is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of visual perception, and particularly relates to a visual perception enhancement method and system for inspection robots based on dynamic light source compensation. BACKGROUND

[0002] In the prior art, although the light source can be adjusted and compensated and the perception range can be expanded, the interference of shadow area, light reflection or humidity diffusion on the visual system of the inspection robot is not considered, that is, the illumination angle, intensity and color temperature cannot be dynamically adjusted according to the real-time position of the inspection robot and the visual perception demand.

[0003] The information disclosed in the background section of this application is only intended to deepen the understanding of the general background of the application and should not be considered as admitting or implying in any form that the information constitutes prior art known to those skilled in the art. SUMMARY

[0004] The present application provides a visual perception enhancement method and system for inspection robots based on dynamic light source compensation, which can solve the technical problem that the illumination angle, intensity and color temperature cannot be dynamically adjusted according to the real-time position of the inspection robot and the visual perception demand in the related art.

[0005] According to a first aspect of the present application, a visual perception enhancement method for inspection robots based on dynamic light source compensation is provided, comprising: acquiring a current detection area image through a visual sensor built in an inspection robot; determining whether to enhance visual perception according to the detection area image; if it is determined to enhance visual perception, acquiring position information of the inspection robot and constructing a three-dimensional lighting environment model of an underground sewage treatment plant, wherein the three-dimensional lighting environment model comprises fixed light source positions, equipment occlusion relationships, pipeline layouts and reflective area parameters; acquiring a maximum reflectivity of the current detection area according to the position information and the three-dimensional lighting environment model; acquiring a humidity influence factor of the current detection area through a humidity sensor carried by the inspection robot; determining a dynamic lighting compensation demand coefficient according to the position information, the three-dimensional lighting environment model, the maximum reflectivity and the humidity influence factor; determining lighting intensity and color temperature according to the dynamic lighting compensation demand coefficient, the detection area image, the maximum reflectivity and the humidity influence factor; acquiring an undetected area image at a preset distance in the forward direction of the inspection robot through a depth camera; determining an illumination angle according to the preset distance, the undetected area image and the detection area image; and performing dynamic light source compensation using a controllable light compensation device carried by the inspection robot according to the lighting intensity, the color temperature and the illumination angle.

[0006] Further, the method further comprises: determining whether to enhance visual perception according to the detection area image, including: obtaining a shadow area and a reflection area according to the detection area image; determining to enhance visual perception if a sum of the shadow area and the reflection area is greater than or equal to a preset area threshold; and determining not to enhance visual perception if the sum of the shadow area and the reflection area is less than the preset area threshold.

[0007] Further, the method further comprises: determining a dynamic light compensation demand coefficient according to the position information, the three-dimensional light environment model, the maximum reflectivity, and the humidity influence factor, including: obtaining an Euclidean distance from the inspection robot to a nearest fixed light source position according to the position information and the three-dimensional light environment model; obtaining a minimum distance at which the light intensity of the fixed light source is not attenuated and a maximum distance at which the light intensity of the fixed light source is attenuated to an ambient light level; determining a normalized distance according to the Euclidean distance, the minimum distance, and the maximum distance; and determining the dynamic light compensation demand coefficient according to the normalized distance, the maximum reflectivity, and the humidity influence factor.

[0008] Further, the method further comprises: determining the dynamic light compensation demand coefficient according to the normalized distance, the maximum reflectivity, and the humidity influence factor, including: determining the dynamic light compensation demand coefficient according to a formula wherein D is the normalized distance, is a light attenuation threshold, is the maximum reflectivity, and H is the humidity influence factor.

[0009] Further, the method further comprises: determining the light intensity and the color temperature according to the dynamic light compensation demand coefficient, the detection area image, the maximum reflectivity, and the humidity influence factor, including: obtaining a gray value of each pixel point according to the detection area image; determining a texture complexity of the detection area according to the gray value of the pixel point; obtaining a moving speed of the inspection robot, an exposure time, and a detection area image width; determining a motion blur risk according to the moving speed of the inspection robot, the exposure time, and the detection area image width; determining the light intensity according to the texture complexity, the motion blur risk, and the dynamic light compensation demand coefficient; obtaining a reference color temperature; and determining the color temperature according to the reference color temperature, the maximum reflectivity, and the humidity influence factor.

[0010] Further, the method further comprises: determining the light intensity according to the texture complexity, the motion blur risk, and the dynamic light compensation demand coefficient, including: determining the light intensity I according to a formula is a reference light intensity, is the dynamic light compensation demand coefficient, is the texture complexity, ​​For motion blur risk.

[0011] Further, determining the color temperature according to the reference color temperature, the maximum reflectivity and the humidity influence factor comprises: determining the color temperature T according to the formula determining the color temperature T according to the formula wherein, T is the color temperature, is the reference color temperature, R is the maximum reflectivity, and H is the humidity influence factor.

[0012] Further, determining the illumination angle according to the preset distance, the undetected area image and the detected area image comprises: obtaining a future shadow area according to the undetected area image; obtaining a shadow area according to the detected area image; and determining the illumination angle according to the future shadow area, the shadow area and the preset distance.

[0013] Further, determining the illumination angle according to the future shadow area, the shadow area and the preset distance comprises: determining the illumination angle according to the formula wherein, is the reference projection angle, is the future shadow area, is the shadow area, is the preset distance, is a preset minimum illumination angle, is a preset maximum illumination angle, max is a maximum value function, and min is a minimum value function.

[0014] ​According to a second aspect of the present application, a visual perception enhancement system based on dynamic light source compensation for a patrol robot is provided, comprising: a detection area image module configured to acquire a current detection area image by a visual sensor built in the patrol robot; a judgment module configured to determine whether to enhance visual perception according to the detection area image; a position information and construction model module configured to acquire position information of the patrol robot and construct a three-dimensional light environment model of an underground sewage treatment plant if it is determined to enhance visual perception, wherein the three-dimensional light environment model comprises fixed light source positions, equipment occlusion relationships, pipeline layouts and reflective area parameters; a highest reflectivity module configured to acquire a highest reflectivity of the current detection area according to the position information and the three-dimensional light environment model; a humidity influence factor module configured to acquire a humidity influence factor of the current detection area by a humidity sensor carried by the patrol robot; a dynamic light compensation demand coefficient module configured to determine a dynamic light compensation demand coefficient according to the position information, the three-dimensional light environment model, the highest reflectivity and the humidity influence factor; a light intensity and color temperature module configured to determine a light intensity and a color temperature according to the dynamic light compensation demand coefficient, the detection area image, the highest reflectivity and the humidity influence factor; an undetected area image module configured to acquire an undetected area image at a preset distance in a forward direction of the patrol robot by a depth camera; an illumination angle module configured to determine an illumination angle according to the preset distance, the undetected area image and the detection area image; and a dynamic light source compensation module configured to perform dynamic light source compensation using a controllable light compensation device carried by the patrol robot according to the light intensity, the color temperature and the illumination angle.

[0015] Technical effects: According to the present application, by acquiring the current detection area image and determining whether the visual perception needs to be enhanced based on image analysis, the real-time response to the change of the lighting environment can be realized, the lighting compensation demand coefficient can be adjusted based on the actual environmental demand, the stability and accuracy of visual perception can be improved, the lighting angle, intensity and color temperature can be dynamically adjusted according to the real-time position of the inspection robot and the visual perception demand, the visual perception enhancement of the shadow area and the reflection area can be realized, which helps to improve the inspection efficiency and reduce the risk of accidents. When determining the dynamic lighting compensation demand coefficient, the dynamic lighting compensation demand coefficient can be determined by the normalized distance, the maximum reflectivity and the humidity influence factor, the influence of the space position, the material reflection characteristic and the environmental humidity on the lighting compensation demand is comprehensively considered, which helps to more accurately evaluate the actual demand of lighting compensation, and the dynamic lighting compensation demand coefficient can be dynamically adjusted according to the change of the position and the environmental condition to adapt to different lighting scenes, thereby improving the accuracy and reliability of visual detection. When determining the lighting intensity, the lighting intensity can be determined by the texture complexity, the motion blur risk and the dynamic lighting compensation demand coefficient, the lighting intensity can be dynamically adjusted by combining the complexity of the texture in the image, the influence of the movement of the inspection robot on the image quality and the demand of the lighting condition with the change of the environment, so that the inspection robot can flexibly adjust the lighting condition according to the actual situation to adapt to the complex and changeable environment, thereby improving the image quality and the detection accuracy. When determining the color temperature, the color temperature can be determined by the reference color temperature, the maximum reflectivity and the humidity influence factor, the demand of color temperature adjustment can be more comprehensively evaluated by combining the reflection ability of the detection area material to the light and the influence of the environmental humidity on the light propagation, which helps to improve the image quality and reduce the color deviation or distortion caused by improper color temperature, thereby improving the detection accuracy. When determining the lighting angle, the shadow space gradient can be used for lighting angle control, the shadow change trend can be predicted by the shadow gradient, and the lighting angle can be adjusted in advance, so that the lighting angle can be dynamically adjusted according to the actual change of the shadow, the influence of the shadow on the detection area can be minimized, and the inspection robot can obtain clear images in various complex shadow environments, thereby improving the visual perception ability of the inspection robot.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory, but not limiting the present application. Other features and aspects of the present application will be more apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to make the technical solutions in the embodiments of the present application or the prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description only need to be some embodiments of the present application, and all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0018] Figure 1 An exemplary flowchart of a visual perception enhancement method of a patrol robot based on dynamic light source compensation according to an embodiment of the present application is shown.

[0019] Figure 2 An exemplary flowchart of judging whether to enhance visual perception according to an embodiment of the present application is shown.

[0020] Figure 3 An exemplary flowchart of calculating a dynamic light compensation demand coefficient according to an embodiment of the present application is shown.

[0021] Figure 4 An exemplary flowchart of calculating light intensity and color temperature according to an embodiment of the present application is shown.

[0022] Figure 5 An exemplary flowchart of calculating an illumination angle according to an embodiment of the present application is shown.

[0023] Figure 6 An exemplary block diagram of a visual perception enhancement system of a patrol robot based on dynamic light source compensation according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] In order to make the technical solutions in the embodiments of the present application or the prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description only need to be some embodiments of the present application, and all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0025] The technical solutions of the present application will be described in detail in the following specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in some embodiments.

[0026] Figure 1An exemplary flowchart of a method for enhancing visual perception of an inspection robot based on dynamic light source compensation according to an embodiment of the present application is shown. The method comprises: step S1, acquiring a current detection area image by a visual sensor built in the inspection robot; step S2, determining whether to enhance visual perception according to the detection area image; step S3, if it is determined to enhance visual perception, acquiring position information of the inspection robot and constructing a three-dimensional lighting environment model of the underground sewage treatment plant, wherein the three-dimensional lighting environment model comprises fixed light source positions, equipment occlusion relationships, pipeline layouts and reflective area parameters; step S4, acquiring a highest reflectivity of the current detection area according to the position information and the three-dimensional lighting environment model; step S5, acquiring a humidity influence factor of the current detection area by a humidity sensor carried by the inspection robot; step S6, determining a dynamic lighting compensation demand coefficient according to the position information, the three-dimensional lighting environment model, the highest reflectivity and the humidity influence factor; step S7, determining lighting intensity and color temperature according to the dynamic lighting compensation demand coefficient, the detection area image, the highest reflectivity and the humidity influence factor; step S8, acquiring an undetected area image at a preset distance in the advancing direction of the inspection robot by a depth camera; step S9, determining an illumination angle according to the preset distance, the undetected area image and the detection area image; and step S10, performing dynamic light source compensation by using a controllable light compensation device carried by the inspection robot according to the lighting intensity, the color temperature and the illumination angle.

[0027] The method for enhancing visual perception of an inspection robot based on dynamic light source compensation according to an embodiment of the present application can acquire a current detection area image and determine whether to enhance visual perception based on image analysis, can respond to lighting environment changes in real time, can adjust based on actual environment demand through a lighting compensation demand coefficient, can improve the stability and accuracy of visual perception, can dynamically adjust an illumination angle, intensity and color temperature according to the real-time position of the inspection robot and the demand for visual perception, can enhance visual perception of shadow areas and reflective areas, and can help improve inspection efficiency and reduce accident risks.

[0028] According to an embodiment of the present application, in step S1, the inspection robot can move, rotate or adjust the posture so that the visual sensor faces the detection area. The visual sensor can collect image data of the current detection area, which contains visual information of the detection area, thereby obtaining the detection area image.

[0029] According to an embodiment of the present application, in step S2, the detection area image is analyzed to identify shadow areas and reflective areas of the current detection area, and then it is determined whether to enhance visual perception.

[0030] Figure 2An exemplary flow chart for determining whether to enhance visual perception according to an embodiment of the present application is shown in the figure;

[0031] According to an embodiment of the present application, step S2 comprises: step S21, obtaining a shadow area and a highlight area according to the detection area image; step S22, if the sum of the shadow area and the highlight area is greater than or equal to a preset area threshold, determining to enhance visual perception; step S23, if the sum of the shadow area and the highlight area is less than the preset area threshold, determining not to enhance visual perception.

[0032] According to an embodiment of the present application, the detection area image is converted into a grayscale image, and the shadow area is separated by threshold segmentation (for example, Otsu algorithm). The Otsu algorithm obtains a grayscale threshold of 50 (the grayscale range is 0-255), that is, the pixels with a grayscale lower than the grayscale threshold are shadow pixels. The shadow area is obtained by multiplying the total number of shadow pixels by the single-pixel area, wherein the single-pixel area is the ratio of the detection area size to the image resolution. For example, if the detection area size is 1 m×1 m and the image resolution is 1000×1000 pixels, the single-pixel area is 1×10 -6 m 2 In the HSV space, the highlight area has a high brightness, and the brightness threshold is 200, that is, the pixels with a brightness higher than the brightness threshold are highlight pixels. The highlight area is obtained by multiplying the total number of highlight pixels by the single-pixel area. The sum of the shadow area and the highlight area reflects the degree of visual interference caused by shadows and highlights in the detection area image. If the sum of the shadow area and the highlight area is greater than or equal to a preset area threshold (for example, 0.3 m 2 ), it is determined that the detection area image quality is greatly affected, and the visual perception is limited. It is determined to enhance the visual perception and improve the image clarity and recognition accuracy. If the sum of the shadow area and the highlight area is less than the preset area threshold, it is determined that the detection area image quality is acceptable, and the visual perception is not significantly disturbed. It is determined not to enhance the visual perception and maintain the existing visual perception state.

[0033] According to an embodiment of the present application, in step S3, the position information of the inspection robot in the underground sewage treatment plant is obtained according to the positioning system, and a three-dimensional lighting environment model is constructed, that is, all fixed light source positions, equipment shielding relationships, pipeline layouts, and highlight area parameters in the underground sewage treatment plant are input into the three-dimensional lighting environment model. There are highlight areas with different surface materials in the underground sewage treatment plant, for example, metal pipeline surfaces, plastics, etc. The corresponding highlight area parameters are set, for example, the reflectivity of stainless steel pipes is 0.7, the reflectivity of concrete walls is 0.3, and the reflectivity of plastic equipment is 0.5. The lighting distribution in the underground sewage treatment plant can be comprehensively and accurately reflected, providing data support for subsequent light source compensation and visual perception enhancement.

[0034] According to an embodiment of the present application, in step S4, the position information of the inspection robot is matched with the three-dimensional lighting environment model, and based on the position information, a specific range of the current detection area is defined in the three-dimensional lighting environment model, which is determined according to the visual sensor field of view range of the inspection robot or a preset detection task, wherein each position information of the inspection robot corresponds to a specific detection area, and each detection area is individually covered by a fixed light source. Various surface materials (for example, metal, concrete, etc.) in the detection area are identified, and the highest reflectivity in the current detection area is determined by querying the light reflection area parameters in the three-dimensional lighting environment model.

[0035] According to an embodiment of the present application, in step S5, the inspection robot starts the humidity sensor device carried thereby, can sense and measure the humidity in the environment, obtains humidity data, and calculates the humidity influence factor according to the humidity influence factor calculation formula, wherein H is the humidity influence factor, W is the humidity data, is a saturated humidity threshold, and min is a minimum value function. By taking the minimum value of the saturated humidity threshold (for example, 90%) and 1, the humidity data can be normalized to obtain the humidity influence factor. The greater the humidity influence factor, the more humid the environment at the current position of the inspection robot, for example, the case of humidity diffusion.

[0036] According to an embodiment of the present application, in step S6, the dynamic lighting compensation demand coefficient is determined according to the position information, the three-dimensional lighting environment model, the highest reflectivity and the humidity influence factor.

[0037] Figure 3 An exemplary flowchart for calculating the dynamic lighting compensation demand coefficient according to an embodiment of the present application is shown.

[0038] According to an embodiment of the present application, step S6 includes: step S61, obtaining the Euclidean distance from the inspection robot to the position of the nearest fixed light source according to the position information and the three-dimensional lighting environment model; step S62, obtaining the minimum distance of the fixed light source lighting intensity without attenuation and the maximum distance of the lighting intensity attenuation to the ambient light level; step S63, determining the normalized distance according to the Euclidean distance, the minimum distance and the maximum distance; and step S64, determining the dynamic lighting compensation demand coefficient according to the normalized distance, the highest reflectivity and the humidity influence factor.

[0039] According to one embodiment of the present application, the relative position relationship between the inspection robot and the fixed light source in space is determined by calculating the straight-line distance (i.e., the Euclidean distance) between the inspection robot and the nearest fixed light source. The minimum distance (e.g., 0.5 m) at which the light intensity emitted by the fixed light source remains constant and is not affected by the distance, and the maximum distance (e.g., 10 m) at which the light intensity of the fixed light source decays to the level of ambient light, beyond which the fixed light source no longer has a significant illuminating effect on the detection area. According to the normalized distance calculation formula, where D is the normalized distance, d is the Euclidean distance from the inspection robot to the position of the nearest fixed light source, is the maximum distance, is the minimum distance. By combining the maximum distance and the minimum distance with the maximum function, the normalized distance can be obtained, which is greater, the farther the current position of the inspection robot is from the fixed light source, i.e., the illumination of the detection area is insufficient, and more illumination compensation is required. The greater the maximum reflectivity, the stronger the reflection of the detection area, and the lower the illumination compensation. The greater the humidity influence factor, the more humid the environment at the current position of the inspection robot, and the more serious the light scattering, and the more illumination compensation is required. By the relationship between the normalized distance, the maximum reflectivity, the humidity influence factor, and the illumination compensation requirement, the dynamic illumination compensation requirement coefficient is determined.

[0040] According to one embodiment of the present application, the dynamic illumination compensation requirement coefficient is determined according to the normalized distance, the maximum reflectivity, and the humidity influence factor, including: determining the dynamic illumination compensation requirement coefficient according to formula (1) ,

[0041] (1),

[0042] where D is the normalized distance, is the illumination decay threshold, is the maximum reflectivity, and H is the humidity influence factor.

[0043] According to one embodiment of the present application, in formula (1), is a monotonically increasing S-shaped function for mapping the input to (0, 1) to achieve smooth transition compensation intensity and reduce mutations. The illumination decay threshold (e.g., 0.5) is the center point of the function, indicating the critical distance of illumination decay, i.e., the illumination significantly weakens when exceeding the illumination decay threshold. The normalized distance is used to non-linearly adjust the illumination compensation requirement, and the greater the normalized distance, the greater the value, i.e., the farther the current position of the inspection robot is from the fixed light source, the higher the required illumination compensation. 1 minus the highest reflectance, indicating that the higher the highest reflectance is close to 1, the stronger the object's ability to reflect light, and the less light compensation is needed. 0.5H represents the higher the humidity, the stronger the scattering and absorption of light by the environment (for example, fog, rain fog), the higher the light compensation needed, and 0.5 is the humidity influence weight, indicating the degree of influence of humidity on the demand for light compensation, and the humidity influence weight is set to half of the highest reflectance, reducing excessive compensation, indicating that the combination of reflectance and humidity corrects the compensation demand. Multiplying and gives the dynamic light compensation demand coefficient, and the larger the dynamic light compensation demand coefficient, the higher the light compensation demand.

[0044] In this way, the dynamic light compensation demand coefficient can be determined by normalizing the distance, the highest reflectance, and the humidity influence factor, and the influence of the spatial position, material reflection characteristics, and environmental humidity on the light compensation demand is comprehensively considered, which helps to more accurately evaluate the actual demand for light compensation. The dynamic light compensation demand coefficient can be dynamically adjusted accordingly as the position and environmental conditions change to adapt to different lighting scenarios, thereby improving the accuracy and reliability of visual inspection.

[0045] According to one embodiment of the present application, in step S7, the light intensity and color temperature are determined according to the dynamic light compensation demand coefficient, the detection area image, the highest reflectance, and the humidity influence factor.

[0046] Figure 4 An exemplary flowchart of calculating the light intensity and color temperature according to an embodiment of the present application is shown.

[0047] According to one embodiment of the present application, step S7 includes: step S71, obtaining the gray value of each pixel point according to the detection area image; step S72, determining the texture complexity of the detection area according to the gray value of the pixel point; step S73, obtaining the moving speed of the inspection robot, the exposure time, and the detection area image width; step S74, determining the motion blur risk according to the moving speed of the inspection robot, the exposure time, and the detection area image width; step S75, determining the light intensity according to the texture complexity, the motion blur risk, and the dynamic light compensation demand coefficient; step S76, obtaining the reference color temperature; and step S77, determining the color temperature according to the reference color temperature, the highest reflectance, and the humidity influence factor.

[0048] According to one embodiment of the present application, the gray value information of each pixel point in the collected detection area image is extracted one by one using image processing technology. The average gray value of the detection area image is obtained by averaging the gray values of all pixel points, and the texture complexity calculation formula is wherein, is a texture complexity, is a gray value of the i-th pixel point, is an average gray value, N is a total number of pixel points, i≤N, and i and N are positive integers. By calculating the gray variance, i.e., the texture complexity, the discrete degree of pixel value distribution, i.e., the texture richness, can be represented. The greater the texture complexity, the more complex the texture (e.g., rust, crack, etc.), and the stronger the light compensation required. According to the motion blur risk calculation formula, wherein, is a motion blur risk, v is a moving speed of the inspection robot, is an exposure time, b is a detection area image width, and min is a minimum value function. is a ratio between a displacement of the inspection robot within the exposure time and the detection area image width. When the displacement exceeds the image width, i.e. , the blur degree is unacceptable, The greater the value, the higher the light intensity required, thereby shortening the exposure time. By the texture complexity, the motion blur risk, and the dynamic light compensation demand coefficient, the light intensity is determined. The reference color temperature (e.g., 4000K) reflects the color characteristics of the light source. In combination with the reference color temperature, the highest reflectivity of the detection area, and the humidity influence factor, the light color temperature finally applied to the detection area is calculated and determined.

[0049] According to one embodiment of the present application, the light intensity is determined according to the texture complexity, the motion blur risk, and the dynamic light compensation demand coefficient, comprising: determining the light intensity I according to formula (2),

[0050] (2),

[0051] wherein, is a reference light intensity, is a dynamic light compensation demand coefficient, is a texture complexity, is a motion blur risk.

[0052] According to one embodiment of the present application, in formula (2), The light intensity is adjusted to indicate that the larger the dynamic light compensation demand coefficient is, the greater the demand for dynamic light compensation is for the scene where the current inspection robot is located, and the higher the light intensity is required. The greater the texture complexity is, the higher the light intensity is required to clearly present the details. For example, when inspecting corroded pipes, the main reliance is on texture complexity to enhance lighting and highlight texture details. The higher the risk of motion blur, the more appropriate increase in light intensity is required to reduce the impact of blur. The weight of 10 is used in the calculation, indicating that the risk of motion blur has a greater impact on light intensity. For example, when passing through a valve quickly, the light intensity is briefly increased to suppress dynamic blur. Through the baseline light intensity (for example, 300 lux) and The sum of the light intensity can be obtained, that is, the light intensity after compensation. The greater the light intensity, the greater the light intensity required for the detection area. For example, when detecting bolt arrays in a high humidity environment, high compensation requirements are required. , bolt shadows interlaced , Inspection robots move at medium speed , the light intensity is 1263.2lux.

[0053] In this way, the illumination intensity can be determined by the texture complexity, motion blur risk and dynamic illumination compensation requirement coefficient. The illumination intensity can be dynamically adjusted accordingly by combining the complexity of the texture in the image, the impact of the inspection robot's movement on the image quality, and the demand for illumination conditions changing with the environment. This allows the inspection robot to flexibly adjust illumination conditions according to actual conditions to adapt to complex and changing environments, thereby improving image quality and detection accuracy.

[0054] According to one embodiment of the present invention, determining the color temperature according to the reference color temperature, the maximum reflectivity and the humidity influencing factor includes: determining the color temperature T according to formula (3),

[0055] (3),

[0056] in, is the base color temperature, is the highest reflectivity, and H is the humidity influence factor.

[0057] According to one embodiment of the present invention, in formula (3), Indicates the degree of deviation of the highest reflectivity from 0.5 (the middle value), Indicates the adjustment effect of the highest reflectivity on the color temperature. Indicates the degree of deviation of the humidity impact factor from the median value of 0.5. Indicates the adjustment effect of humidity factor on color temperature. 、 and The color temperature can be obtained by addition. High reflectivity causes the high light area in the image to be too bright, and the shadow area to be too dark, reducing the contrast and clarity of the image. Properly increasing the color temperature can increase the cool tone component of the light, help to enhance the contrast of the image, make the object outline clearer, and facilitate the accurate detection and recognition of the inspection robot. In a high humidity environment, the number of water vapor molecules in the air increases, and more scattering phenomena occur in the light during propagation. Scattering can increase the low frequency (reddish) component in the light, causing the light to be overall warm. To offset the color deviation caused by scattering, the color temperature of the light needs to be increased to increase the high frequency (bluish) component, so that the color of the light is closer to the normal state, ensuring the accuracy of the detection result. High reflectivity or high humidity increases the color temperature (the picture is cooler / bluish), and low reflectivity or low humidity decreases the color temperature (the picture is warmer / yellowish).

[0058] In this way, the color temperature can be determined by the reference color temperature, the maximum reflectivity, and the humidity influence factor. Combining the reflection ability of the material of the detection area to the light and the influence of the environmental humidity on the propagation of the light, the demand for color temperature adjustment can be more comprehensively evaluated, which helps to improve the image quality, reduce the color deviation or distortion caused by improper color temperature, and thus improve the detection accuracy.

[0059] According to an embodiment of the present application, in step S8, the depth camera has the ability to measure the distance of the object, and can simultaneously obtain the image and the distance information between each point in the image and the camera, thereby obtaining the image of the undetected area at a preset distance (for example, 30 cm) in the forward direction of the inspection robot.

[0060] According to an embodiment of the present application, in step S9, the illumination angle is determined according to the preset distance, the undetected area image, and the detection area image.

[0061] Figure 5 An exemplary flowchart for calculating the illumination angle according to an embodiment of the present application is shown.

[0062] According to an embodiment of the present application, step S9 includes: step S91, obtaining the future shadow area according to the undetected area image; step S92, obtaining the shadow area according to the detection area image; and step S93, determining the illumination angle according to the future shadow area, the shadow area, and the preset distance.

[0063] According to an embodiment of the present application, the undetected area image and the detection area image are converted into grayscale images, the same image segmentation algorithm is used to identify the shadow area therein, and the total number of shadow pixels is multiplied by the area of a single pixel to obtain the future shadow area of the undetected area image and the shadow area of the detection area image.

[0064] According to one embodiment of the present application, the lighting angle is determined according to the future shadow area, the shadow area and the preset distance, comprising: determining the lighting angle according to formula (4) ,

[0065] (4),

[0066] wherein, is a reference projection angle, is a future shadow area, is a shadow area, is a preset distance, is a preset minimum lighting angle, is a preset maximum lighting angle, max is a maximum function, and min is a minimum function.

[0067] According to one embodiment of the present application, in formula (4), is a change rate of the shadow area in the forward direction of the inspection robot, indicating a spatial gradient of the shadow distribution, is a gradient value converted into an angle correction amount, is a sum of the reference projection angle and the angle correction amount, that is, a preliminary adjusted lighting angle. When , , it indicates that the front is a uniform shadow, and the reference projection angle is maintained, without adjustment. When , , it indicates that the front shadow is more dense, and the light source is tilted forward to cover the shadow area (for example, a pipeline corner) in advance. When , , it indicates that the front shadow is reduced, and the light source is tilted backward to reduce the waste of light resources. The lighting angle is limited in a physically feasible range, that is, a value selected between a preset minimum lighting angle (for example, 10°) and a preset maximum lighting angle (for example, 60°) is the lighting angle, indicates that the minimum value of the preset maximum lighting angle and the preliminary adjusted lighting angle is taken, so that the lighting angle is less than the preset maximum lighting angle, preventing the lighting angle from exceeding the upper limit, indicates that the result of the above selection is compared with the preset minimum lighting angle, and the larger one is taken, so that the lighting angle is not lower than the lower limit, wherein the reference projection angle (for example, 20°), for example, , , that is .

[0068] In this way, the shadow space gradient can be used for illumination angle control, the shadow change trend can be predicted through the shadow gradient, the illumination angle can be adjusted in advance, the illumination angle can be dynamically adjusted according to the actual change of the shadow, the influence of the shadow on the detection area is minimized, the inspection robot can obtain clear images in various complex shadow environments, and the visual perception ability of the inspection robot is improved.

[0069] According to one embodiment of the present application, in step S10, the calculated illumination intensity, color temperature and illumination angle data are input into the controllable light compensation device carried by the inspection robot, so as to perform dynamic light source compensation.

[0070] The method for enhancing visual perception of the inspection robot based on dynamic light source compensation according to the embodiment of the present application can improve the stability and accuracy of visual perception by acquiring the current image of the detection area and determining whether the visual perception needs to be enhanced based on image analysis, responding to the change of the light environment in real time, adjusting based on the actual environmental demand through the light compensation demand coefficient, and dynamically adjusting the lighting angle, intensity and color temperature according to the real-time position of the inspection robot and the visual perception demand, so as to realize the visual perception enhancement of the shadow area and the light reflection area, help to improve the inspection efficiency and reduce the risk of accidents. When determining the dynamic light compensation demand coefficient, the dynamic light compensation demand coefficient can be determined by the normalized distance, the maximum reflectivity and the humidity influence factor, which comprehensively considers the influence of the spatial position, the material reflection characteristics and the environmental humidity on the light compensation demand, helps to more accurately evaluate the actual demand of light compensation, and the dynamic light compensation demand coefficient can be dynamically adjusted according to the change of the position and the environmental condition to adapt to different light scenes, so as to improve the accuracy and reliability of visual detection. When determining the light intensity, the light intensity can be determined by the texture complexity, the motion blur risk and the dynamic light compensation demand coefficient, the light intensity can be dynamically adjusted by combining the complexity of the texture in the image, the influence of the movement of the inspection robot on the image quality and the demand of the light condition changing with the environment, so that the inspection robot can flexibly adjust the light condition according to the actual situation to adapt to the complex and changeable environment, improve the image quality and the accuracy of detection. When determining the color temperature, the color temperature can be determined by the reference color temperature, the maximum reflectivity and the humidity influence factor, which combines the reflection ability of the material in the detection area to light and the influence of the environmental humidity on the light propagation, can more comprehensively evaluate the demand of color temperature adjustment, helps to improve the image quality, reduce the color deviation or distortion caused by improper color temperature, and thus improve the accuracy of detection. When determining the lighting angle, the shadow space gradient can be used for lighting angle control, the shadow gradient is used to predict the trend of shadow change, the lighting angle is adjusted in advance, the lighting angle can be dynamically adjusted according to the actual change of the shadow, the influence of the shadow on the detection area is minimized, the inspection robot can obtain clear images in various complex shadow environments, and thus the visual perception ability of the inspection robot is improved.

[0071] Figure 6An example of a block diagram of a dynamic light source compensation-based inspection robot vision perception enhancement system according to an embodiment of the present application is shown, the system comprising: a detection area image module for acquiring a current detection area image through a visual sensor built in an inspection robot; a judgment module for determining whether to enhance vision perception according to the detection area image; a position information and construction model module for acquiring position information of the inspection robot and constructing a three-dimensional lighting environment model of an underground sewage treatment plant if it is determined to enhance vision perception, wherein the three-dimensional lighting environment model comprises fixed light source positions, equipment occlusion relationships, pipeline layouts, and reflective area parameters; a highest reflectivity module for acquiring the highest reflectivity of the current detection area according to the position information and the three-dimensional lighting environment model; a humidity influence factor module for acquiring a humidity influence factor of the current detection area through a humidity sensor carried by the inspection robot; a dynamic lighting compensation demand coefficient module for determining a dynamic lighting compensation demand coefficient according to the position information, the three-dimensional lighting environment model, the highest reflectivity, and the humidity influence factor; a lighting intensity and color temperature module for determining lighting intensity and color temperature according to the dynamic lighting compensation demand coefficient, the detection area image, the highest reflectivity, and the humidity influence factor; an undetected area image module for acquiring an undetected area image at a preset distance in the advancing direction of the inspection robot through a depth camera; an illumination angle module for determining an illumination angle according to the preset distance, the undetected area image, and the detection area image; and a dynamic light source compensation module for performing dynamic light source compensation using a controllable light compensation device carried by the inspection robot according to the lighting intensity, the color temperature, and the illumination angle.

[0072] The present application can be a method, apparatus, system, and / or computer program product. The computer program product can include a computer-readable storage medium having computer-readable program instructions loaded thereon, the computer-readable program instructions being used to program computers to perform various aspects of the present application.

[0073] Those skilled in the art will understand that the embodiments of the present application described above and shown in the accompanying drawings are only exemplary and do not limit the present application. The purpose of the present application has been fully and effectively achieved. The functional and structural principles of the present application have been demonstrated and described in the embodiments, and the embodiments of the present application can be modified or changed in any way without departing from the principles.

Claims

1. A method for enhancing visual perception of an inspection robot based on dynamic light source compensation, characterized in that: include: The inspection robot acquires the current inspection area image through its built-in visual sensor; determining whether to enhance visual perception based on the detection area image; If it is determined to enhance visual perception, the position information of the inspection robot is obtained, and a three-dimensional lighting environment model of the underground sewage treatment plant is constructed, wherein the three-dimensional lighting environment model includes a fixed light source position, equipment occlusion relationship, pipeline layout and reflective area parameters; based on the position information and the three-dimensional lighting environment model, the maximum reflectivity of the current detection area is obtained; through the humidity sensor carried by the inspection robot, the humidity influencing factor of the current detection area is obtained; based on the position information, the three-dimensional lighting environment model, the maximum reflectivity and the humidity influencing factor, the dynamic lighting compensation requirement coefficient is determined; based on the dynamic lighting compensation requirement coefficient, the detection area image, the maximum reflectivity and the humidity influencing factor, the light intensity and color temperature are determined; through the depth camera, an image of an undetected area at a preset distance in the forward direction of the inspection robot is obtained; based on the preset distance, the image of the undetected area and the image of the detection area, the lighting angle is determined; based on the light intensity, the color temperature and the lighting angle, the controllable fill light device of the inspection robot is used to perform dynamic light source compensation.

2. The method for enhancing visual perception of an inspection robot based on dynamic light source compensation according to claim 1, characterized in that: Determining whether to enhance visual perception based on the detection area image includes: obtaining a shadow area and a reflection area based on the detection area image; if the sum of the shadow area and the reflection area is greater than or equal to a preset area threshold, determining that visual perception is enhanced; if the sum of the shadow area and the reflection area is less than the preset area threshold, determining that visual perception does not need to be enhanced.

3. The method for enhancing visual perception of an inspection robot based on dynamic light source compensation according to claim 1, characterized in that: The dynamic lighting compensation requirement coefficient is determined based on the position information, the three-dimensional lighting environment model, the maximum reflectivity and the humidity influencing factor, including: obtaining the Euclidean distance from the inspection robot to the nearest fixed light source position based on the position information and the three-dimensional lighting environment model; obtaining the minimum distance at which the light intensity of the fixed light source is not attenuated and the maximum distance at which the light intensity attenuates to the ambient light level; determining the normalized distance based on the Euclidean distance, the minimum distance and the maximum distance; determining the dynamic lighting compensation requirement coefficient based on the normalized distance, the maximum reflectivity and the humidity influencing factor.

4. The method for enhancing visual perception of an inspection robot based on dynamic light source compensation according to claim 3 is characterized in that: Determining a dynamic illumination compensation requirement coefficient according to the normalized distance, the maximum reflectivity, and the humidity influencing factor includes: according to the formula Determine the dynamic lighting compensation requirement coefficient , where D is the normalized distance, is the light attenuation threshold, is the highest reflectivity, and H is the humidity influence factor.

5. The method for enhancing visual perception of an inspection robot based on dynamic light source compensation according to claim 1, characterized in that: Determine the light intensity and color temperature according to the dynamic light compensation requirement coefficient, the detection area image, the maximum reflectivity and the humidity influencing factor, including: obtaining the grayscale value of each pixel point according to the detection area image; determining the texture complexity of the detection area according to the grayscale value of the pixel point; obtaining the movement speed, exposure time and detection area image width of the inspection robot; determining the motion blur risk according to the movement speed, exposure time and detection area image width of the inspection robot; determining the light intensity according to the texture complexity, the motion blur risk and the dynamic light compensation requirement coefficient; obtaining the reference color temperature; and determining the color temperature according to the reference color temperature, the maximum reflectivity and the humidity influencing factor.

6. The method for enhancing visual perception of an inspection robot based on dynamic light source compensation according to claim 5, characterized in that: Determining the illumination intensity according to the texture complexity, the motion blur risk, and the dynamic illumination compensation requirement coefficient includes: determining the illumination intensity according to the formula Determine the light intensity I, where is the baseline light intensity, is the dynamic lighting compensation requirement coefficient, is the texture complexity, Risk of motion blur.

7. The method for enhancing visual perception of an inspection robot based on dynamic light source compensation according to claim 5, characterized in that: Determining the color temperature according to the reference color temperature, the maximum reflectivity, and the humidity influencing factor includes: determining the color temperature according to the formula Determine the color temperature T, where is the base color temperature, is the highest reflectivity, and H is the humidity influence factor.

8. The method for enhancing visual perception of an inspection robot based on dynamic light source compensation according to claim 1, characterized in that: Determining the lighting angle according to the preset distance, the undetected area image and the detected area image includes: obtaining the future shadow area according to the undetected area image; obtaining the shadow area according to the detected area image; and determining the lighting angle according to the future shadow area, the shadow area and the preset distance.

9. The method for enhancing visual perception of an inspection robot based on dynamic light source compensation according to claim 8, characterized in that: Determining the lighting angle according to the future shadow area, the shadow area, and the preset distance includes: determining the lighting angle according to the formula Determine the lighting angle ,in, is the base projection angle, is the future shadow area, is the shaded area, For the preset distance, To preset the minimum lighting angle, To preset the maximum lighting angle, max is the maximum value function, and min is the minimum value function.

10. A visual perception enhancement system for an inspection robot based on dynamic light source compensation, used to execute the visual perception enhancement method for an inspection robot based on dynamic light source compensation as claimed in any one of claims 1 to 9, characterized in that: include: The detection area image module is used to obtain the current detection area image through the built-in visual sensor of the inspection robot; a judgment module, configured to determine whether to enhance visual perception based on the detection area image; A position information and model building module is used to obtain the position information of the inspection robot and build a three-dimensional lighting environment model of the underground sewage treatment plant if it is determined to enhance visual perception, wherein the three-dimensional lighting environment model includes the fixed light source position, equipment occlusion relationship, pipeline layout and reflective area parameters; a maximum reflectivity module is used to obtain the maximum reflectivity of the current detection area based on the position information and the three-dimensional lighting environment model; a humidity influence factor module is used to obtain the humidity influence factor of the current detection area through the humidity sensor carried by the inspection robot; a dynamic lighting compensation demand coefficient module is used to obtain the humidity influence factor of the current detection area based on the position information, the three-dimensional lighting environment model, the maximum reflectivity and The humidity influencing factor determines the dynamic lighting compensation requirement coefficient; the light intensity and color temperature module is used to determine the light intensity and color temperature based on the dynamic lighting compensation requirement coefficient, the detection area image, the maximum reflectivity and the humidity influencing factor; the undetected area image module is used to obtain the undetected area image at a preset distance in the forward direction of the inspection robot through a depth camera; the lighting angle module is used to determine the lighting angle based on the preset distance, the undetected area image and the detection area image; the dynamic light source compensation module is used to perform dynamic light source compensation using the controllable fill light device of the inspection robot according to the light intensity, the color temperature and the lighting angle.

Images