Method and system for enhancing visual perception of inspection robot based on dynamic light source compensation

Abstract

The invention provides an inspection robot visual perception enhancement method and system based on dynamic light source compensation, and relates to the technical field of visual perception. The method comprises the following steps: acquiring a current detection area image; determining whether to enhance visual perception; if it is determined that visual perception is enhanced, position information of the inspection robot is obtained, and a three-dimensional illumination environment model of the underground sewage treatment plant is constructed; obtaining the highest reflectivity; obtaining a humidity influence factor; determining a dynamic illumination compensation demand coefficient; determining illumination intensity and color temperature; acquiring an undetected area image at a preset distance in the advancing direction of the inspection robot; determining an illumination angle; and according to the illumination intensity, the color temperature and the illumination angle, a controllable light supplementing device of the inspection robot is used for dynamic light source compensation. According to the invention, the illumination angle, intensity and color temperature can be dynamically adjusted according to the real-time position and visual perception requirements of the inspection robot, and visual perception enhancement of a shadow area and a reflective area is realized.

Description

Technical Field

[0001] The present invention relates to the field of visual perception technology, and in particular to a method and system for enhancing the visual perception of an inspection robot based on dynamic light source compensation. Background Art

[0002] Although current related technologies can adjust the compensation light source and expand the perception range, they do not take into account the interference of shadow areas, light reflections or moisture diffusion on the inspection robot's visual system. In other words, it is impossible to dynamically adjust the lighting angle, intensity and color temperature according to the real-time position and visual perception requirements of the inspection robot.

[0003] The information disclosed in the background technology section of this application is only intended to deepen the understanding of the general background technology of this application, and should not be regarded as an admission or any form of suggestion that the information constitutes the prior art already known to those skilled in the art. Summary of the Invention

[0004] The present invention provides a method and system for enhancing the visual perception of an inspection robot based on dynamic light source compensation, which can solve the technical problem that related technologies cannot dynamically adjust the lighting angle, intensity and color temperature according to the real-time position and visual perception requirements of the inspection robot.

[0005] According to a first aspect of the present invention, a method for enhancing visual perception of an inspection robot based on dynamic light source compensation is provided, comprising: obtaining an image of a current detection area through a built-in visual sensor of the inspection robot; determining whether to enhance visual perception based on the image of the detection area; if it is determined to enhance visual perception, obtaining the position information of the inspection robot and constructing a three-dimensional lighting environment model of an underground sewage treatment plant, wherein the three-dimensional lighting environment model includes fixed light source positions, equipment occlusion relationships, pipeline layouts, and reflective area parameters; obtaining the maximum reflectivity of the current detection area based on the position information and the three-dimensional lighting environment model; obtaining the humidity sensor carried by the inspection robot. Take the humidity impact factor of the current detection area; determine the dynamic lighting compensation requirement coefficient based on the position information, the three-dimensional lighting environment model, the maximum reflectivity and the humidity impact factor; determine the light intensity and color temperature based on the dynamic lighting compensation requirement coefficient, the detection area image, the maximum reflectivity and the humidity impact factor; obtain the image of the undetected area at a preset distance in the forward direction of the inspection robot through the depth camera; determine the lighting angle based on the preset distance, the image of the undetected area and the image of the detection area; use the controllable fill light device of the inspection robot to perform dynamic light source compensation based on the light intensity, the color temperature and the lighting angle.

[0006] Furthermore, based on the detection area image, determining whether to enhance visual perception includes: obtaining the shadow area and the reflection area based on the detection area image; if the sum of the shadow area and the reflection area is greater than or equal to a preset area threshold, determining that visual perception is enhanced; if the sum of the shadow area and the reflection area is less than the preset area threshold, determining that visual perception does not need to be enhanced.

[0007] Furthermore, a dynamic lighting compensation requirement coefficient is determined based on the position information, the three-dimensional lighting environment model, the maximum reflectivity and the humidity influencing factor, including: obtaining the Euclidean distance from the inspection robot to the nearest fixed light source position based on the position information and the three-dimensional lighting environment model; obtaining the minimum distance at which the light intensity of the fixed light source is not attenuated and the maximum distance at which the light intensity decays to the ambient light level; determining a normalized distance based on the Euclidean distance, the minimum distance and the maximum distance; and determining a dynamic lighting compensation requirement coefficient based on the normalized distance, the maximum reflectivity and the humidity influencing factor.

[0008] Further, according to the normalized distance, the maximum reflectivity and the humidity influence factor, a dynamic illumination compensation requirement coefficient is determined, including: according to the formula Determine the dynamic lighting compensation requirement coefficient , where D is the normalized distance, is the light attenuation threshold, is the highest reflectivity, and H is the humidity influence factor.

[0009] Furthermore, the light intensity and color temperature are determined based on the dynamic light compensation requirement coefficient, the detection area image, the maximum reflectivity and the humidity influencing factor, including: obtaining the grayscale value of each pixel point based on the detection area image; determining the texture complexity of the detection area based on the grayscale value of the pixel point; obtaining the movement speed, exposure time and detection area image width of the inspection robot; determining the motion blur risk based on the movement speed, exposure time and detection area image width of the inspection robot; determining the light intensity based on the texture complexity, the motion blur risk and the dynamic light compensation requirement coefficient; obtaining the reference color temperature; and determining the color temperature based on the reference color temperature, the maximum reflectivity and the humidity influencing factor.

[0010] Further, determining the illumination intensity according to the texture complexity, the motion blur risk and the dynamic illumination compensation requirement coefficient includes: according to the formula Determine the light intensity I, where is the baseline light intensity, is the dynamic lighting compensation requirement coefficient, is the texture complexity, Risk of motion blur.

[0011] Further, determining the color temperature according to the reference color temperature, the maximum reflectivity and the humidity influencing factor includes: according to the formula Determine the color temperature T, where is the base color temperature, is the highest reflectivity, and H is the humidity influence factor.

[0012] Furthermore, the lighting angle is determined based on the preset distance, the undetected area image and the detected area image, including: obtaining the future shadow area based on the undetected area image; obtaining the shadow area based on the detected area image; and determining the lighting angle based on the future shadow area, the shadow area and the preset distance.

[0013] Further, determining the lighting angle according to the future shadow area, the shadow area and the preset distance includes: according to the formula Determine the lighting angle ,in, is the base projection angle, is the future shadow area, is the shaded area, For the preset distance, To preset the minimum lighting angle, To preset the maximum lighting angle, max is the maximum value function, and min is the minimum value function.

[0014] According to the second aspect of the present invention, a visual perception enhancement system for an inspection robot based on dynamic light source compensation is provided, comprising: a detection area image module, for acquiring the current detection area image through the built-in visual sensor of the inspection robot; a judgment module, for determining whether to enhance visual perception based on the detection area image; a position information and model construction module, for acquiring the position information of the inspection robot if it is determined to enhance visual perception, and constructing a three-dimensional lighting environment model of the underground sewage treatment plant, wherein the three-dimensional lighting environment model includes fixed light source position, equipment occlusion relationship, pipeline layout and reflective area parameters; a maximum reflectivity module, for acquiring the maximum reflectivity of the current detection area based on the position information and the three-dimensional lighting environment model; a humidity influencing factor module, for acquiring the current detection area through the humidity sensor carried by the inspection robot. The humidity influencing factor of the inspection area is measured; a dynamic lighting compensation requirement coefficient module is used to determine the dynamic lighting compensation requirement coefficient according to the position information, the three-dimensional lighting environment model, the maximum reflectivity and the humidity influencing factor; the lighting intensity and color temperature module is used to determine the lighting intensity and color temperature according to the dynamic lighting compensation requirement coefficient, the detection area image, the maximum reflectivity and the humidity influencing factor; the undetected area image module is used to obtain the undetected area image at a preset distance in the forward direction of the inspection robot through a depth camera; the lighting angle module is used to determine the lighting angle according to the preset distance, the undetected area image and the detection area image; a dynamic light source compensation module is used to perform dynamic light source compensation using the controllable fill light device of the inspection robot according to the light intensity, the color temperature and the lighting angle.

[0015] Technical effect: According to the present invention, by acquiring the current detection area image and determining whether visual perception needs to be enhanced based on image analysis, it can respond to changes in the lighting environment in real time. Through the lighting compensation requirement coefficient, it can be adjusted based on actual environmental requirements to improve the stability and accuracy of visual perception. According to the real-time position and visual perception requirements of the inspection robot, the lighting angle, intensity and color temperature can be dynamically adjusted to achieve enhanced visual perception of shadow areas and reflective areas, which helps to improve inspection efficiency and reduce accident risks. When determining the dynamic lighting compensation requirement coefficient, the dynamic lighting compensation requirement coefficient can be determined by normalizing the distance, maximum reflectivity and humidity influencing factors, which fully considers the influence of spatial position, material reflection characteristics and environmental humidity on the lighting compensation requirement, which helps to more accurately evaluate the actual demand for lighting compensation. The dynamic lighting compensation requirement coefficient can be dynamically adjusted accordingly with changes in position and environmental conditions to adapt to different lighting scenarios, thereby improving the accuracy and reliability of visual detection. When determining illumination intensity, texture complexity, motion blur risk, and the dynamic illumination compensation requirement factor can be used to determine illumination intensity. Light intensity can be dynamically adjusted based on the complexity of textures in the image, the impact of the inspection robot's movement on image quality, and the need for changing lighting conditions. This allows the inspection robot to flexibly adjust lighting conditions based on actual conditions, adapting to complex and changing environments and improving image quality and detection accuracy. When determining color temperature, a baseline color temperature, maximum reflectivity, and humidity influencing factors can be used to determine color temperature. By combining the light reflectivity of the inspection area's material and the effect of ambient humidity on light propagation, a more comprehensive assessment of the need for color temperature adjustment can be made, helping to improve image quality and reduce color deviation or distortion caused by inappropriate color temperature, thereby enhancing detection accuracy. When determining the lighting angle, the spatial gradient of shadows can be used to control the lighting angle. Shadow gradients can be used to predict shadow change trends, allowing for pre-adjusted lighting angles. This allows for dynamic adjustment based on actual shadow changes, minimizing the impact of shadows on the inspection area and enabling the inspection robot to obtain clear images in a variety of complex shadow environments, thereby improving the robot's visual perception capabilities.

[0016] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and not limiting of the present invention. Other features and aspects of the present invention will become more apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can derive other embodiments based on these drawings without inventive efforts. Figure 1 A schematic diagram exemplarily illustrates a flow chart of a method for enhancing visual perception of an inspection robot based on dynamic light source compensation according to an embodiment of the present invention; Figure 2 A flowchart for determining whether to enhance visual perception according to an embodiment of the present invention is exemplarily shown; Figure 3 The flowchart of calculating the dynamic illumination compensation requirement coefficient according to an embodiment of the present invention is exemplarily shown; Figure 4 The following is a flowchart showing calculation of light intensity and color temperature according to an embodiment of the present invention; Figure 5 The flowchart of calculating the lighting angle according to an embodiment of the present invention is exemplarily shown; Figure 6 A block diagram of a visual perception enhancement system for an inspection robot based on dynamic light source compensation according to an embodiment of the present invention is exemplarily shown. DETAILED DESCRIPTION

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0019] The following specific embodiments are used to describe the technical solution of the present invention in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0020] Figure 1A flow chart of a method for enhancing visual perception of an inspection robot based on dynamic light source compensation according to an embodiment of the present invention is exemplarily shown, the method comprising: step S1, obtaining an image of the current detection area through a built-in visual sensor of the inspection robot; step S2, determining whether to enhance visual perception based on the image of the detection area; step S3, if it is determined to enhance visual perception, obtaining the position information of the inspection robot, and constructing a three-dimensional lighting environment model of the underground sewage treatment plant, wherein the three-dimensional lighting environment model includes a fixed light source position, equipment occlusion relationship, pipeline layout, and reflective area parameters; step S4, obtaining the highest reflectivity of the current detection area based on the position information and the three-dimensional lighting environment model; step S5, obtaining the maximum reflectivity of the current detection area through the humidity sensor carried by the inspection robot sensor, obtains the humidity impact factor of the current detection area; step S6, determines the dynamic lighting compensation requirement coefficient according to the position information, the three-dimensional lighting environment model, the maximum reflectivity and the humidity impact factor; step S7, determines the light intensity and color temperature according to the dynamic lighting compensation requirement coefficient, the detection area image, the maximum reflectivity and the humidity impact factor; step S8, obtains the image of the undetected area at a preset distance in the forward direction of the inspection robot through the depth camera; step S9, determines the lighting angle according to the preset distance, the undetected area image and the detection area image; step S10, uses the controllable fill light device of the inspection robot to perform dynamic light source compensation according to the light intensity, the color temperature and the lighting angle.

[0021] According to the visual perception enhancement method of the inspection robot based on dynamic light source compensation in an embodiment of the present invention, by acquiring the current detection area image and determining whether visual perception needs to be enhanced based on image analysis, it can respond to changes in the lighting environment in real time. Through the lighting compensation demand coefficient, it can be adjusted based on actual environmental requirements to improve the stability and accuracy of visual perception. According to the real-time position and visual perception requirements of the inspection robot, the lighting angle, intensity and color temperature can be dynamically adjusted to achieve enhanced visual perception of shadow areas and reflective areas, which helps to improve inspection efficiency and reduce accident risks.

[0022] According to one embodiment of the present invention, in step S1, the inspection robot may move, rotate, or adjust its posture so that the visual sensor faces the inspection area. The visual sensor may collect image data of the current inspection area, including visual information of the inspection area, thereby obtaining an image of the inspection area.

[0023] According to one embodiment of the present invention, in step S2, image analysis is performed on the detection area image to identify the shadow area and the reflective area of ​​the current detection area, and then determine whether to enhance visual perception.

[0024] Figure 2A flowchart for determining whether to enhance visual perception according to an embodiment of the present invention is exemplarily shown; According to one embodiment of the present invention, step S2 includes: step S21, obtaining the shadow area and the reflection area based on the detection area image; step S22, if the sum of the shadow area and the reflection area is greater than or equal to a preset area threshold, determining to enhance visual perception; step S23, if the sum of the shadow area and the reflection area is less than the preset area threshold, determining that visual perception does not need to be enhanced.

[0025] According to one embodiment of the present invention, the detection area image is converted into a grayscale image, and the shadow area is separated by threshold segmentation (for example, Otsu algorithm). The grayscale threshold obtained by Otsu algorithm is 50 (the grayscale range is 0-255), that is, pixels with grayscale lower than the grayscale threshold are shadow pixels. The shadow area is obtained by multiplying the total number of shadow pixels by the single pixel area, where the single pixel area is the ratio of the detection area size to the image resolution. For example, if the detection area size is 1m×1m and the image resolution is 1000×1000 pixels, the single pixel area is 1×10 -6 m 2 In HSV space, the highlight area has high brightness, and the brightness threshold is 200, that is, the pixels with brightness higher than the brightness threshold are highlight pixels, and the reflection area can be obtained by multiplying the total number of highlight pixels by the area of ​​a single pixel. The sum of the shadow area and the reflection area reflects the degree of visual interference caused by shadows and reflections in the detection area image. If the sum of the shadow area and the reflection area is greater than or equal to the preset area threshold (for example, 0.3m 2 ), it is determined that the image quality of the detection area is greatly affected and the visual perception is limited, and it is determined that the visual perception should be enhanced to improve the image clarity and recognition accuracy. If the sum of the shadow area and the reflective area is less than the preset area threshold, it is determined that the image quality of the detection area is acceptable and the visual perception is not significantly disturbed, and it is determined that there is no need to enhance the visual perception and the existing visual perception state is maintained.

[0026] According to one embodiment of the present invention, in step S3, the position information of the inspection robot in the underground sewage treatment plant is obtained according to the positioning system, and a three-dimensional lighting environment model is constructed, that is, all fixed light source positions, equipment occlusion relationships, pipeline layouts and reflective area parameters in the underground sewage treatment plant are entered into the three-dimensional lighting environment model. There are reflective areas with different surface materials in the underground sewage treatment plant, such as metal pipe surfaces, plastics, etc., and corresponding reflective area parameters are set. For example, the reflectivity of stainless steel pipes is 0.7, the reflectivity of concrete walls is 0.3, and the reflectivity of plastic equipment is 0.5. This can comprehensively and accurately reflect the light distribution in the underground sewage treatment plant and provide data support for subsequent light source compensation and visual perception enhancement.

[0027] According to one embodiment of the present invention, in step S4, the inspection robot's position information is matched with a three-dimensional lighting environment model. Based on the position information, the specific range of the current detection area is defined in the three-dimensional lighting environment model. This range is determined by the inspection robot's visual sensor field of view or the preset inspection task. Each inspection robot position information corresponds to a specific detection area, and each detection area is individually covered by a fixed light source. The various surface materials (e.g., metal, concrete, etc.) within the detection area are identified, and the highest reflectivity within the current detection area is determined by querying the reflective area parameters in the three-dimensional lighting environment model.

[0028] According to one embodiment of the present invention, in step S5, the inspection robot starts the humidity sensor device it carries, which can sense and measure the humidity in the environment, obtain humidity data, and calculate the humidity influence factor according to the humidity influence factor formula. , where H is the humidity impact factor, W is the humidity data, is the saturation humidity threshold, and min is the minimum function. By comparing the saturation humidity threshold (for example, 90%) with 1 and taking the minimum value, the humidity data can be normalized to obtain the humidity impact factor. The larger the humidity impact factor, the more humid the environment at the inspection robot's current location, for example, if there is pervasive humidity.

[0029] According to an embodiment of the present invention, in step S6, a dynamic lighting compensation requirement coefficient is determined based on the position information, the three-dimensional lighting environment model, the maximum reflectivity, and the humidity impact factor.

[0030] Figure 3 A flow chart for calculating a dynamic illumination compensation requirement coefficient according to an embodiment of the present invention is exemplarily shown.

[0031] According to one embodiment of the present invention, step S6 includes: step S61, obtaining the Euclidean distance from the inspection robot to the nearest fixed light source position based on the position information and the three-dimensional lighting environment model; step S62, obtaining the minimum distance at which the light intensity of the fixed light source is not attenuated and the maximum distance at which the light intensity decays to the ambient light level; step S63, determining the normalized distance based on the Euclidean distance, the minimum distance and the maximum distance; step S64, determining the dynamic lighting compensation requirement coefficient based on the normalized distance, the maximum reflectivity and the humidity influencing factor.

[0032] According to one embodiment of the present invention, the relative position relationship between the inspection robot and the fixed light source in space is determined by calculating the straight-line distance (i.e., Euclidean distance) between the inspection robot and the nearest fixed light source. The minimum distance at which the light intensity does not decay (e.g., 0.5m) is the minimum distance at which the light intensity emitted by the fixed light source remains constant and is not affected by distance. The maximum distance at which the light intensity decays to the level of ambient light (e.g., 10m) is the maximum distance at which the light intensity of the fixed light source decays to a level equivalent to the ambient light and no longer has a significant lighting impact on the detection area. According to the normalized distance calculation formula, , where D is the normalized distance, d is the Euclidean distance from the inspection robot to the nearest fixed light source position, is the maximum distance, is the minimum distance. By combining the maximum and minimum distances and taking the maximum function, we can obtain the normalized distance. The larger the normalized distance, the farther the inspection robot's current location is from the fixed light source. This means that the inspection area is under-illuminated and requires more light compensation. The larger the peak reflectivity, the stronger the reflection in the inspection area, requiring less light compensation. The larger the humidity impact factor, the more humid the environment at the inspection robot's current location, the more severe the light scattering, and the more light compensation required. The dynamic light compensation requirement coefficient is determined by the relationship between normalized distance, peak reflectivity, humidity impact factor, and light compensation requirements.

[0033] According to one embodiment of the present invention, determining the dynamic illumination compensation requirement coefficient according to the normalized distance, the maximum reflectivity and the humidity influence factor includes: determining the dynamic illumination compensation requirement coefficient according to formula (1): , (1), Where D is the normalized distance, is the light attenuation threshold, is the highest reflectivity, and H is the humidity influence factor.

[0034] According to one embodiment of the present invention, in formula (1), is an S-shaped monotonically increasing function, which is used to convert the input Mapped to (0, 1), a smooth transition compensation intensity is achieved to reduce mutations. The light attenuation threshold (for example, 0.5) is the center point of the function, indicating the critical distance of light attenuation. That is, when the light attenuation threshold is exceeded, the light is significantly weakened. The normalized distance is used to nonlinearly adjust the light compensation demand. When the normalized distance is larger, The larger the value of , that is, the farther the current position of the inspection robot is from the fixed light source, the higher the required illumination compensation. It is 1 minus the maximum reflectivity, indicating that the closer the maximum reflectivity is to 1, the stronger the object's ability to reflect light, and the less light compensation is required. 0.5H means that the higher the humidity, the stronger the environment's scattering and absorption of light (for example, fog, rain and mist), and the higher the required light compensation. 0.5 is the humidity impact weight, indicating the degree of influence of humidity on the light compensation requirement. The humidity impact weight is set to half of the maximum reflectivity to reduce over-compensation. Indicates the need for correction compensation based on reflectivity and humidity. and The dynamic lighting compensation demand coefficient can be obtained by multiplication. The larger the dynamic lighting compensation demand coefficient is, the higher the lighting compensation demand is.

[0035] In this way, the dynamic lighting compensation requirement coefficient can be determined by normalizing the distance, maximum reflectivity and humidity influencing factors, which comprehensively considers the impact of spatial position, material reflectance characteristics and ambient humidity on the lighting compensation demand, and helps to more accurately evaluate the actual demand for lighting compensation. The dynamic lighting compensation requirement coefficient can be dynamically adjusted accordingly with changes in position and environmental conditions to adapt to different lighting scenarios, thereby improving the accuracy and reliability of visual inspection.

[0036] According to an embodiment of the present invention, in step S7, the illumination intensity and color temperature are determined according to the dynamic illumination compensation requirement coefficient, the detection area image, the maximum reflectivity, and the humidity influence factor.

[0037] Figure 4 The flowchart for calculating the light intensity and color temperature according to an embodiment of the present invention is exemplarily shown.

[0038] According to one embodiment of the present invention, step S7 includes: step S71, obtaining the grayscale value of each pixel point based on the detection area image; step S72, determining the texture complexity of the detection area based on the grayscale value of the pixel point; step S73, obtaining the moving speed, exposure time and detection area image width of the inspection robot; step S74, determining the motion blur risk based on the moving speed of the inspection robot, the exposure time and the detection area image width; step S75, determining the light intensity based on the texture complexity, the motion blur risk and the dynamic lighting compensation requirement coefficient; step S76, obtaining the reference color temperature; step S77, determining the color temperature based on the reference color temperature, the maximum reflectivity and the humidity influencing factor.

[0039] According to one embodiment of the present invention, the grayscale value information of each pixel in the collected detection area image is extracted one by one by applying image processing technology. The grayscale values ​​of all pixels are averaged to obtain the average grayscale value of the detection area image. According to the texture complexity calculation formula, ,in, is the texture complexity, is the gray value of the i-th pixel, is the average grayscale value, N is the total number of pixels, i≤N, and both i and N are positive integers. By calculating the grayscale variance, that is, the texture complexity, the discrete degree of pixel value distribution, that is, the richness of the texture, can be expressed. The greater the texture complexity, the more complex the texture (for example, rust, cracks, etc.), and the stronger the lighting compensation required. According to the motion blur risk calculation formula, ,in, is the motion blur risk, v is the moving speed of the inspection robot, is the exposure time, b is the image width of the detection area, and min is the minimum value function. is the ratio between the displacement of the inspection robot during the exposure time and the image width of the detection area. When the displacement exceeds the image width, , the blur is unacceptable, Larger values ​​require higher light intensity, thus shortening exposure time. Light intensity is determined by considering texture complexity, motion blur risk, and the need for dynamic lighting compensation. The base color temperature (e.g., 4000K) reflects the color characteristics of the light source. The final light color temperature applied to the inspection area is calculated and determined by combining the base color temperature, the maximum reflectivity of the inspection area, and the humidity impact factor.

[0040] According to one embodiment of the present invention, determining the illumination intensity according to the texture complexity, the motion blur risk and the dynamic illumination compensation requirement coefficient includes: determining the illumination intensity I according to formula (2), (2), in, is the baseline light intensity, is the dynamic lighting compensation requirement coefficient, is the texture complexity, Risk of motion blur.

[0041] According to one embodiment of the present invention, in formula (2), The light intensity is adjusted to indicate that the larger the dynamic light compensation demand coefficient is, the greater the demand for dynamic light compensation is for the scene where the current inspection robot is located, and the higher the light intensity is required. The greater the texture complexity is, the higher the light intensity is required to clearly present the details. For example, when inspecting corroded pipes, the main reliance is on texture complexity to enhance lighting and highlight texture details. The higher the risk of motion blur, the more appropriate increase in light intensity is required to reduce the impact of blur. The weight of 10 is used in the calculation, indicating that the risk of motion blur has a greater impact on light intensity. For example, when passing through a valve quickly, the light intensity is briefly increased to suppress dynamic blur. Through the baseline light intensity (for example, 300 lux) and The sum of the light intensity can be obtained, that is, the light intensity after compensation. The greater the light intensity, the greater the light intensity required for the detection area. For example, when detecting bolt arrays in a high humidity environment, high compensation requirements are required. , bolt shadows interlaced , Inspection robots move at medium speed , the light intensity is 1263.2lux.

[0042] In this way, the illumination intensity can be determined by the texture complexity, motion blur risk and dynamic illumination compensation requirement coefficient. The illumination intensity can be dynamically adjusted accordingly by combining the complexity of the texture in the image, the impact of the inspection robot's movement on the image quality, and the demand for illumination conditions changing with the environment. This allows the inspection robot to flexibly adjust illumination conditions according to actual conditions to adapt to complex and changing environments, thereby improving image quality and detection accuracy.

[0043] According to one embodiment of the present invention, determining the color temperature according to the reference color temperature, the maximum reflectivity and the humidity influencing factor includes: determining the color temperature T according to formula (3), (3), in, is the base color temperature, is the highest reflectivity, and H is the humidity influence factor.

[0044] According to one embodiment of the present invention, in formula (3), Indicates the degree of deviation of the highest reflectivity from 0.5 (the middle value), Indicates the adjustment effect of the highest reflectivity on the color temperature. Indicates the degree of deviation of the humidity impact factor from the median value of 0.5. Indicates the adjustment effect of humidity factor on color temperature. 、 and Adding these values ​​together gives the color temperature. High reflectivity causes highlights in an image to appear overly bright, while shadows appear overly dark, reducing image contrast and clarity. Appropriately increasing the color temperature can enhance the cool tones of the light, helping to improve image contrast and sharpen object outlines, facilitating accurate detection and identification by inspection robots. In high-humidity environments, the presence of more water vapor in the air increases, leading to more scattered light. This scattering increases the low-frequency (reddish) component of the light, resulting in an overall warmer light. To offset this color cast, the color temperature of the light needs to be increased, boosting the high-frequency (bluish) component to bring the light color closer to its normal state and ensure accurate detection results. High reflectivity or high humidity increases the color temperature (resulting in a cooler / bluish image), while low reflectivity or low humidity decreases the color temperature (resulting in a warmer / yellowish image).

[0045] In this way, the color temperature can be determined through the baseline color temperature, maximum reflectivity and humidity influencing factor. Combined with the light reflectivity of the material in the detection area and the impact of ambient humidity on light propagation, the need for color temperature adjustment can be more comprehensively evaluated, which helps to improve image quality and reduce color deviation or distortion caused by improper color temperature, thereby improving detection accuracy.

[0046] According to one embodiment of the present invention, in step S8, the depth camera has the ability to measure the distance of an object, and can simultaneously obtain an image and the distance information between each point in the image and the camera, thereby obtaining an image of an undetected area at a preset distance (for example, 30 cm) in the forward direction of the inspection robot.

[0047] According to an embodiment of the present invention, in step S9, the lighting angle is determined according to the preset distance, the undetected area image, and the detected area image.

[0048] Figure 5 The flowchart of calculating the lighting angle according to an embodiment of the present invention is exemplarily shown.

[0049] According to one embodiment of the present invention, step S9 includes: step S91, obtaining the future shadow area based on the undetected area image; step S92, obtaining the shadow area based on the detected area image; step S93, determining the lighting angle based on the future shadow area, the shadow area and the preset distance.

[0050] According to one embodiment of the present invention, the undetected area image and the detected area image are converted into grayscale images, and the same image segmentation algorithm is used to identify the shadow areas therein. The total number of shadow pixels is multiplied by the single pixel area to obtain the future shadow area of ​​the undetected area image and the shadow area of ​​the detected area image.

[0051] According to one embodiment of the present invention, determining the lighting angle according to the future shadow area, the shadow area and the preset distance includes: determining the lighting angle according to formula (4): , (4), in, is the base projection angle, is the future shadow area, is the shaded area, For the preset distance, To preset the minimum lighting angle, To preset the maximum lighting angle, max is the maximum value function, and min is the minimum value function.

[0052] According to one embodiment of the present invention, in formula (4), is the rate of change of the shadow area in the forward direction of the inspection robot, which represents the spatial gradient of the shadow distribution. To convert the gradient value into an angle correction, The sum of the base projection angle and the angle correction is the initial adjusted lighting angle. , , indicating that the shadow in front is uniform, and the base projection angle is maintained without adjustment. , , which means that the shadows in front are denser and the light source is tilted forward to cover the shadow area in advance (for example, the corner of the pipe). , , which means that the shadow in front is reduced and the light source is recycled backwards, reducing the waste of lighting resources. Limit the lighting angle to a physically feasible range, that is, select a value between a preset minimum lighting angle (e.g., 10°) and a preset maximum lighting angle (e.g., 60°) as the lighting angle. It means taking the minimum value between the preset maximum lighting angle and the initially adjusted lighting angle, so that the lighting angle is less than the preset maximum lighting angle to prevent the lighting angle from exceeding the upper limit. It means comparing the above selected result with the preset minimum lighting angle and taking the larger value so that the lighting angle does not fall below the lower limit, where the reference projection angle (for example, 20°), for example, 、 、 ,Right now .

[0053] In this way, the shadow space gradient can be used to control the lighting angle. The shadow change trend can be predicted by the shadow gradient, and the lighting angle can be adjusted in advance. The lighting angle can be dynamically adjusted according to the actual changes in the shadow, minimizing the impact of the shadow on the detection area, allowing the inspection robot to obtain clear images in various complex shadow environments, thereby improving the visual perception ability of the inspection robot.

[0054] According to one embodiment of the present invention, in step S10, the calculated light intensity, color temperature and lighting angle data are input into a controllable fill light device of the inspection robot, thereby performing dynamic light source compensation.

[0055] According to the visual perception enhancement method of the inspection robot based on dynamic light source compensation according to the embodiment of the present invention, by acquiring the current detection area image and determining whether visual perception needs to be enhanced based on image analysis, it can respond to changes in the lighting environment in real time. Through the lighting compensation requirement coefficient, it can be adjusted based on the actual environmental requirements to improve the stability and accuracy of visual perception. According to the real-time position and visual perception requirements of the inspection robot, the lighting angle, intensity and color temperature can be dynamically adjusted to achieve visual perception enhancement in shadow areas and reflective areas, which helps to improve inspection efficiency and reduce accident risks. When determining the dynamic lighting compensation requirement coefficient, the dynamic lighting compensation requirement coefficient can be determined by normalizing the distance, maximum reflectivity and humidity influencing factors. The dynamic lighting compensation requirement coefficient is fully considered. The influence of spatial position, material reflection characteristics and ambient humidity on the lighting compensation requirement is considered, which helps to more accurately evaluate the actual need for lighting compensation. The dynamic lighting compensation requirement coefficient can be dynamically adjusted accordingly with changes in position and environmental conditions to adapt to different lighting scenarios, thereby improving the accuracy and reliability of visual detection. When determining illumination intensity, texture complexity, motion blur risk, and the dynamic illumination compensation requirement factor can be used to determine illumination intensity. Light intensity can be dynamically adjusted based on the complexity of textures in the image, the impact of the inspection robot's movement on image quality, and the need for changing lighting conditions. This allows the inspection robot to flexibly adjust lighting conditions based on actual conditions, adapting to complex and changing environments and improving image quality and detection accuracy. When determining color temperature, a baseline color temperature, maximum reflectivity, and humidity influencing factors can be used to determine color temperature. By combining the light reflectivity of the inspection area's material and the effect of ambient humidity on light propagation, a more comprehensive assessment of the need for color temperature adjustment can be made, helping to improve image quality and reduce color deviation or distortion caused by inappropriate color temperature, thereby enhancing detection accuracy. When determining the lighting angle, the spatial gradient of shadows can be used to control the lighting angle. Shadow gradients can be used to predict shadow change trends, allowing for pre-adjusted lighting angles. This allows for dynamic adjustment based on actual shadow changes, minimizing the impact of shadows on the inspection area and enabling the inspection robot to obtain clear images in a variety of complex shadow environments, thereby improving the robot's visual perception capabilities.

[0056] Figure 6A block diagram of a visual perception enhancement system for an inspection robot based on dynamic light source compensation according to an embodiment of the present invention is exemplarily shown, wherein the system comprises: a detection area image module, for acquiring a current detection area image through a built-in visual sensor of the inspection robot; a judgment module, for determining whether to enhance visual perception based on the detection area image; a position information and model construction module, for acquiring the position information of the inspection robot and constructing a three-dimensional lighting environment model of the underground sewage treatment plant if it is determined to enhance visual perception, wherein the three-dimensional lighting environment model includes fixed light source positions, equipment occlusion relationships, pipeline layouts and reflective area parameters; a maximum reflectivity module, for acquiring the maximum reflectivity of the current detection area based on the position information and the three-dimensional lighting environment model; a humidity influencing factor module, for acquiring the humidity influencing factor of the inspection robot through the humidity sensor carried by the inspection robot. Get the humidity impact factor of the current detection area; a dynamic lighting compensation requirement coefficient module is used to determine the dynamic lighting compensation requirement coefficient based on the position information, the three-dimensional lighting environment model, the maximum reflectivity and the humidity impact factor; a light intensity and color temperature module is used to determine the light intensity and color temperature based on the dynamic lighting compensation requirement coefficient, the detection area image, the maximum reflectivity and the humidity impact factor; an undetected area image module is used to obtain the undetected area image at a preset distance in the forward direction of the inspection robot through a depth camera; a lighting angle module is used to determine the lighting angle based on the preset distance, the undetected area image and the detection area image; a dynamic light source compensation module is used to perform dynamic light source compensation using the controllable fill light device of the inspection robot based on the light intensity, the color temperature and the lighting angle.

[0057] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0058] Those skilled in the art will appreciate that the embodiments of the present invention described above and shown in the accompanying drawings are intended to be illustrative only and are not intended to limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functional and structural principles of the present invention have been demonstrated and illustrated in the embodiments. Any variations or modifications may be made to the embodiments of the present invention without departing from the principles described.

Claims

1. A method for enhancing visual perception of an inspection robot based on dynamic light source compensation, characterized in that: include: The inspection robot acquires the current inspection area image through its built-in visual sensor; determining whether to enhance visual perception based on the detection area image; If it is determined to enhance visual perception, the position information of the inspection robot is obtained, and a three-dimensional lighting environment model of the underground sewage treatment plant is constructed, wherein the three-dimensional lighting environment model includes a fixed light source position, equipment occlusion relationship, pipeline layout and reflective area parameters; based on the position information and the three-dimensional lighting environment model, the maximum reflectivity of the current detection area is obtained; through the humidity sensor carried by the inspection robot, the humidity influencing factor of the current detection area is obtained; based on the position information, the three-dimensional lighting environment model, the maximum reflectivity and the humidity influencing factor, the dynamic lighting compensation requirement coefficient is determined; based on the dynamic lighting compensation requirement coefficient, the detection area image, the maximum reflectivity and the humidity influencing factor, the light intensity and color temperature are determined; through the depth camera, an image of an undetected area at a preset distance in the forward direction of the inspection robot is obtained; based on the preset distance, the image of the undetected area and the image of the detection area, the lighting angle is determined; based on the light intensity, the color temperature and the lighting angle, the controllable fill light device of the inspection robot is used to perform dynamic light source compensation.

2. The method for enhancing visual perception of an inspection robot based on dynamic light source compensation according to claim 1, characterized in that: Determining whether to enhance visual perception based on the detection area image includes: obtaining a shadow area and a reflection area based on the detection area image; if the sum of the shadow area and the reflection area is greater than or equal to a preset area threshold, determining that visual perception is enhanced; if the sum of the shadow area and the reflection area is less than the preset area threshold, determining that visual perception does not need to be enhanced.

3. The method for enhancing visual perception of an inspection robot based on dynamic light source compensation according to claim 1, characterized in that: The dynamic lighting compensation requirement coefficient is determined based on the position information, the three-dimensional lighting environment model, the maximum reflectivity and the humidity influencing factor, including: obtaining the Euclidean distance from the inspection robot to the nearest fixed light source position based on the position information and the three-dimensional lighting environment model; obtaining the minimum distance at which the light intensity of the fixed light source is not attenuated and the maximum distance at which the light intensity attenuates to the ambient light level; determining the normalized distance based on the Euclidean distance, the minimum distance and the maximum distance; determining the dynamic lighting compensation requirement coefficient based on the normalized distance, the maximum reflectivity and the humidity influencing factor.

4. The method for enhancing visual perception of an inspection robot based on dynamic light source compensation according to claim 3 is characterized in that: Determining a dynamic illumination compensation requirement coefficient according to the normalized distance, the maximum reflectivity, and the humidity influencing factor includes: according to the formula Determine the dynamic lighting compensation requirement coefficient , where D is the normalized distance, is the light attenuation threshold, is the highest reflectivity, and H is the humidity influence factor.

5. The method for enhancing visual perception of an inspection robot based on dynamic light source compensation according to claim 1, characterized in that: Determine the light intensity and color temperature according to the dynamic light compensation requirement coefficient, the detection area image, the maximum reflectivity and the humidity influencing factor, including: obtaining the grayscale value of each pixel point according to the detection area image; determining the texture complexity of the detection area according to the grayscale value of the pixel point; obtaining the movement speed, exposure time and detection area image width of the inspection robot; determining the motion blur risk according to the movement speed, exposure time and detection area image width of the inspection robot; determining the light intensity according to the texture complexity, the motion blur risk and the dynamic light compensation requirement coefficient; obtaining the reference color temperature; and determining the color temperature according to the reference color temperature, the maximum reflectivity and the humidity influencing factor.

6. The method for enhancing visual perception of an inspection robot based on dynamic light source compensation according to claim 5, characterized in that: Determining the illumination intensity according to the texture complexity, the motion blur risk, and the dynamic illumination compensation requirement coefficient includes: determining the illumination intensity according to the formula Determine the light intensity I, where is the baseline light intensity, is the dynamic lighting compensation requirement coefficient, is the texture complexity, Risk of motion blur.

7. The method for enhancing visual perception of an inspection robot based on dynamic light source compensation according to claim 5, characterized in that: Determining the color temperature according to the reference color temperature, the maximum reflectivity, and the humidity influencing factor includes: determining the color temperature according to the formula Determine the color temperature T, where is the base color temperature, is the highest reflectivity, and H is the humidity influence factor.

8. The method for enhancing visual perception of an inspection robot based on dynamic light source compensation according to claim 1, characterized in that: Determining the lighting angle according to the preset distance, the undetected area image and the detected area image includes: obtaining the future shadow area according to the undetected area image; obtaining the shadow area according to the detected area image; and determining the lighting angle according to the future shadow area, the shadow area and the preset distance.

9. The method for enhancing visual perception of an inspection robot based on dynamic light source compensation according to claim 8, characterized in that: Determining the lighting angle according to the future shadow area, the shadow area, and the preset distance includes: determining the lighting angle according to the formula Determine the lighting angle ,in, is the base projection angle, is the future shadow area, is the shaded area, For the preset distance, To preset the minimum lighting angle, To preset the maximum lighting angle, max is the maximum value function, and min is the minimum value function.

10. A visual perception enhancement system for an inspection robot based on dynamic light source compensation, used to execute the visual perception enhancement method for an inspection robot based on dynamic light source compensation as claimed in any one of claims 1 to 9, characterized in that: include: The detection area image module is used to obtain the current detection area image through the built-in visual sensor of the inspection robot; a judgment module, configured to determine whether to enhance visual perception based on the detection area image; A position information and model building module is used to obtain the position information of the inspection robot and build a three-dimensional lighting environment model of the underground sewage treatment plant if it is determined to enhance visual perception, wherein the three-dimensional lighting environment model includes the fixed light source position, equipment occlusion relationship, pipeline layout and reflective area parameters; a maximum reflectivity module is used to obtain the maximum reflectivity of the current detection area based on the position information and the three-dimensional lighting environment model; a humidity influence factor module is used to obtain the humidity influence factor of the current detection area through the humidity sensor carried by the inspection robot; a dynamic lighting compensation demand coefficient module is used to obtain the humidity influence factor of the current detection area based on the position information, the three-dimensional lighting environment model, the maximum reflectivity and The humidity influencing factor determines the dynamic lighting compensation requirement coefficient; the light intensity and color temperature module is used to determine the light intensity and color temperature based on the dynamic lighting compensation requirement coefficient, the detection area image, the maximum reflectivity and the humidity influencing factor; the undetected area image module is used to obtain the undetected area image at a preset distance in the forward direction of the inspection robot through a depth camera; the lighting angle module is used to determine the lighting angle based on the preset distance, the undetected area image and the detection area image; the dynamic light source compensation module is used to perform dynamic light source compensation using the controllable fill light device of the inspection robot according to the light intensity, the color temperature and the lighting angle.

Images