Low-illumination tunnel camera identification and positioning method based on depth camera

Abstract

The invention discloses a low-illumination tunnel camera recognition and positioning method based on a depth camera, and relates to the technical field of pavement maintenance, and the method comprises the steps: collecting a camera image in a low-illumination environment in a tunnel through a depth camera, carrying out the enhancement processing of the image through a Zero-DCE + + algorithm, and obtaining an enhanced image; marking the enhanced image, constructing a training data set comprising a gun type camera and a circular camera, adopting different marking rules for the gun type camera and the circular camera, and training a YOLOv8 model by using the marking data set to obtain a recognition model; calculating a positioning point of a camera based on the recognition model detection frame, obtaining a spatial three-dimensional coordinate of the positioning point through mapping of a depth camera, and for a detection frame center point and a cleaning positioning point, performing calculation according to coordinates of the left upper corner and the right lower corner of the detection frame and the height of the detection frame; and analyzing the lateral or forward view of the camera through a geometric extraction method.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of road maintenance technology, and particularly relates to a low-illumination tunnel camera recognition and positioning method based on a depth camera. BACKGROUND

[0002] The tunnel camera is usually installed on the top of the tunnel to monitor the inside of the tunnel in real time, and is used for safety monitoring, illegal behavior snapshot, accident investigation and traffic flow analysis and the like. Therefore, how to improve the intelligent level and safety of road maintenance by using advanced technical means has become one of the problems to be solved at present.

[0003] In the field of road maintenance, due to long-term exposure to tunnel air, suspended particles in the air, automobile exhaust and dust and oil stains will accumulate on the camera mirror surface, which not only blocks the monitoring picture, but also damages the internal circuit of the monitoring equipment, reduces the service life of the equipment, at the same time, the uncleaned lens pollution will cause the monitoring picture to appear blurred and distorted, affecting the safe operation of the tunnel, due to the small size of the camera, plus the low tunnel illumination, the camera is difficult to position, and it is inconvenient to realize the mechanical cleaning of the camera. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides a low-illumination tunnel camera recognition and positioning method based on a depth camera, which solves the problem that due to long-term exposure to tunnel air, suspended particles in the air, automobile exhaust and dust and oil stains will accumulate on the camera mirror surface, which not only blocks the monitoring picture, but also damages the internal circuit of the monitoring equipment, reduces the service life of the equipment, at the same time, the uncleaned lens pollution will cause the monitoring picture to appear blurred and distorted, affecting the safe operation of the tunnel, due to the small size of the camera, plus the low tunnel illumination, the camera is difficult to position, and it is inconvenient to realize the mechanical cleaning of the camera.

[0006] To solve the above technical problems, the present application provides the following technical scheme:

[0007] In a first aspect, the present application provides a low-illumination tunnel camera recognition and positioning method based on a depth camera, which comprises:

[0008] A depth camera is used to collect camera images in a low-illumination environment in the tunnel, and a Zero-DCE++ algorithm is used to enhance the images to obtain enhanced images;

[0009] The enhanced images are labeled to construct a training data set containing gun-type cameras and circular cameras, wherein different labeling rules are used for gun-type cameras and circular cameras;

[0010] A YOLOv8 model was trained using the labeled dataset to obtain a recognition model;

[0011] The camera's positioning point is calculated based on the detection frame of the recognition model, and the spatial three-dimensional coordinates of the positioning point are obtained through depth camera mapping. For the center point of the detection frame and the cleaning positioning point, the coordinates of the upper left and lower right corners of the detection frame and the height of the detection frame are calculated.

[0012] By analyzing the side or front view of the camera using geometric extraction, the horizontal and pitch angles of the camera are calculated. Rectangular elements are extracted from the RGB image, and the two-dimensional direction is mapped to the actual depth map to obtain the actual pose angle of the camera.

[0013] As a preferred embodiment of the low-light tunnel camera identification and positioning method based on a depth camera described in this invention, the step of using the Zero-DCE++ algorithm to enhance the image to obtain the enhanced image specifically includes:

[0014] The enhanced images were annotated using Labelimg annotation software, with different annotation rules set for bullet cameras and circular cameras. The bullet cameras were further categorized into monitor cameras. column There are two categories: monitorcam and monitorcam_head. Circular cameras are classified as monitorcam. circle The class is used to construct a training dataset from the labeled image data.

[0015] As a preferred embodiment of the low-light tunnel camera identification and localization method based on a depth camera described in this invention, the step of training a YOLOv8 model using a labeled dataset to obtain a low-light camera identification model specifically includes:

[0016] The training dataset was trained using a YOLOv8s model;

[0017] First, the training dataset is divided into training set, validation set and test set according to the proportions;

[0018] During training, the number of training rounds and the receptive field size are set.

[0019] For each training round, the RGB image stream and corresponding annotation information are input into the YOLOv8s model. The loss function value is calculated through forward propagation, and the model parameters are updated using the backpropagation algorithm.

[0020] During training, the model performance is evaluated periodically using the validation set, and hyperparameters are adjusted to optimize model performance until all scheduled training rounds are completed.

[0021] After training, the performance of the obtained model is validated using a test set.

[0022] As a preferred embodiment of the low-light tunnel camera identification and positioning method based on a depth camera according to the present invention, the step of calculating the camera positioning point based on the detection box of the recognition model and obtaining the three-dimensional coordinates through depth camera mapping specifically includes:

[0023] Based on the coordinates (a1, b1) and (a2, b2) of the top-left and bottom-right corners of the detection box, and the height h of the detection box, the formula is used. Calculate the center point (ac, bc) of the detection box;

[0024] For circular cameras with spherical protective covers, additional cleaning positioning points (ad, bd) are used, calculated using the formula ad = ac.

[0025] Based on the coordinates of the two-dimensional positioning points, the two-dimensional coordinates are converted into three-dimensional coordinates (x, y, z) in actual space using the depth map information provided by the depth camera and the registration alignment.

[0026] As a preferred embodiment of the low-light tunnel camera identification and positioning method based on a depth camera according to the present invention, the step of obtaining the spatial three-dimensional coordinates of the positioning point through depth camera mapping specifically includes:

[0027] On the depth map, find the depth value D of the corresponding location based on the coordinates (ad, bd) of the two-dimensional positioning point;

[0028] Using the intrinsic parameters of the depth camera, including focal length and optical center coordinates, and combining them with the depth value D, the two-dimensional coordinates are converted into three-dimensional spatial coordinates (x, y, z). The depth value D is directly used as the z-value in the three-dimensional spatial coordinates, expressed as:

[0029] x = (ac - cx) * D / fx;

[0030] y = (bc - cy) * D / fy;

[0031] z = D;

[0032] Where ac is the x-coordinate of the center point of the detection box, bc is the y-coordinate of the center point of the detection box, cx is the coordinate of the optical center in the x-axis direction in the intrinsic parameters of the depth camera, cy is the coordinate of the optical center in the y-axis direction in the intrinsic parameters of the depth camera, fx is the value of the focal length in the x-axis direction in the intrinsic parameters of the depth camera, and fy is the value of the focal length in the y-axis direction in the intrinsic parameters of the depth camera.

[0033] As a preferred embodiment of the low-light tunnel camera identification and positioning method based on a depth camera described in this invention, the step of analyzing the side or front view of the camera using a geometric extraction method to calculate the horizontal and vertical angles of the camera specifically includes:

[0034] Based on the three-dimensional spatial coordinates (x, y, z), rectangular elements representing the camera outline are extracted from the RGB image;

[0035] Map the two-dimensional orientation onto the actual depth map to determine the corresponding position on the depth map;

[0036] The horizontal and vertical pitch angles of the camera relative to the reference plane are calculated using geometric relationships, expressed as follows:

[0037] α = atan2(y,x);

[0038]

[0039] Where α is the horizontal angle and β is the pitch angle.

[0040] As a preferred embodiment of the low-light tunnel camera identification and positioning method based on a depth camera according to the present invention, the step of calculating the pose angle by analyzing the side or front view of the camera specifically includes:

[0041] When analyzing the side view of the camera, the side vector of the camera is determined based on the rectangular elements, and then the horizontal angle α is calculated using the angle between this vector and the horizontal axis.

[0042] When analyzing the frontal view of the camera, the vertical vector is determined by imaging the head surface, and the angle between this vector and the vertical axis is calculated as the pitch angle β.

[0043] By combining the two angle values, the actual pose angle of the camera can be accurately located.

[0044] As a preferred embodiment of the low-light tunnel camera identification and positioning method based on a depth camera according to the present invention, wherein: adjusting the angle of the cleaning device to align it with the lens or protective cover of the camera specifically includes:

[0045] Based on the calculated pose angle information, determine the position and orientation of the camera relative to the cleaning device;

[0046] The horizontal angle represents the rotation angle of the camera in the horizontal direction, while the pitch angle represents the tilt angle of the camera in the vertical direction.

[0047] Use the calculated horizontal angle as the angle that the cleaning device needs to be adjusted in the horizontal direction to ensure that the nozzle can be accurately aligned with the center of the camera on the horizontal plane.

[0048] Use the calculated pitch angle as the angle that needs to be adjusted in the vertical direction of the cleaning device to ensure that the nozzle can be moved up and down to the correct height to align with the camera's mirror or protective cover.

[0049] In a second aspect, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, it implements any step of the low-light tunnel camera identification and positioning method based on a depth camera as described in the first aspect of the present invention.

[0050] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the low-light tunnel camera identification and positioning method based on a depth camera as described in the first aspect of the present invention.

[0051] The beneficial effects of this invention are as follows: By combining the Zero-DCE++ image enhancement algorithm with the YOLOv8 target detection model, the recognition accuracy of cameras in tunnels under low-light conditions is effectively improved. By acquiring spatial three-dimensional information through a depth camera and combining geometric extraction and pose angle calculation methods, accurate positioning and attitude analysis of the camera are achieved. This not only solves the problem of low recognition rate of traditional methods in environments with insufficient light and complex backgrounds, but also provides reliable spatial coordinates and angles for subsequent cleaning or maintenance operations, significantly improving the automation level and work efficiency of tunnel monitoring equipment maintenance. Attached Figure Description

[0052] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a flowchart of the low-light tunnel camera identification and positioning method based on a depth camera in Example 1.

[0054] Figure 2 This is a schematic diagram of the camera location planning for data acquisition inside the tunnel in Example 2.

[0055] Figure 3 This is a schematic diagram of the Zero-DCE++ enhancement process in Example 2.

[0056] Figure 4 This is a schematic diagram of the labeling method for the bullet camera in Example 2.

[0057] Figure 5 This is a schematic diagram of the circular camera labeling method in Example 2.

[0058] Figure 6 This is a flowchart illustrating the camera positioning function implementation in Example 2.

[0059] Figure 7 This is a schematic diagram of the positioning point design for the camera being tested in Example 2.

[0060] Figure 8 This is a schematic diagram of the types of pose angles that need to be detected in Example 2.

[0061] Figure 9 The images show two scenarios for pose detection in Example 2 and their corresponding views.

[0062] Figure 10 This is a schematic diagram of the camera pose angle acquisition method based on geometric extraction in Example 2.

[0063] Figure 11 This is a schematic diagram of the performance parameters in Example 2. Detailed Implementation

[0064] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0065] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0066] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0067] Example 1, referring to Figure 1 - Figure 11 This is the first embodiment of the present invention, which provides a low-light tunnel camera identification and positioning method based on a depth camera, including the following steps:

[0068] S1. Use a depth camera to acquire camera images in low-light conditions inside the tunnel, and use the Zero-DCE++ algorithm to enhance the images to obtain the enhanced images;

[0069] Furthermore, Labelimg annotation software was used to annotate the enhanced images, with different annotation rules set for bullet cameras and circular cameras. The bullet cameras were further categorized into monitor cameras. column There are two categories: monitorcam and monitorcam_head. Circular cameras are classified as monitorcam.circle The class is then used to construct a training dataset from the labeled image data;

[0070] It should be noted that in low-light tunnel environments, ordinary cameras struggle to capture clear images. However, using a depth camera combined with the Zero-DCE++ image enhancement algorithm can effectively improve image brightness and detail clarity, providing high-quality image input for subsequent target recognition. Furthermore, when using Labelimg for annotation, setting separate annotation rules for bullet cameras and circular cameras can enhance the targeting and generalization ability of model training.

[0071] S2. Label the enhanced images and construct a training dataset containing bullet cameras and circular cameras. Different labeling rules are used for bullet cameras and circular cameras. Use the labeled dataset to train the YOLOv8 model and obtain the recognition model.

[0072] Furthermore, the YOLOv8s model is used to train the training dataset;

[0073] First, the training dataset is divided into training set, validation set and test set according to the proportions;

[0074] During training, the number of training rounds and the receptive field size are set.

[0075] For each training round, the RGB image stream and corresponding annotation information are input into the YOLOv8s model. The loss function value is calculated through forward propagation, and the model parameters are updated using the backpropagation algorithm.

[0076] During training, the model performance is evaluated periodically using the validation set, and hyperparameters are adjusted to optimize model performance until all scheduled training rounds are completed.

[0077] After training, the performance of the obtained model is validated using a test set;

[0078] It should be noted that during model training, YOLOv8s is used as the base model, balancing recognition accuracy and inference speed. The training set, validation set, and test set are divided in a reasonable ratio to ensure the stability and generalization ability of the model training. Hyperparameters such as the number of training rounds and the size of the receptive field are set according to the actual dataset size and training convergence to improve the model's adaptability and recognition robustness in low-light environments.

[0079] S3. Calculate the camera's positioning point based on the detection box of the recognition model, and obtain the spatial three-dimensional coordinates of the positioning point through depth camera mapping. For the center point of the detection box and the cleaning positioning point, calculate based on the coordinates of the upper left and lower right corners of the detection box and the height of the detection box.

[0080] Furthermore, the camera positioning points are calculated based on the detection bounding boxes of the recognition model, and the 3D coordinates are obtained through depth camera mapping, specifically including:

[0081] Based on the coordinates (a1, b1) and (a2, b2) of the top-left and bottom-right corners of the detection box, and the height h of the detection box, the formula is used. Calculate the center point (ac, bc) of the detection box;

[0082] For circular cameras with spherical protective covers, additional cleaning positioning points (ad, bd) are used, calculated using the formula ad = ac.

[0083] Based on the coordinates of the two-dimensional positioning points, the depth map information provided by the depth camera is used to convert the two-dimensional coordinates into three-dimensional coordinates (x, y, z) in the actual space.

[0084] The spatial three-dimensional coordinates of the positioning point are obtained through depth camera mapping, specifically including:

[0085] On the depth map, find the depth value D of the corresponding location based on the coordinates (ad, bd) of the two-dimensional positioning point;

[0086] Using the intrinsic parameters of the depth camera, including focal length and optical center coordinates, and combining them with the depth value D, the two-dimensional coordinates are converted into three-dimensional spatial coordinates (x, y, z). The depth value D is directly used as the z-value in the three-dimensional spatial coordinates, expressed as:

[0087] x = (ac - cx) * D / fx;

[0088] y = (bc - cy) * D / fy;

[0089] z = D;

[0090] Where ac is the x-coordinate of the center point of the detection box, bc is the y-coordinate of the center point of the detection box, cx is the coordinate of the optical center in the x-axis direction in the intrinsic parameters of the depth camera, cy is the coordinate of the optical center in the y-axis direction in the intrinsic parameters of the depth camera, fx is the value of the focal length in the x-axis direction in the intrinsic parameters of the depth camera, and fy is the value of the focal length in the y-axis direction in the intrinsic parameters of the depth camera.

[0091] It should be noted that by recognizing the detection box information output by the model and combining it with the depth map data provided by the depth camera, the target point can be mapped from the two-dimensional image coordinate system to the three-dimensional spatial coordinate system. The process relies on the precise matching of the camera intrinsic parameters (focal length, optical center) and the depth value to ensure the accuracy of the spatial coordinates of the positioning point and provide a reliable spatial basis for subsequent attitude analysis and cleaning operations.

[0092] S4. Analyze the side or front view of the camera using geometric extraction method, calculate the horizontal and pitch angles of the camera, extract rectangular elements from the RGB image, map the two-dimensional direction to the actual depth map, and obtain the actual pose angle of the camera.

[0093] Furthermore, based on the three-dimensional spatial coordinates (x, y, z), rectangular elements representing the camera outline are extracted from the RGB image;

[0094] Map the two-dimensional orientation onto the actual depth map to determine the corresponding position on the depth map;

[0095] The horizontal and vertical pitch angles of the camera relative to the reference plane are calculated using geometric relationships, expressed as follows:

[0096] α = atan2(y,x);

[0097]

[0098] Where α is the horizontal angle and β is the pitch angle;

[0099] Analyzing the side or front view of the camera to calculate the pose angle specifically includes:

[0100] When analyzing the side view of the camera, the side vector of the camera is determined based on the rectangular elements, and then the horizontal angle α is calculated using the angle between this vector and the horizontal axis.

[0101] When analyzing the frontal view of the camera, the vertical vector is determined by imaging the head surface, and the angle between this vector and the vertical axis is calculated as the pitch angle β.

[0102] By combining the two angle values, the actual pose angle of the camera can be accurately determined.

[0103] Based on the calculated pose angle information, determine the position and orientation of the camera relative to the cleaning device;

[0104] The horizontal angle represents the rotation angle of the camera in the horizontal direction, while the pitch angle represents the tilt angle of the camera in the vertical direction.

[0105] Use the calculated horizontal angle as the angle that the cleaning device needs to be adjusted in the horizontal direction to ensure that the nozzle can be accurately aligned with the center of the camera on the horizontal plane.

[0106] Use the calculated pitch angle as the angle that needs to be adjusted in the vertical direction of the cleaning device to ensure that the nozzle can be moved up and down to the correct height to align with the camera's mirror or protective cover.

[0107] It should be noted that when analyzing camera attitude using the geometric extraction method, the horizontal and pitch angles of the camera can be effectively extracted based on the correspondence between the rectangular structure in the RGB image and the three-dimensional direction in the depth map. This method does not require additional sensor support and can achieve accurate calculation of attitude angles by relying solely on the depth camera, providing an angular basis for the automatic alignment of the cleaning device. It has good engineering applicability and deployability.

[0108] This embodiment also provides a computer device applicable to the low-light tunnel camera identification and positioning method based on a depth camera, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the low-light tunnel camera identification and positioning method based on a depth camera as proposed in the above embodiment.

[0109] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0110] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the low-light tunnel camera identification and positioning method based on a depth camera as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0111] In summary, this invention effectively improves the recognition accuracy of cameras in tunnels under low-light conditions by combining the Zero-DCE++ image enhancement algorithm with the YOLOv8 target detection model. By acquiring spatial 3D information through a depth camera and combining geometric extraction and pose angle calculation methods, it achieves precise positioning and attitude analysis of the camera. This not only solves the problem of low recognition rate of traditional methods in insufficient lighting and complex background environments, but also provides reliable spatial coordinates and angles for subsequent cleaning or maintenance operations, significantly improving the automation level and work efficiency of tunnel monitoring equipment maintenance.

[0112] Example 2, refer to Figures 2-11 This is the second embodiment of the present invention, which provides an implementation method for tunnel camera identification and positioning based on a depth camera, specifically including:

[0113] Step 1: Take photos of the cameras inside the tunnel in real time, acquiring image data from different positions and shooting angles under low light conditions inside the tunnel, such as... Figure 2 As shown;

[0114] After completing the actual data acquisition, the Zero-DCE++ enhancement method was used to enhance the images, reducing the impact of insufficient lighting on target recognition due to low distinction between target edge features and background. Figure 3 As shown.

[0115] Step 2: Use Labelimg annotation software to annotate the image data, mainly targeting bullet and circular camera types. Bullet camera annotations are categorized into two label types; the annotation categories and methods are as follows: Figure 4 As shown;

[0116] The first is the monitorcam_column category, which is the entire bullet camera. It requires that the bounding box can encompass the entire image of the bullet camera rectangle in the frame, regardless of the camera's rotation angle within the frame.

[0117] The second category is monitorcam_head, also known as bullet camera head. It requires that the annotation box can encompass the entire lens surface of the bullet camera, and the center of the annotation box should be as close as possible to the center of the entire surface where the lens is located.

[0118] Circular cameras are categorized as monitorcam_circle, such as... Figure 5 As shown, the label box should encompass the entire image of the camera in the frame. If the lower part of the circular camera has a spherical protective cover, the camera head category is not labeled; if the lens is exposed without a protective cover, the camera head category is labeled. For spherical cameras with a protective cover, the lower protective cover can be located based on the overall label box, while for spherical cameras with exposed lenses, the lens part can be located directly.

[0119] Step 3: Use YOLOv8s to train the recognition model on the labeled dataset. The dataset is divided into training, validation, and test sets in a 7:1.5:1.5 ratio, and then trained separately. After labeling, a dataset is created, and a low-light camera recognition model is obtained by training the YOLOv8 model. Since the training and validation sets are used for training, while the test set is isolated and used specifically as a test set for model recognition performance, the recognition effects of the various models are verified using the test sets in their respective datasets.

[0120] 6093 low-light images were obtained from the Qinling Zhongnanshan Highway Tunnel and the Nanwutai Highway Tunnel. After image enhancement, the images were labeled using the Labeming annotation software. The dataset was divided into training, validation, and test sets, and then trained using YOLOv8s. The training epochs were set to 400, the stride to 16, and the receptive field size to 640×640, with no other default hyperparameters changed.

[0121] The final model obtained after training, after being validated on the test set, has the following performance parameters: Figure 11 As shown in the table, the parameters generally perform very well, with the four parameters for each category having high values, indicating a good recognition effect.

[0122] Step 4: Obtain the positioning center point and perform 3D coordinate transformation. The specific process of spatial 3D coordinate positioning for camera target recognition is as follows: Figure 6 As shown, the depth camera simultaneously outputs the RGB color image and the registered and aligned depth map from the shooting perspective as a video stream. The low-light camera recognition model performs target detection on the processed RGB image, obtains two-dimensional detection boxes and positioning points, maps them onto the depth map, performs the conversion of actual three-dimensional coordinates, and finally obtains the actual spatial three-dimensional coordinates of the camera.

[0123] The detection bounding box based on the YOLOv8 recognition model, and the completed localization point design are as follows: Figure 7 As shown, with the top left corner as the origin, the vertical coordinate y points downwards and the horizontal coordinate x points to the right, the coordinates of the top left corner (a1, b1) and the bottom right corner (a2, b2) of the detection box are output, and the detection box is drawn. To achieve specific location positioning of various cameras, the center point (ac, bc) of the detection box is selected as the positioning point, and the relevant calculation is shown in Equation (1).

[0124]

[0125] In the formula, w is the width of the detection frame and h is the height of the detection frame.

[0126] Since some circular cameras have models where the lower lens surface is not exposed and has a spherical protective cover, calculations are required.

[0127] Additional positioning points are used to clean the protective cover below. Taking the height h of the detection frame as a reference, the point (ad, bd) that is shifted downwards by h / 4 from the center point is used as the cleaning positioning point, and the calculation is as shown in equation (2).

[0128]

[0129] Based on the above positioning point design, the two-dimensional positioning point coordinates of the color image obtained after recognition are transformed into actual three-dimensional spatial point coordinates.

[0130] Step 5: After implementing the camera positioning function, it is also necessary to implement the camera pose detection function on the rear detection camera so that the nozzle of the cleaning device can be aligned vertically or at a certain angle with the camera lens. Inside the tunnel, the spherical camera with a protective cover at the bottom can be aligned for cleaning based on the corresponding positioning point, such as... Figure 8 The demand for camera pose detection is mainly concentrated on various models of bullet cameras.

[0131] The pose angles that a bullet camera needs to detect are mainly the horizontal and pitch angles, such as... Figure 9As shown. Since the camera needs to provide feedback on the situation inside the tunnel from a normal, non-rotating perspective when it is working, the roll angle of the camera itself remains unchanged, and there is no need to obtain the roll angle of the gun-type camera when performing cleaning operations.

[0132] The shape of the gun-type camera inside the tunnel can basically be regarded as a four-sided prism, while the rear-mounted detection camera may encounter two kinds of viewing angles when performing detection tasks: frontal view and side view, such as... Figure 10 As shown, in the frontal view, the tested camera faces the camera directly, and the image acquired by the camera is mainly of the camera's head, with less side imagery. In the side view, the tested camera faces the camera from the side, and the image acquired by the camera is mainly of the camera's side, with less or no head imagery. When detecting pose, because the cleaning nozzle is below the rear-mounted detection camera, and cleaning requires the nozzle to be aligned with the camera's head, the camera's view does not face the rear, directly above, or directly below the tested camera.

[0133] Based on the detection camera's perspective and camera imaging analysis, combined with the geometric features of the bullet camera's shape, this invention proposes two camera pose angle detection methods based on geometric extraction: side vector acquisition and front vector acquisition.

[0134] Depend on Figure 9 As can be seen from the side view, the side image of the camera is basically rectangular. The rectangular elements are extracted by using a two-dimensional RGB image. Based on the two-dimensional direction of the rectangle, it is mapped to the actual depth map to obtain the actual side vector, and then the pose angle is obtained.

[0135] For a forward view, the image of the head surface captured by the camera can be mapped to the actual head surface in the depth map to obtain the vertical vector of that surface, thereby obtaining the pose angle.

[0136] The method for obtaining the camera's pose is as follows: Figure 10 As shown, based on the implemented camera target localization method and the method for obtaining the side vector and surface vertical vector of the gun-type camera, image processing is performed according to the physical object of the gun-type camera and the two-dimensional imaging in the RGB image to extract effective geometric elements, obtain pose calculation sampling points, and combine the localization function to obtain the actual three-dimensional coordinates of the sampling points, calculate the spatial vector and the horizontal angle and pitch angle of the pose.

[0137] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A low-light tunnel camera identification and positioning method based on a depth camera, characterized in that: include: A depth camera is used to acquire camera images in a low-light environment inside the tunnel, and the images are enhanced using the Zero-DCE++ algorithm to obtain an enhanced image; The enhanced images are labeled to construct a training dataset containing bullet cameras and circular cameras. Different labeling rules are used for bullet cameras and circular cameras. The YOLOv8 model is trained using the labeled dataset to obtain a recognition model. The camera's positioning point is calculated based on the detection frame of the recognition model, and the spatial three-dimensional coordinates of the positioning point are obtained through depth camera mapping. For the center point of the detection frame and the cleaning positioning point, the coordinates of the upper left and lower right corners of the detection frame and the height of the detection frame are calculated. By analyzing the side or front view of the camera using geometric extraction, the horizontal and pitch angles of the camera are calculated. Rectangular elements are extracted from the RGB image, and the two-dimensional direction is mapped to the actual depth map to obtain the actual pose angle of the camera.

2. The low-light tunnel camera identification and positioning method based on a depth camera as described in claim 1, characterized in that: The enhancement process using the Zero-DCE++ algorithm to obtain the enhanced image specifically includes: The enhanced images were annotated using Labelimg annotation software, with different annotation rules set for bullet cameras and circular cameras. The bullet cameras were further categorized into monitor cameras. column There are two categories: monitorcam and monitorcam_head. Circular cameras are classified as monitorcam. circle The class is used to construct a training dataset from the labeled image data.

3. The low-light tunnel camera identification and positioning method based on a depth camera as described in claim 2, characterized in that: The process of training a YOLOv8 model using a labeled dataset to obtain a low-light camera recognition model specifically includes: The training dataset was trained using a YOLOv8s model; First, the training dataset is divided into training set, validation set and test set according to the proportions; During training, the number of training rounds and the receptive field size are set. For each training round, the RGB image stream and corresponding annotation information are input into the YOLOv8s model. The loss function value is calculated through forward propagation, and the model parameters are updated using the backpropagation algorithm. During training, the model performance is evaluated periodically using the validation set, and hyperparameters are adjusted to optimize model performance until all scheduled training rounds are completed. After training, the performance of the obtained model is validated using a test set.

4. The low-light tunnel camera identification and positioning method based on a depth camera as described in claim 3, characterized in that: The process of calculating camera positioning points based on the detection bounding box of the recognition model and obtaining three-dimensional coordinates through depth camera mapping specifically includes: Based on the coordinates (a1, b1) and (a2, b2) of the top-left and bottom-right corners of the detection box, and the height h of the detection box, the formula is used. Calculate the center point (ac, bc) of the detection box; For circular cameras with spherical protective covers, additional cleaning positioning points (ad, bd) are used, calculated using the formula ad = ac. Based on the coordinates of the two-dimensional positioning points, the two-dimensional coordinates are converted into three-dimensional coordinates (x, y, z) in actual space using the depth map information provided by the depth camera and the registration alignment.

5. The low-light tunnel camera identification and positioning method based on a depth camera as described in claim 4, characterized in that: The process of obtaining the spatial three-dimensional coordinates of the positioning point through depth camera mapping specifically includes: On the depth map, find the depth value D of the corresponding location based on the coordinates (ad, bd) of the two-dimensional positioning point; Using the intrinsic parameters of the depth camera, including focal length and optical center coordinates, and combining them with the depth value D, the two-dimensional coordinates are converted into three-dimensional spatial coordinates (x, y, z). The depth value D is directly used as the z-value in the three-dimensional spatial coordinates, expressed as: x = (ac - cx) * D / fx; y = (bc - cy) * D / fy; z = D; Where ac is the x-coordinate of the center point of the detection box, bc is the y-coordinate of the center point of the detection box, cx is the coordinate of the optical center in the x-axis direction in the intrinsic parameters of the depth camera, cy is the coordinate of the optical center in the y-axis direction in the intrinsic parameters of the depth camera, fx is the value of the focal length in the x-axis direction in the intrinsic parameters of the depth camera, and fy is the value of the focal length in the y-axis direction in the intrinsic parameters of the depth camera.

6. The low-light tunnel camera identification and positioning method based on a depth camera as described in claim 5, characterized in that: The step of analyzing the side or front view of the camera using a geometric extraction method to calculate the camera's horizontal and vertical angles specifically includes: Based on the three-dimensional spatial coordinates (x, y, z), rectangular elements representing the camera outline are extracted from the RGB image; Map the two-dimensional direction onto the actual depth map to determine the corresponding position on the depth map; The horizontal and vertical pitch angles of the camera relative to the reference plane are calculated using geometric relationships, expressed as follows: α = atan2(y,x); Where α is the horizontal angle and β is the pitch angle.

7. The low-light tunnel camera identification and positioning method based on a depth camera as described in claim 6, characterized in that: The calculation of pose angles based on the side or front view of the camera specifically includes: When analyzing the side view of the camera, the side vector of the camera is determined based on the rectangular elements, and then the horizontal angle α is calculated using the angle between this vector and the horizontal axis. When analyzing the frontal view of the camera, the vertical vector is determined by imaging the head surface, and the angle between this vector and the vertical axis is calculated as the pitch angle β. By combining the two angle values, the actual pose angle of the camera can be accurately located.

8. The low-light tunnel camera identification and positioning method based on a depth camera as described in claim 7, characterized in that: Adjusting the angle of the cleaning device to align it with the lens or protective cover of the camera specifically includes: Based on the calculated pose angle information, determine the position and orientation of the camera relative to the cleaning device; The horizontal angle represents the rotation angle of the camera in the horizontal direction, while the pitch angle represents the tilt angle of the camera in the vertical direction. Use the calculated horizontal angle as the angle that the cleaning device needs to be adjusted in the horizontal direction to ensure that the nozzle can be accurately aligned with the center of the camera on the horizontal plane. Use the calculated pitch angle as the angle that needs to be adjusted in the vertical direction of the cleaning device to ensure that the nozzle can be moved up and down to the correct height to align with the camera's mirror or protective cover.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the low-light tunnel camera identification and positioning method based on a depth camera as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the low-light tunnel camera identification and positioning method based on a depth camera as described in any one of claims 1 to 8.

Images