Method for environment detection and facility identification of radio and television machine room

Abstract

The invention discloses a method for environment detection and facility identification of a radio and television machine room. The method is used for environment detection and facility identification of the radio and television machine room. The algorithm aims at improving the autonomous operation capability of the broadcasting and television machine room auxiliary inspection robot in complex broadcasting and television working environments such as a traditional launching machine room, so that the robot can efficiently replace manpower to execute equipment detection and maintenance tasks. Therefore, a visual understanding technology based on deep learning is introduced, so that the robot can sense the internal environment of the machine room and accurately recognize and classify various broadcast and television facilities. In order to realize core functions of environment detection and facility identification, the robot is equipped with an infrared camera and a high-definition camera so as to enhance the sensing ability of the robot under different illumination conditions. Experimental results show that the robot carrying the algorithm can efficiently and accurately complete environment detection and facility identification tasks of the radio and television machine room, and technical support is provided for intelligent operation and maintenance of radio and television equipment.

Description

Technical Field

[0001] The present invention relates to the field of algorithm technology for robots in the broadcasting and television field, and in particular to an algorithm for a robot based on visual understanding to detect the environment and identify facilities in a broadcasting and television room. Background Art

[0002] With the rapid development of computer vision technology, the use of robots equipped with cameras to replace human operators in visual processing and automated operations has become a hot research topic. In traditional broadcasting and television, the application of computer vision technology is becoming increasingly important, providing unique capabilities and advantages for multiple key scenarios. Leveraging advanced visual algorithms and intelligent perception capabilities, camera-equipped robots can gain a more comprehensive understanding of their environment and accurately identify various targets, demonstrating greater efficiency and reliability than traditional human operators in complex operating environments. This technology not only reduces labor costs but also improves the stability and accuracy of task execution.

[0003] Traditional robots rely primarily on manual operation. Directly deploying such robots in broadcast and television equipment rooms is not only inefficient and limited in accuracy, but also suffers from numerous limitations. They are unable to meet the demands of broadcast and television transmitter room environmental detection and intelligent identification of loaded broadcast and television equipment. Furthermore, broadcast and television transmitter rooms are often located in harsh natural environments, such as high mountains. Transmitting and relaying broadcast and television signals in these environments is not only complex but also poses a significant safety threat to human maintenance personnel. With the rapid advancement of robotics technology, an increasing number of robots have been introduced for broadcast and television applications. However, current broadcast and television room inspection robots generally lack facility recognition capabilities, making it difficult to accurately determine the status and operational status of various equipment within the room. They still require manual remote control, failing to achieve true intelligence and autonomy. To address these issues, there is an urgent need to develop a broadcast and television room inspection robot with autonomous environmental detection and facility identification capabilities. Such a robot would be able to automatically perceive its surroundings, accurately identify various equipment within the broadcast and television room, and perform real-time analysis and decision-making, significantly improving work efficiency, reducing the need for manual intervention, and effectively ensuring operator safety.

[0004] The radio and television room auxiliary inspection robot is designed for radio and television room environmental detection and facility identification. It must have accurate target differentiation capabilities to identify whether the objects it encounters are staff members or equipment facilities. When the radio and television room auxiliary inspection robot detects that the target is a staff member, it will automatically adjust its path and intelligently avoid obstacles, and continue to move forward without additional operations, thereby effectively avoiding collisions and ensuring safety. When the robot identifies that the target is an object other than a staff member (such as room equipment or other obstacles), it needs to accurately determine the location of the object and autonomously perform obstacle avoidance operations based on its position information. This includes dynamically planning new movement paths to bypass obstacles and maintain normal operation.

[0005] Currently, mainstream object detection algorithms can be divided into three categories: Transformer-based algorithms, two-stage detection algorithms, and single-stage detection algorithms. Single-stage detection algorithms excel in detection speed and performance, becoming an important deep learning model for image-level object detection and demonstrating numerous advantages in computer vision tasks. Single-stage detection algorithms focus on extracting high-level features from the overall image without processing detailed information pixel by pixel. This feature significantly reduces computational complexity and parameter requirements while maintaining efficient detection capabilities. Therefore, compared to other detection methods, single-stage detection algorithms are more adaptable in environments with limited computing resources and are particularly suitable for deployment on robotic platforms in scenarios such as broadcast and television transmission.

[0006] With the continued development of computer vision and robotics technologies, it has become possible to combine camera-equipped robots with advanced algorithms to solve practical problems in the broadcasting and television industry. This technological integration not only significantly improves the robots' detection accuracy and efficiency but also promotes a higher level of automation in the operation and maintenance of broadcasting and television equipment. Equipped with high-definition and infrared cameras, the robots can fully perceive the operating status of broadcasting and television transmission rooms and use advanced object detection algorithms to quickly identify critical facilities and potential anomalies. This intelligent detection approach not only reduces the workload of manual inspections but also improves the timeliness and reliability of problem detection, providing a strong guarantee for the safe and stable operation of broadcasting and television systems. Summary of the Invention

[0007] In view of the shortcomings of the existing technology, the present invention proposes an algorithm for robot based on visual understanding to detect the environment and identify facilities in radio and television room.

[0008] This paper proposes a visual understanding-based algorithm for a radio and television room inspection robot, which is used for environmental detection and facility identification in radio and television rooms. This algorithm is integrated into the radio and television room inspection robot system, enabling it to move autonomously via a tracked chassis and equipped with infrared and high-definition cameras for high-precision image acquisition. Furthermore, the radio and television room inspection robot is equipped with a robotic arm to perform routine operations and maintenance. This paper constructs a high-quality dataset and trains a deep learning model on the collected image data, thereby improving the accuracy of environmental detection and facility identification. Experimental results show that the radio and television room inspection robot equipped with this algorithm can efficiently and accurately perceive the room environment and identify radio and television transmission facilities, providing reliable technical support for the intelligent operation and maintenance of radio and television rooms.

[0009] A method for detecting the environment and identifying facilities in a radio and television room comprises the following steps:

[0010] 1) The execution equipment acquires real-time infrared images and high-definition images in a broadcast and television room environment;

[0011] 2) Read temperature information from infrared images and use visual understanding technology to identify targets in infrared images and high-definition images;

[0012] In step 2), visual understanding technology is used to identify targets in infrared images and high-definition images, including:

[0013] 2.1) Align and register the infrared image and the high-definition image;

[0014] 2.2) Extract features from the same regions in the registered infrared image and the registered high-definition image to obtain key visual features of the infrared image and key visual features of the visible light image;

[0015] 2.3) The infrared key visual features and the visible light key visual features are combined through feature fusion and feature matching to obtain matching results;

[0016] 2.4) Perform target identification, classification and status analysis on the matching results to obtain the target, target location and target working status in the broadcasting and television working environment in real time.

[0017] In step 1), the infrared image is obtained by an infrared camera on the execution device.

[0018] In step 1), the high-definition image is obtained by a high-definition camera on the execution device.

[0019] In step 2.3), feature fusion adopts weighted fusion, feature level fusion or deep neural network fusion.

[0020] In step 2.3), feature matching uses similarity metric matching or neural network matching.

[0021] In step 2.4), the targets are equipment, obstacles and fault points.

[0022] This invention proposes a vision-based robotic algorithm for environmental detection and facility identification in radio and television equipment rooms. This algorithm, which deeply integrates computer vision and artificial intelligence technologies, equips radio and television room inspection robots with autonomous perception, analysis, and decision-making capabilities, effectively replacing manual inspection and maintenance tasks within the room and improving operational efficiency and safety. The robot, equipped with this patented algorithm, can operate stably in complex equipment room environments. Equipped with infrared and high-definition cameras, it captures multispectral, high-precision images to meet detection requirements under varying lighting conditions. Furthermore, the robot is equipped with a high-precision robotic arm, enabling routine maintenance tasks such as equipment operation and button debugging, further expanding its application scenarios. Regarding data processing, this invention builds a high-quality dataset and trains a deep learning model on the collected image data to enhance the robot's understanding of the room environment. The algorithm utilizes advanced object detection and image classification techniques, enabling the radio and television room inspection robot to accurately identify and classify various radio and television facilities and provide real-time feedback on detection results.

[0023] Furthermore, an algorithm for using a robot based on visual understanding to detect the environment and identify facilities in a radio and television room includes the following steps:

[0024] 1) Information collection.

[0025] During the operation of the radio and television transmitter room, the radio and television room auxiliary inspection robot relies on its onboard infrared camera and high-definition camera to obtain real-time image data of the environment in front, and uses it as the input of the recognition network model. Through matching and fusion analysis of infrared images and high-definition images, the radio and television room auxiliary inspection robot can accurately judge the category of the objects encountered, and adjust and optimize its own movement strategy based on the recognition results. This visual perception and intelligent decision-making mechanism enables the robot to have autonomous environmental perception capabilities, and can judge in real time whether there are obstacles in front, and further identify the type of obstacles. For different types of obstacles, the radio and television room auxiliary inspection robot can execute corresponding obstacle avoidance or detour strategies to ensure the smooth progress of the inspection task. When encountering obstacles during the inspection process, the robot can not only accurately identify their categories, but also take appropriate countermeasures according to the specific situation, thereby improving the autonomous operation capability in the complex radio and television room environment. The specific implementation framework is as follows: Figure 1 shown.

[0026] The broadcasting and television room inspection robot system combines infrared and conventional cameras to accurately distinguish between living objects (such as staff) and inanimate objects (such as broadcasting and television equipment). This capability enables the robot to identify both dynamic and static objects in the room environment, enabling it to adapt to different types of obstacles. For example, if the robot detects moving objects such as pedestrians or animals, it can adjust its path or slow down to avoid them. Static obstacles such as cabinets, walls, or broadcasting equipment can be avoided by detouring or optimizing its path.

[0027] In practical applications, this feature not only significantly enhances the autonomy of the auxiliary inspection robot in the radio and television room, enabling it to more intelligently perceive and adapt to the complex room environment, but also strengthens its obstacle avoidance capabilities, ensuring safety and efficiency in various working scenarios. Especially in environments with limited space and dense equipment, such as radio and television rooms, this technology helps the robot maneuver flexibly, avoid collisions, and improve the stability and reliability of task execution.

[0028] 2) Visual understanding.

[0029] The auxiliary inspection robot in the radio and television room uses the complementary characteristics of infrared images and high-definition visible light images to pair, match and analyze the two types of images captured simultaneously, thereby extracting useful information and features, and realizing the precise positioning and identification of objects in the room environment. The core principle of this method is to integrate the advantages of infrared and visible light images to enhance target detection and recognition capabilities, and provide more reliable technical support for intelligent operation and maintenance in complex environments. Infrared images are mainly used to detect the thermal energy distribution of objects, and can clearly present the temperature characteristics of the target, and are suitable for low-light or light-limited scenes. High-definition visible light images provide detailed information such as the shape, texture and color of the object, allowing the robot to identify the appearance characteristics of the object. Through the joint analysis of the two images, the robot can achieve more comprehensive target perception and effectively improve the accuracy of recognition and positioning. In addition, combined with deep learning recognition algorithms, the robot can classify and fuse information for targets in infrared and visible light images, thereby improving recognition accuracy. This algorithm can not only accurately identify and locate various types of radio and television room facilities, but also assist the robot in performing obstacle avoidance tasks in complex environments, ensuring the stability and safety of its operation. The process of the overall solution is as follows Figure 2 shown.

[0030] The infrared and visible light image matching and recognition method is a technology that comprehensively utilizes multispectral information for target recognition and environmental perception. It mainly includes the following key steps:

[0031] Image alignment and registration. In a radio and television room environment, an auxiliary inspection robot needs to acquire infrared and visible light images from different sensors. Because these two types of images differ in imaging mechanism, field of view, and resolution, precise alignment and registration are essential to ensure they share a consistent spatial reference. Typically, methods based on feature point matching, image transformation, or deep learning are used to achieve high-precision registration of the two images, ensuring the accuracy of subsequent feature extraction and fusion.

[0032] Feature extraction. After registration, key visual features are extracted from the infrared and visible light images. Infrared images primarily reflect the thermal energy distribution of an object and are suitable for target detection in low-light environments. Visible light images, on the other hand, contain information such as the object's shape, texture, and color, providing rich details and structural features. To improve recognition accuracy, it is necessary to combine edge detection, shape analysis, and deep learning feature extraction methods to obtain multidimensional features that aid in target recognition.

[0033] Feature fusion and matching. After acquiring infrared and visible light features, feature fusion and matching techniques are used to comprehensively analyze the information from the two image types. Common methods include weighted fusion, feature-level fusion, and deep neural network fusion. The fused data can more comprehensively describe the spatial structure and temperature characteristics of the target object, thereby improving the stability and accuracy of target recognition. In addition, precise cross-modal feature matching is achieved through the calculation of similarity metrics and neural network matching techniques.

[0034] Target recognition and processing. Based on the results of fusion matching, targets are identified, classified, and their status analyzed. Specifically, the radio and television room auxiliary inspection robot can accurately determine the target type (such as equipment, obstacles, fault points, etc.), location, and operating status based on the target's thermal distribution characteristics, geometry, and texture information. Furthermore, the recognition results can be used to formulate corresponding robot action strategies, such as obstacle avoidance, inspection, alarm, or execution of specific maintenance tasks, thereby improving the robot's autonomous decision-making capabilities and operational efficiency.

[0035] By matching infrared and visible light for identification and classification, the radio and television room inspection robot can more accurately perceive and understand the room environment. This not only improves its autonomous navigation capabilities within the complex radio and television room environment, but also enhances its intelligent operation capabilities, enabling it to efficiently complete tasks such as equipment inspection and fault detection, providing important technical support for the intelligent management of radio and television rooms.

[0036] 3) Environmental monitoring.

[0037] In deep learning models, as network depth increases, the structure becomes more complex, enabling the model to have stronger feature extraction capabilities and higher target recognition accuracy. However, the computational cost of deep networks also increases, requiring not only greater computing power but also significantly increasing the demand for hardware resources. For example, deep models often involve a large number of parameter calculations and floating-point operations, which not only increases the computational burden during inference but also causes the resulting model weight files to occupy a large amount of storage space. Therefore, in actual deployment, high-performance computing hardware, such as a powerful CPU, sufficient RAM, and ample storage space, is required to ensure stable and efficient model operation. Considering the actual application requirements in radio and television room environments, radio and television room auxiliary inspection robots need to perform environmental detection and facility identification tasks while also possessing excellent real-time performance and automated obstacle avoidance capabilities.

[0038] Therefore, in order to reduce the computing cost while ensuring detection accuracy, and to improve the inference speed and overall operating efficiency of the model, this study designed a lightweight deep learning model, DeloV1, based on the coordinate attention mechanism. This model uses an optimized neural network architecture to reduce the occupation of computing resources while maintaining a high recognition accuracy. Compared with traditional deep learning models, DeloV1 has a smaller number of parameters and lower storage occupancy, enabling it to run efficiently on embedded devices or robot platforms with limited computing resources. In addition, the model has a faster detection speed in the complex environment of the radio and television room, and can process multimodal image data collected by infrared cameras and high-definition cameras in real time to achieve accurate environmental perception and facility recognition. Experimental results show that DeloV1 significantly improves the inference speed and system responsiveness while ensuring high detection accuracy, enabling robots to perform tasks more efficiently. The architecture of the coordinate attention mechanism is as follows: Figure 3 As shown, the model architecture of DeloV1 is as follows Figure 4 shown.

[0039] 4) Facility identification.

[0040] This study uses the DeloV1 model to implement intelligent identification of facilities in radio and television room inspection robots. The overall process consists of three core steps: data preprocessing, network training, and detection result output. To improve the model's detection accuracy, this paper introduces additional data augmentation strategies during the data preprocessing stage to address issues such as poor image quality and blurred object boundaries caused by infrared imaging. These issues often affect detection accuracy. Therefore, we use customized data augmentation algorithms for specific object categories in the image to enhance the object's feature representation and improve the final detection accuracy. During the network training stage, we selected DeloV1 as the core deep learning model. DeloV1 mainly consists of four modules: input layer, backbone network, bottleneck layer, and object prediction layer. The input layer is responsible for receiving and processing input data. To mitigate the negative impact of small sample datasets on model training stability, we introduce a cross-domain transfer learning strategy at the input layer. This strategy leverages previously trained domain knowledge, enabling the model to learn efficiently even with limited data, improving training stability and generalization. The backbone network is responsible for extracting deep-level features of the image and achieving step-by-step refinement and enhancement of features through multi-layer convolution calculations; the bottleneck layer further compresses and filters important feature information to reduce redundant features and improve the computational efficiency of the model; the target prediction layer is used for final target detection and classification to ensure accurate identification and positioning of facility targets.

[0041] Cross-domain transfer learning is a method that leverages knowledge gained from training on large-scale datasets to improve performance on smaller datasets. In this study, cross-domain transfer learning involves transferring the network weights of an object detection model trained on a visible light image dataset containing a large number of samples and a rich variety of object categories to the infrared target detection task, which contains only a small number of samples and a limited number of target categories. Specifically, this method first trains an object detection model on a large-scale visible light image dataset, enabling it to learn rich feature representations, marginal probability distributions, and optimized network parameters. Subsequently, these trained model weights are used as pre-trained parameters and further fine-tuned on a small-scale infrared dataset to adapt to the specific characteristics of infrared images, such as illumination invariance and thermal radiation characteristics. The core advantage of this transfer learning method is that it fully leverages the generalization capabilities of large-scale datasets to extract more robust feature representations, effectively improving detection performance on small-sample infrared datasets. During the training of the DeloV1 object detection model, this method not only helped the model converge faster but also enhanced its generalization capabilities in infrared target detection tasks, thereby improving detection accuracy. Figure 5 The implementation and effect of this transfer learning process are intuitively demonstrated. Compared with the existing technology, the present invention has the following advantages:

[0042] Compared with existing technologies, the present invention adopts a cross-domain transfer learning method to transfer the target detection model trained on a large-scale visible light image dataset to the small-sample infrared target detection task, and improves its generalization ability and detection accuracy through retraining. Compared with traditional methods, the present invention fully utilizes the rich feature space, edge probability, and optimized network parameters in large-scale datasets to enable the model to obtain more robust feature representations on small-sample infrared datasets, effectively reducing the risk of overfitting and improving model stability and adaptability. At the same time, with the help of pre-trained models, the present invention can accelerate the training convergence speed, significantly reduce the dependence on large-scale infrared annotated data, and improve training efficiency.

[0043] Furthermore, this method demonstrates superior accuracy and reliability in infrared target detection tasks, overcoming the generalization issues inherent in small sample datasets. Through transfer learning, the model, after fine-tuning on infrared data, can more accurately identify targets and enhance detection results. This method is not only applicable to auxiliary inspection robots in radio and television rooms, but can also be widely applied in various scenarios, including nighttime target detection and industrial inspection, demonstrating its strong practical value and promising prospects for widespread adoption. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 Specific implementation framework for information collection;

[0045] Figure 2 To visually understand the overall solution process;

[0046] Figure 3 This is the coordinate attention mechanism architecture;

[0047] Figure 4 This is the model architecture of DeloV1;

[0048] Figure 5 For the transfer learning process;

[0049] Figure 6 It is an infrared image detection model;

[0050] Figure 7 Specific process for facility-specific identification algorithms;

[0051] Figure 8 It is a variation trend diagram of the total loss of the present invention. DETAILED DESCRIPTION

[0052] The following further describes an algorithm for using a robot based on visual understanding to detect the environment and identify facilities in a radio and television room in conjunction with the accompanying drawings.

[0053] 1. Radio and TV room environmental detection

[0054] The Radio and Television Room Assisted Inspection Robot performs data collection tasks in environments such as radio and television rooms, primarily responsible for recording and storing captured infrared and visible light images. This data provides the foundation for subsequent intelligent analysis and model training.

[0055] During the operation, the robot based on visual understanding collects the image data in front of it in real time through the camera and uses it as the input of the recognition network model. By matching the images of the infrared camera with the visible light camera, the robot can identify the type of object in front of it and feed the recognition results back to the motion control system. This mechanism enables the robot to independently determine whether there are obstacles in front of it and their types, and perform corresponding obstacle avoidance actions accordingly. When encountering obstacles during the inspection process, the robot can accurately identify the type of obstacle and adopt corresponding processing strategies according to different situations. The overall implementation process is as follows: Figure 1 shown.

[0056] To improve data usability and accuracy, the researchers first used the visual image annotation tool LABELIMG to annotate the collected image data, adding appropriate labels to build a high-quality dataset. Subsequently, they trained a deep learning model based on this annotated dataset to improve the accuracy of infrared image detection. After training, the research team validated and optimized the model's detection performance to ensure its stability and reliability in practical applications.

[0057] In complex environment inspection tasks, single-modality image information often fails to meet the requirements for accurate obstacle detection and identification. Infrared images have excellent anti-interference capabilities and can stably detect targets in low light, smoke, and haze conditions, but their ability to represent detailed information is limited. Visible light images excel in extracting detailed features such as texture, shape, and color, but are susceptible to changes in ambient lighting. Therefore, fusing infrared and visible light images can leverage their complementary nature in feature perception, improving the environmental adaptability and perception accuracy of inspection robots, and providing reliable data support for subsequent autonomous navigation and obstacle avoidance. Through infrared and visible light image matching, inspection robots can pair and compare simultaneously acquired infrared and visible light images to extract key feature information and accurately locate and identify objects in the environment. This method leverages the complementary nature of infrared and visible light images to enhance the overall performance of target detection and recognition. Specifically, infrared images primarily reflect the thermal energy distribution characteristics of an object and can effectively detect its thermal radiation information, while visible light images provide detailed appearance information such as shape, texture, and color. By jointly registering and matching the two types of image data, multimodal perception of the target can be achieved, significantly improving the accuracy of recognition and positioning. In addition, combined with deep learning recognition algorithms, the features of infrared images and visible light images are classified and fused, further optimizing the collaborative recognition effect of multi-source information. Based on the above image matching and fusion methods, the inspection robot can effectively achieve obstacle avoidance and improve its autonomous navigation capabilities in complex environments. The overall algorithm process is as follows: Figure 2 shown.

[0058] The second step is to organize the annotated infrared images and convert them into the VOC dataset format for subsequent model training and testing. During training, the system dynamically records the algorithm's performance metrics (such as loss, precision, and recall) to analyze model convergence and optimization directions. During the initial stages of model training, relevant parameters must be appropriately set to ensure training stability and convergence speed. Specific parameters include the network's initial learning rate, weight decay coefficient, momentum coefficient, and number of iterations. To address the complex and dynamically changing environmental characteristics of the optoelectronic room, this paper innovatively proposes a method that deeply integrates the lightweight object detection network YOLOv5s with a robot path planning system. By invoking the YOLOv5s model in real time during inspections, the robot can quickly identify equipment, obstacles, and abnormal conditions within the room. This allows the robot to not only perform preliminary path planning based on a static map but also implement online path adjustment and optimization based on YOLOv5s' dynamic detection results. Unlike traditional path planning that relies solely on hard-coded obstacle avoidance in sensors, this method leverages YOLOv5s's efficient detection capabilities for small targets and obstructions in the machine room, building an intelligent inspection framework that integrates perception and decision-making. This significantly improves the robot's autonomy, flexibility, and safety in complex and narrow spaces, providing a more efficient and reliable solution for intelligent inspections in optoelectronic rooms. The overall architecture is as follows: Figure 3 shown.

[0059] In view of the characteristics of dense equipment, complex obstacles and diverse information modalities in the radio and television room environment, this paper innovatively introduces the attention mechanism into the path planning module and constructs a perception-decision coupling architecture that integrates channel and spatial dual attention (such as Figure 4 ). The structure first extracts channel-level feature weights through maximum pooling and average pooling, and then dynamically generates channel attention weights through two layers of fully connected networks and Sigmoid functions. At the same time, the spatial attention module further extracts spatially significant regions through pooling, feature splicing, and convolution operations. This dual attention mechanism can effectively enhance the perception and response of key obstacles, traversable areas, and key equipment during the path planning process, thereby achieving local enhancement and global optimization of path planning. Compared with traditional path planning algorithms that rely solely on raw visual features for path calculation, this method enables the robot to dynamically adjust its planning strategy based on the weight of the environment's semantic features, significantly improving the robot's autonomous obstacle avoidance capability and planning robustness in complex optoelectronic room scenarios.

[0060] To validate the effectiveness of the proposed attention-integrated path planning method in complex equipment room environments, this paper conducted comparative experiments comparing the robot's inspection performance before and after the introduction of the attention mechanism. Multiple rounds of testing in a standard broadcasting and electrical equipment room environment revealed that, after introducing the attention mechanism, the robot's detection accuracy for narrow passages and low-contrast obstacles increased from 84.7% to 92.3%, the average path planning time decreased by approximately 18.5%, and the obstacle avoidance success rate increased by nearly 12 percentage points. Traditional methods tend to overlook small obstacles or blind spots, particularly in equipment-intensive areas and low-light conditions. However, the YOLOv5s perception model, which incorporates both channel and spatial attention modules, significantly enhances the response to critical areas, effectively improving the rationality and safety of the robot's path selection. These experimental results fully demonstrate the superiority of this method in typical scenarios, such as broadcasting and electrical equipment rooms, and provide strong technical support for achieving highly reliable autonomous robot inspections.

[0061] The third step, cross-domain transfer learning, involves applying network weights obtained from training on a large-scale dataset covering multiple categories (e.g., 80 object classes) to a specific task trained on a smaller dataset with a limited number of classes (typically one or two). In the optoelectronics room visual understanding robot system, cross-domain transfer learning effectively transfers deep learning models used for visible light image object detection to infrared object detection. The core idea behind this process is that despite significant differences in visual features between the source task (e.g., visible light object detection) and the target task (e.g., infrared object detection), transfer learning techniques can still effectively initialize the target task with the common features and model knowledge learned from the source task, thereby improving detection performance on the target task even with limited labeled data. Specifically, in cross-domain transfer learning, a network is first trained on a large-scale visible light image dataset to obtain a robust network model that performs well on multi-category object detection. The weights of this network are then transferred to infrared image object detection, a task with a smaller dataset and fewer classes. By retraining or fine-tuning the transferred network, the model is adapted to the new target task, effectively improving object detection accuracy in infrared images. like Figure 5 As shown in Figure 2, cross-domain transfer learning can not only reduce the demand for labeled data but also improve training efficiency and detection effect by connecting the visible light target detection model with the infrared target detection task.

[0062] The fourth step is to improve the inspection robot's target recognition and detection capabilities under different environmental conditions. This paper conducts data collection experiments outdoors and under various complex working conditions, covering typical scenarios such as day and night, strong light interference, low illumination, and complex backgrounds. During the execution of the task, the robot collects and records infrared image and visible light image data respectively, and saves them synchronously as standard image data sets. To ensure the accuracy and practicality of model training, the open source annotation tool LabelImg is used to finely annotate the collected images, manually add target bounding boxes and category labels, and construct a multimodal image sample library with semantic information. Subsequently, the infrared image target detection model is trained and verified based on the constructed dataset. The model uses lightweight detection networks such as YOLOv5s to efficiently identify obstacles, key equipment, and traffic areas in the environment. The architecture and workflow of the entire infrared image detection system are as follows. Figure 6 As shown in the figure, it covers key modules such as image input, preprocessing, label training, model inference and recognition result visualization.

[0063] The fifth step is to analyze the model's performance on the test set by setting different confidence thresholds. Specifically, the image matching recognition detection model used in this article will have varying precision and recall under different confidence thresholds. A higher confidence threshold generally improves the model's precision but may reduce recall, while a lower confidence threshold may improve recall but reduce precision. Therefore, to comprehensively evaluate the model's detection capabilities, we selected multiple confidence thresholds and recorded the corresponding precision and recall to observe how the model's performance changes under different conditions.

[0064] 2. Identification of radio and television room facilities

[0065] The specific process of the identification algorithm of the auxiliary inspection robot in the radio and television room for facilities is as follows Figure 7 This algorithm uses an obstacle avoidance method based on infrared and visible light image matching and recognition to ensure the robot's efficient and safe operation within the radio and television equipment room. The core of this method is to use an industrial camera to capture environmental images and simultaneously import infrared and regular video streams for real-time analysis and target recognition.

[0066] During the recognition process, the algorithm accurately distinguishes between different types of target objects. When a person is detected, the system clearly marks their location, allowing the robot to plan its path appropriately and avoid collisions or misjudgments. When a broadcast and television equipment facility (such as a server cabinet, distribution box, or cable rack) is detected, the system also accurately marks its location, ensuring the robot does not interfere with or damage these critical devices as it moves within the room. In the broadcast and television room robot's facility recognition algorithm, conventional cameras and infrared cameras work together to form a dual-modal detection mechanism to improve recognition accuracy and reliability. Conventional cameras primarily capture visible light images to identify people, equipment, and other environmental features, while infrared cameras focus on temperature information, enabling them to identify heat sources, particularly living organisms. When the conventional camera identifies an image suspected of being human and the infrared camera simultaneously detects a warning signal (indicating a temperature anomaly in the detection area, possibly due to human thermal radiation), the system determines the target to be a living organism. In this case, the robot system immediately sends feedback to the computer, including an alarm or tone, to enable a quick response from the operator or control system. At the same time, the robot will take corresponding avoidance measures based on the detection results, such as adjusting the movement path, bypassing obstacles, or suspending operation when necessary to ensure safety and stability in complex environments.

[0067] If the detected target is a living organism, such as a person or animal, and is moving, the system can choose to continue forward without taking additional obstacle avoidance measures. This is because these organisms may simply be passing by or performing normal activities, such as walking, running, or briefly stopping, and typically do not pose a persistent obstacle to the route of the radio and television room inspection robot. However, to enhance safety, the system can monitor the movement trajectory and behavior patterns of these objects in real time to ensure they are truly in motion and do not suddenly stop or change direction, creating a potential collision risk. On the other hand, if the system detects that the target remains stationary for an extended period of time, or its movement trends indicate a potential fixed obstacle, such as a person remaining in one location for an extended period, sitting down, or an animal lying motionless on the ground, the system needs to take appropriate obstacle avoidance measures to prevent a collision with the radio and television room inspection robot. These measures may include replanning the path, adjusting the travel speed, or, if necessary, pausing until the target moves or its status is determined. Furthermore, the system can incorporate other sensor data, such as thermal imaging or depth perception technology, to further confirm whether the target is truly stationary, enabling the radio and television room inspection robot to make more intelligent decisions in complex environments.

[0068] On the other hand, when the target detected by a conventional camera is a non-living object (such as a camera, audio equipment, or other stationary object) and the infrared camera does not detect an abnormal temperature signal, the system will use another feedback mechanism for processing. Specifically, if the target object is confirmed to be inanimate and not emitting significant heat, the system will combine environmental perception information to determine whether it poses a potential threat to the device's operation. For example, during navigation, if a machine is detected in front of it and the object does not have an abnormal temperature, the system will identify it as a standard static obstacle and provide feedback based on a pre-set obstacle avoidance strategy. Furthermore, if the system identifies a stationary object with a temperature (such as an operating projector or other object emitting heat), it will not only provide appropriate feedback but also perform obstacle avoidance operations to ensure safety and stability. Obstacle avoidance strategies may include adjusting the travel path, slowing down, or rerouting to effectively avoid potential collision risks. This process ensures that the device can independently judge and respond appropriately in complex environments, thereby enhancing its intelligent perception and decision-making capabilities.

[0069] During the identification process of the radio and television room auxiliary inspection robot, the system will identify and respond accordingly when the robot encounters different types of obstacles. Specifically, when a living organism appears in front of the robot, the host computer receives a real-time alarm signal and plays a specific alarm sound to alert the operator to the possible abnormality. Conversely, if the obstacle detected by the robot is a non-living object, such as a wall, equipment, or other stationary object, the system will issue a different type of alarm sound to distinguish the identification result. In addition, to further reduce false alarms and avoid unnecessary intervention, the system has also implemented a special strategy. If neither the standard camera captures the signal characteristics of the target object nor the infrared camera detects abnormal temperature changes in the image stream collected by the radio and television room auxiliary inspection robot, the system will temporarily suspend feedback and enter a standby state, awaiting subsequent operational instructions. This strategy effectively reduces the false alarm rate, minimizes excessive system intervention in the environment, and improves the robot's intelligence level.

[0070] Robot recognition test process is as follows Figure 7As shown in the figure, the inspection robot's obstacle avoidance method utilizes matching and recognition technology based on infrared and visible light images. This method uses an industrial camera to capture real-time environmental images, synchronously imports infrared and standard video streams, and performs real-time analysis. By processing and fusing the acquired images, the robot can accurately identify different types of targets. Specifically, when the detected target is a device, the system identifies and marks the device's location. When encountering an obstacle, the robot provides timely warnings and prompts for obstacle avoidance. If a fault point or abnormality is detected, the system provides further guidance to the operator through various display prompts. Through this real-time analysis and recognition process, the inspection robot can effectively avoid obstacles and ensure the smooth progress of inspection tasks.

[0071] During the training process, the labeled infrared images are first made into a data set and used for model training and testing. During each training, the performance indicators of the algorithm are dynamically recorded for subsequent analysis. As the number of iterations increases, the loss value of the model gradually changes, and the trend of its total loss is as follows: Figure 8 shown.

[0072] The initial model training parameters were set as follows: the network's initial learning rate was 0.002, the weight decay coefficient was 0.001, the momentum coefficient was 1, and the number of iterations was set to 400. For the radio and television room environment detection and facility identification algorithm, the loss value was approximately 0.15 at the beginning of training, decreasing rapidly. After 50 generations of training, the loss value dropped to around 0.04, and the rate of loss decrease gradually slowed as training progressed. By the 400th generation, the loss value had further decreased to around 0.02, indicating that the model training had stabilized.

[0073] After model training, performance evaluation of the dataset demonstrated that the system demonstrated excellent target recognition performance. This provided reliable data support for robot navigation, ensuring efficient and accurate inspection tasks. Furthermore, during the testing phase, different confidence thresholds were set to evaluate the model's precision and recall. Different confidence thresholds affect the model's detection performance. By selecting multiple confidence thresholds, corresponding precision and recall can be obtained, which helps optimize model performance in practical applications, thereby providing more accurate target recognition results.

[0074] The above is merely an illustration of the present invention and is not intended to limit the present invention. Those skilled in the art should recognize that any changes or modifications made to the present invention will fall within the scope of protection of the present invention.

Claims

1. A method for detecting the environment and identifying facilities in a radio and television room, characterized in that: The following steps are involved: 1) The execution equipment acquires real-time infrared images and high-definition images in a broadcast and television room environment; 2) Read temperature information from infrared images and use visual understanding technology to identify targets in infrared images and high-definition images; In step 2), visual understanding technology is used to identify targets in infrared images and high-definition images, including: 2.1) Align and register the infrared image and the high-definition image; 2.2) Extract features from the same regions in the registered infrared image and the registered high-definition image to obtain key visual features of the infrared image and key visual features of the visible light image; 2.3) The infrared key visual features and the visible light key visual features are combined through feature fusion and feature matching to obtain matching results; 2.4) Perform target identification, classification and status analysis on the matching results to obtain the target, target location and target working status in the broadcasting and television working environment in real time.

2. The method for radio and television room environment detection and facility identification according to claim 1, characterized in that: In step 1), the infrared image is obtained by an infrared camera on the execution device.

3. The method for radio and television room environment detection and facility identification according to claim 1, characterized in that: In step 1), the high-definition image is obtained by a high-definition camera on the execution device.

4. The method for radio and television room environment detection and facility identification according to claim 1, characterized in that: In step 2.3), feature fusion adopts weighted fusion, feature level fusion or deep neural network fusion.

5. The method for radio and television room environment detection and facility identification according to claim 1, characterized in that: In step 2.3), feature matching uses similarity metric matching or neural network matching.

6. The method for radio and television room environment detection and facility identification according to claim 1, characterized in that: In step 2.4), the targets are equipment, obstacles and fault points.

Images