Infrared-vision cooperative tracking navigation method applied to mobile robot and related device

Abstract

The invention discloses an infrared-vision cooperative tracking navigation method applied to a mobile robot and a related device, and relates to the field of robot navigation. Judging whether the infrared tracking is in a normal state or a failure state: in the normal state, generating an infrared tracking motion control signal in combination with a path surface reflected light intensity analog signal and a PID control algorithm so as to control the mobile robot to perform tracking motion; and in a failure state, starting a top camera to acquire a path image, and generating a visual tracking motion control signal in combination with the path surface image and a PID control algorithm so as to control the mobile robot to perform tracking motion. Through automatic conversion and synergistic effect of infrared and visual tracking, stable and accurate tracking navigation is realized, and the problem that tracking stability and accuracy are difficult to consider in the current tracking navigation field is solved.

Description

Technical Field

[0001] This application relates to the field of robot navigation, and in particular to an infrared-visual cooperative line-following navigation method and related apparatus for mobile robots. Background Technology

[0002] Mobile robots, as an important branch of robotics, are currently widely used in industrial automation, intelligent logistics, healthcare, search and rescue, and other fields, significantly improving the efficiency and convenience of production processes. To achieve autonomous movement, mobile robots rely on navigation systems for real-time positioning and motion planning. Using ground-marked lines as a motion reference to guide the mobile robot is a low-cost and easy-to-implement line-following navigation method. This method is suitable for scenarios with stable environmental conditions, fixed routes, and the need for high-precision navigation, such as automated logistics warehouses, large-scale supermarkets, automobile assembly workshops, and path guidance for indoor medical service robots.

[0003] Currently, infrared tracking using infrared sensor arrays to identify ground markings works on the core principle of emitting infrared light of a specific wavelength onto the ground using infrared emitters and detecting differences in the intensity of reflected light using infrared receivers to identify the path. When infrared light shines on a white surface, most of the light is reflected, and the infrared sensor outputs a low level. When infrared light shines on a dark marking line, much of the light is absorbed, resulting in a weak reflected signal, and the infrared sensor outputs a high level. By analyzing the output levels of multiple infrared sensors in real time, the control system can determine the positional deviation of the mobile robot relative to the ground marking line. Then, using PID control algorithms, it drives the robot's motors to correct its direction, achieving precise path tracking.

[0004] However, infrared sensors have significant drawbacks: they are susceptible to interference from ambient light. In strong light, the infrared components in sunlight can cause the infrared sensor to "go blind," resulting in tracking failure. Furthermore, as the usage time increases, ground markings may become damaged or contaminated. The threshold-based judgment method of infrared sensors cannot robustly handle such discontinuous or blurred ground markings, leading to a significant increase in the number of tracking failures. Summary of the Invention

[0005] The purpose of this application is to provide an infrared-vision cooperative line-following navigation method and related device for mobile robots. By combining infrared and visual line-following methods, it achieves a balance between high precision and high stability in line-following navigation, thus solving the problem that precision and stability cannot be achieved simultaneously in the current field of line-following navigation.

[0006] To achieve the above objectives, this application provides the following solution: In a first aspect, this application provides an infrared-vision cooperative line-following navigation method for mobile robots, comprising: The simulated signal of the intensity of reflected light from the path surface is acquired by each infrared sensor in the infrared sensor array; the infrared sensor array is set at the bottom of the mobile robot.

[0007] Based on the simulated signals of reflected light intensity from all the paths, the infrared tracking state is determined; the infrared tracking state is either a normal state or a failure state.

[0008] When the infrared tracking state is in a failed state, the path surface image captured by the camera is acquired, and based on the path surface image and the PID control algorithm, a visual tracking motion control signal is determined; the visual tracking motion control signal is used to control the mobile robot to perform tracking motion; the camera is set on the top of the mobile robot.

[0009] When the infrared tracking state is normal, the infrared tracking motion control signal is determined based on the simulated signals of reflected light intensity from all the path surfaces and the PID control algorithm; the infrared tracking motion control signal is used to control the mobile robot to perform tracking motion.

[0010] Secondly, this application provides a mobile robot device, comprising: The mobile robot, the control module mounted on the mobile robot, the camera mounted on the top of the mobile robot, and the infrared sensor array mounted on the bottom of the mobile robot.

[0011] The infrared sensor array includes multiple infrared emitters and an infrared sensor associated with each of the infrared emitters.

[0012] The infrared sensor is used to collect simulated signals of the intensity of reflected light from the path surface.

[0013] The camera is used to acquire images of the path surface.

[0014] The control module is used to execute one of the infrared-visual cooperative line-following navigation methods for mobile robots described in any of the above-mentioned methods.

[0015] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the infrared-visual cooperative line-following navigation method for a mobile robot as described in any one of the above-mentioned methods.

[0016] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the infrared-visual cooperative line-following navigation method for a mobile robot described in any one of the above-mentioned methods.

[0017] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the infrared-visual cooperative line-following navigation method for a mobile robot described in any of the above-mentioned embodiments.

[0018] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides an infrared-vision cooperative line-following navigation method and related apparatus for mobile robots. It uses an infrared sensor array on the bottom of the mobile robot to collect simulated signals of reflected light intensity from the path surface to determine the infrared line-following status, solving the navigation interruption problem caused by ambient light interference or path damage in infrared line-following mode. It achieves automatic mode switching and maintains overall stability when the infrared line-following mode experiences partial failure. By combining path surface images acquired by a camera with the control algorithm, it solves the problems of unstable line recognition and unreliable position information when the path is discontinuous in a single infrared line-following mode, achieving stable line recognition and reliable position information output, compensating for the poor stability of the infrared line-following mode. Through the infrared-vision cooperative navigation mechanism, it solves the problem that a single navigation mode cannot simultaneously achieve both accuracy and stability, achieving a dynamic balance between high accuracy and high stability. This allows the robot to prioritize the use of the high-precision infrared line-following mode when it is available, and seamlessly switch to visual line-following mode for continuous operation when the infrared line-following mode fails. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is an application environment diagram of an infrared-visual cooperative line-following navigation method for a mobile robot according to an embodiment of this application; Figure 2 This is a flowchart illustrating an infrared-visual cooperative line-following navigation method for a mobile robot according to an embodiment of this application. Figure 3 for Figure 2 A detailed flowchart illustrating the steps involved in determining visual tracking motion control signals; Figure 4 This application provides a schematic diagram illustrating the implementation principle of two tracking modes in an infrared-vision cooperative tracking navigation method for mobile robots, as provided in one embodiment of the present application. Figure 5 This is a schematic diagram showing the positions of various devices in a mobile robot device according to an embodiment of this application; Figure 6 A flowchart illustrating the operation of a mobile robot device according to an embodiment of this application; Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] Currently, in the field of line-following navigation, infrared line-following navigation achieves accurate line-following but suffers from poor stability, while visual line-following navigation achieves high stability but poor accuracy. Recognizing that the advantages and disadvantages of these two approaches are complementary, this application aims to propose an infrared-visual collaborative line-following navigation method and related device. By fusing these two navigation methods, it achieves stable and accurate line-following navigation, solving the current problem of the inability to simultaneously achieve both in the field of line-following navigation.

[0023] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0024] This application provides an infrared-vision cooperative line-following navigation method for mobile robots, which can be applied to, for example... Figure 1The flowchart shown illustrates the application environment. Terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be set up independently, integrated into server 104, or placed in the cloud or on another server. Terminal 102 can send simulated signals of path surface reflected light intensity collected by each infrared sensor in the infrared sensor array or path surface images collected by the camera to server 104. Server 104 determines the infrared tracking state based on all received simulated signals of path surface reflected light intensity; the infrared tracking state is either normal or ineffective. When the infrared tracking state is ineffective, a visual tracking motion control signal is determined based on the received path surface images and a PID control algorithm. When the infrared tracking state is normal, an infrared tracking motion control signal is determined based on all simulated signals of path surface reflected light intensity and the PID control algorithm. Server 104 can feed back the infrared tracking motion control signal or the visual tracking motion control signal to terminal 102. Furthermore, in some embodiments, the infrared-visual cooperative tracking navigation method applied to mobile robots can also be implemented separately by the server 104 or the terminal 102. For example, the terminal 102 can directly process the simulated signal of the reflected light intensity of the path surface collected by each infrared sensor in the infrared sensor array or the path surface image collected by the camera. Alternatively, the server 104 can obtain the simulated signal of the reflected light intensity of the path surface collected by each infrared sensor in the infrared sensor array or the path surface image collected by the camera from the data storage system, and process the simulated signal of the reflected light intensity of the path surface collected by each infrared sensor in the infrared sensor array or the path surface image collected by the camera.

[0025] Among them, terminal 102 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, etc., and server 104 can be implemented by independent servers or server clusters composed of multiple servers, or it can be a cloud server.

[0026] In one exemplary embodiment, such as Figure 2 As shown, an infrared-vision cooperative line-following navigation method for mobile robots is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method is applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps 201 to 204.

[0027] Step 201: Obtain the simulated signal of the intensity of reflected light from the path surface collected by each infrared sensor in the infrared sensor array.

[0028] Step 202: Determine the infrared tracking state based on the simulated signals of reflected light intensity from all the path surfaces; wherein the infrared tracking state is a normal state or a failure state.

[0029] Step 203: When the infrared tracking state is in a failed state, acquire the path surface image collected by the camera, and determine the visual tracking motion control signal based on the path surface image and the PID control algorithm; the visual tracking motion control signal is used to control the mobile robot to perform tracking motion.

[0030] Step 204: When the infrared tracking state is normal, the infrared tracking motion control signal is determined based on the simulated signals of reflected light intensity from all the path surfaces and the PID control algorithm; the infrared tracking motion control signal is used to control the mobile robot to perform tracking motion.

[0031] The infrared sensor array is located at the bottom of the mobile robot, and the camera is located at the top of the mobile robot.

[0032] In another exemplary embodiment of this application, an infrared-vision cooperative line-following navigation method for a mobile robot is provided. This method is executed by a core control unit, achieving the cooperative effect of infrared and visual line-following modes through steps 201 to 204. The core control unit in this embodiment uses NVIDIA's Jetson OrinNano developer kit, each infrared sensor in the infrared sensor array is a three-way line-following infrared sensor from Orbbec, the camera is an Orbbec Gemini 335 camera, and the mobile robot is Orbbec's JetAuto intelligent mobile robot. The core control unit has abundant interface resources, including USB, UART, and IIC interfaces. The infrared sensors are connected to the mobile robot via the UART interface, and the camera is connected to the mobile robot via the USB interface.

[0033] In another exemplary embodiment of this application, the infrared sensor array includes: a left-turn infrared sensor array, a straight-line infrared sensor array, and a right-turn infrared sensor array. Step 202 specifically includes: first, preprocessing the simulated signal of reflected light intensity from each path surface to obtain a binary array corresponding to the left-turn infrared sensor array, the straight-line infrared sensor array, and the right-turn infrared sensor array; second, determining the infrared tracking state based on the binary arrays corresponding to the left-turn infrared sensor array, the straight-line infrared sensor array, and the right-turn infrared sensor array.

[0034] It should be noted that the formula for binarizing the simulated signal of reflected light intensity from each path surface is: (1); in, The output is a binarized signal; is the output of a single infrared sensor; x is the number of the infrared sensor array from left to right as (1, 2, 3); This is the binarization threshold; values ​​above this threshold output 1, and values ​​below or equal to this threshold output 0. Experimental testing showed that setting the binarization threshold to 80% of the maximum output of the infrared sensor effectively reflects the detection of the black tracking line. Based on the binarized arrays corresponding to the left-turn infrared sensor array, the straight-line infrared sensor array, and the right-turn infrared sensor array, the current path position can be detected, and the infrared tracking status can also be determined.

[0035] Furthermore, based on the binarized arrays corresponding to the left-turn infrared sensor array, the straight-ahead infrared sensor array, and the right-turn infrared sensor array, the infrared tracking state is determined, specifically including: When the binarized array corresponding to the left-turn infrared sensor array indicates that the left-turn direction has been detected and the binarized array corresponding to the right-turn infrared sensor array indicates that the right-turn direction has not been detected, the infrared tracking state is determined to be normal.

[0036] When the binary array corresponding to the right-turn infrared sensor array indicates that the right-turn direction has been detected and the binary array corresponding to the left-turn infrared sensor array indicates that the left-turn direction has not been detected, the infrared tracking state is determined to be normal.

[0037] When the binary array corresponding to the straight-going infrared sensor array indicates that the straight-going direction has been detected, the binary array corresponding to the right-turn infrared sensor array indicates that the right-turn direction has not been detected, and the binary array corresponding to the left-turn infrared sensor array indicates that the left-turn direction has not been detected, the infrared tracking state is determined to be normal.

[0038] When the binary array corresponding to the right-turn infrared sensor array indicates that a right-turn direction has been detected and the binary array corresponding to the left-turn infrared sensor array indicates that a left-turn direction has been detected, the infrared tracking state is determined to be in a failed state.

[0039] When the binarized array corresponding to the straight-going infrared sensor array indicates that the straight-going direction cannot be detected, the binarized array corresponding to the right-turn infrared sensor array indicates that the right-turn direction cannot be detected, and the binarized array corresponding to the left-turn infrared sensor array indicates that the left-turn direction cannot be detected, the infrared tracking state is determined to be in a failed state.

[0040] To better understand the determination of infrared tracking status in the infrared-visual collaborative tracking navigation method provided in this application embodiment, taking three infrared sensors as an example, the specific method includes: First, simulated signals of reflected light intensity from the path surface collected by three infrared sensors are acquired. By comparing the magnitudes of the three simulated signals, the relative position of the current path position and the sensor position can be detected. The simulated signals of reflected light intensity from the three path surfaces are then binarized to generate a binary array. There are eight possible combinations of the binary output from the three infrared sensors, which can be divided into two types: a normal state (corresponding to five combinations), in which at least one sensor should detect the path and be able to complete infrared tracking; and a failure state (corresponding to three combinations). Normal tracking would not output these three combinations, therefore, it can be determined that the system has entered a failure state, which may be due to path damage, deviation from the path, or interference from ambient light. See Table 1 for details.

[0041] Table 1 Binarized arrays of three infrared sensors

[0042] Where 1 indicates that the infrared sensor detected the path; 0 indicates that the infrared sensor did not detect the path; and NA indicates that no data was entered.

[0043] For infrared sensor arrays with three or more infrared sensors, all infrared sensors should be divided into three groups based on the path width: left-turn, straight-ahead, and right-turn. For example, when there are four infrared sensors, if the path width is greater than the coverage area of ​​two infrared sensors, then the two middle infrared sensors form a straight-ahead infrared sensor array, the left infrared sensor forms a left-turn infrared sensor array, and the right infrared sensor forms a right-turn infrared sensor array. Then, all possible scenarios during normal tracking are listed again, representing the normal state. In this state, at least one group of sensors detects the path, and the remaining impossible scenarios represent the failure state.

[0044] In another exemplary embodiment of this application, such as Figure 3 As shown, step 203 above is replaced by steps 301 to 304.

[0045] Step 301: Preprocess the path surface image to obtain a preprocessed path surface image.

[0046] Step 302: Perform path localization on the preprocessed path surface image to obtain the final path localization points.

[0047] Step 303: Calculate the deviation angle based on the path positioning points and the preprocessed path surface image.

[0048] Step 304: Determine the visual tracking motion control signal based on the deviation angle and the PID control algorithm.

[0049] It should be noted that, in the preprocessing of the path surface image, firstly, the path surface image is binarized. The binarization colors are selected from the HSV color space; the black tracking line is set to (255, 255, 255) to represent white, and the other background elements are set to (0, 0, 0) to represent black. Then, erosion and dilation operations are performed on the binarized path surface image to eliminate noise. Finally, the preprocessed path surface image is obtained.

[0050] Further explanation is needed regarding path localization on the preprocessed path surface image. First, three specific regions are selected within the preprocessed path surface image. The largest white contour within each of these three regions is detected, and the smallest bounding rectangle of that largest white contour is drawn. Then, the coordinates of the center points of these three smallest bounding rectangles are obtained. These coordinates represent the detected path location, and the center points of these three smallest bounding rectangles are the three path localization points. To improve robustness, three path localization points are obtained in each detection cycle. Finally, the coordinates of the three obtained path localization points are weighted and averaged to obtain the final path localization point.

[0051] In this embodiment of the application, the formula for calculating the deviation angle is: (2); in, H is the deviation angle; H is the height of the preprocessed path surface image; W is the width of the preprocessed path surface image. The x-coordinate of the final path location point; This represents the ordinate of the final path location point.

[0052] In this embodiment of the application, the visual tracking motion control signal is determined based on the deviation angle and the PID control algorithm, specifically including: First, the deviation in the PID control algorithm is determined as the deviation angle.

[0053] Secondly, based on the deviation in the formula and PID control algorithm determined by the visual tracking motion control signal, the visual tracking motion control signal is determined.

[0054] The formula for determining the visual tracking motion control signal is as follows: (3); in, This is the visual tracking motion control signal corresponding to the nth sampling time. The proportional coefficient of the PID control algorithm; For the integral coefficients of the PID control algorithm; represents the differential coefficients of the PID control algorithm; n represents the sampling time. This represents the deviation from the tracking line corresponding to the nth sampling time of the mobile robot. This represents the deviation from the tracking line corresponding to the (n-1)th sampling time of the mobile robot.

[0055] In this embodiment of the application, the visual tracking motion control signal is used to control the mobile robot to perform tracking motion.

[0056] The visual tracking motion control signal is mapped to the mobile robot's speed range. Given a speed range defined as [-s, s], the straight-line speed is set to s / 2, and the turning speed is calculated using the following formula: (4); Where s is the maximum straight-line speed of the mobile robot; The maximum value of the PID control algorithm output signal in visual tracking mode, i.e., the deviation angle. This was calculated when the value is π / 2.

[0057] The default steering speed for left turns is greater than 0, when the deviation angle... When the deviation angle is greater than 0, the mobile robot should turn right, and the turning speed should be less than 0; when the deviation angle is greater than 0, the robot should turn right. When the value is less than or equal to 0, the mobile robot should turn to the left, and the turning speed should be greater than 0.

[0058] It should be noted that the above straight-line speed and turning speed are converted into PWM signals for the motor. The motor drives the wheels to rotate, completing one visual tracking motion control cycle, and then returns to step 201 to perform the next tracking motion control cycle.

[0059] In another exemplary embodiment of this application, step 204 above specifically includes: first, calculating the deviation amount based on the simulated signal of the reflected light intensity of the path surface.

[0060] The formula for calculating the deviation is as follows: (5); in, This is the reference value of the output analog signal at the nth sampling moment when the infrared sensor array is located at the center of the path; The output value of the infrared analog signal corresponding to the nth sampling moment during the infrared tracking process.

[0061] Secondly, based on the aforementioned deviation and the PID control algorithm, the infrared tracking motion control signal is determined.

[0062] The formula for determining the infrared tracking motion control signal is as follows: (6); in, This is the infrared tracking motion control signal corresponding to the nth sampling time. The proportional gain of the PID controller; The integral coefficient of the PID controller; The derivative coefficients of the PID controller.

[0063] In this embodiment, the infrared tracking motion control signal is used to control the mobile robot to perform tracking motion.

[0064] The infrared tracking motion control signal is mapped to the mobile robot's speed range. Given a speed range defined as [-s, s], with the straight-line speed set to s / 2, the formula for calculating the turning speed is: (7); in, This represents the maximum value of the PID control algorithm output signal in infrared tracking mode, i.e., the deviation. The maximum value is measured; P1 is the output value of the left-turn infrared sensor array; P3 is the output value of the right-turn infrared sensor array.

[0065] The default turning speed is greater than 0 for left turns. When the output value of the right-turn infrared sensor array P3 is greater than or equal to that of the left-turn infrared sensor array P1, it indicates that the mobile robot is turning right and the turning speed should be less than 0. When the output value of the right-turn infrared sensor array P3 is less than that of the left-turn infrared sensor array P1, it indicates that the mobile robot is turning left and the turning speed should be greater than 0.

[0066] It should be noted that the above straight-line speed and turning speed are converted into PWM signals for the motor. The motor drives the wheels to rotate, completing one infrared tracking motion control cycle, and then returns to step 201 to perform the next tracking motion control cycle.

[0067] To better understand the implementation principle of the infrared-vision cooperative line-following navigation method provided in the embodiments of this application, let's take an infrared sensor array consisting of three infrared sensors as an example: Figure 4 (a) and Figure 4 (b) illustrates the principle behind infrared tracking. Figure 4 Part (a) is the corresponding binary array 010 in the infrared tracking mode. Infrared tracking is in normal state and the mobile robot executes the straight-line command. Figure 4 In part (b), the corresponding binarized array in the infrared tracking mode is 100, the infrared tracking is in normal state, and the mobile robot executes the left turn command.

[0068] Figure 4 (d) and Figure 4 (e) represents the principle behind visual tracking. Figure 4 (d) and Figure 4 (e) represents the process of visual tracking when infrared tracking fails; Figure 4 Part (d) is the result of preprocessing the path surface image under visual tracking mode; Figure 4 Part (e) is the process of locating the path on the preprocessed path surface image in visual tracking mode.

[0069] Figure 4 Part (c) is that in infrared tracking mode, when the ground marking line is damaged, the binary array output of the infrared sensor is 000, and the infrared tracking is determined to be in a failed state. Figure 4 Part (f) is the location of the ground marking line that is visually identified under the same conditions, which can generate two path positioning points to complete the subsequent visual tracking.

[0070] In existing technologies, while infrared tracking can achieve high-precision navigation, it is susceptible to ambient light interference or path damage, resulting in poor stability. Visual tracking, while robust, suffers from blind spots due to camera mounting locations, hindering high-precision navigation. Compared to these two single-mode tracking navigation technologies, the hybrid infrared-visual cooperative tracking control method proposed in this application has significant advantages, primarily in the unification of high precision and high stability.

[0071] This advantage primarily stems from the fact that infrared tracking uses a three-sensor array to detect path deviations, while the visual mode estimates the deviation angle by processing RGB images. Furthermore, in the aforementioned embodiment, step 202 determines the infrared tracking state based on simulated signals of reflected light intensity from all the path surfaces, and in step 203, when the infrared tracking state is in a failed state, the path surface image captured by the camera is acquired, and the visual tracking motion control signal is determined based on the path surface image and the PID control algorithm. Step 202 determines the infrared tracking state based on simulated signals of reflected light intensity from all the path surfaces. This application can monitor the binarized output combination of infrared sensors in real time. Once environmental interference or path damage is detected, it automatically switches to visual tracking mode, avoiding navigation interruption caused by local failures in infrared tracking mode, thus maintaining overall stability. At the same time, thanks to step 203, when the infrared tracking state is in a failed state, the system acquires the path surface image collected by the camera, and determines the visual tracking motion control signal based on the path surface image and PID control algorithm. The system performs HSV color space binarization processing and erosion dilation denoising on the RGB image, which ensures the stable recognition of the tracking line (i.e., ground marking line). The system calculates the path deviation angle θ by weighted averaging of multiple detection points, which can provide reliable position information even when the path is discontinuous, further improving the stability of visual tracking mode and making up for the shortcomings of infrared tracking mode.

[0072] This infrared-vision cooperative navigation mechanism supports automatic switching of tracking modes based on environmental conditions, enabling mobile robots to prioritize the use of infrared tracking mode when it provides high-precision data, and seamlessly switch to visual tracking mode when it fails. This robust handling ultimately achieves a dynamic balance between accuracy and stability, solving the problem that a single mode cannot simultaneously address both.

[0073] Based on the same inventive concept, this application also provides a mobile robot device for implementing the infrared-visual cooperative line-following navigation method for mobile robots described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more mobile robot device embodiments provided below can be found in the above-described limitations regarding the infrared-visual cooperative line-following navigation method for mobile robots, and will not be repeated here.

[0074] In one exemplary embodiment, such as Figure 5 As shown, a mobile robot device is provided, including: a mobile robot, a control module disposed on the mobile robot, a camera disposed on the top of the mobile robot, and an infrared sensor array disposed on the bottom of the mobile robot.

[0075] The infrared sensor array includes multiple infrared emitters and an infrared sensor associated with each of the infrared emitters.

[0076] The infrared sensor is used to collect simulated signals of the intensity of reflected light from the path surface.

[0077] The camera is used to acquire images of the path surface.

[0078] The control module is used to execute an infrared-visual cooperative line-following navigation method for mobile robots as described in any of the above embodiments.

[0079] It should be noted that the control module uses NVIDIA's Jetson Orin Nano developer kit.

[0080] As another alternative implementation method, such as Figure 6 As shown, the operation flow of a mobile robot device includes: a system input module, a processing module, and a system output module.

[0081] In the system input module, the path surface image captured by the camera on top of the mobile robot is an RGB image, and the infrared sensor array at the bottom of the mobile robot acquires infrared information.

[0082] In the processing module, the position of the tracking line is identified through infrared tracking mode and visual tracking mode to obtain identification information. At the same time, based on the infrared information obtained by the infrared sensor array, the infrared failure judgment unit determines whether the infrared tracking mode is invalid. Then, the tracking mode selection unit selects the tracking mode according to the infrared status. If it is invalid, the deviation of the visual tracking motion is output under the visual tracking mode; if it is normal, the deviation of the infrared tracking motion is output under the infrared tracking mode. The PID control unit generates the corresponding motion control signal based on the above deviation and transmits the motion control signal to the system output module. The system output module outputs the corresponding running speed command, which is then converted into a PWM signal for the motor to drive the wheel to rotate and complete the tracking motion.

[0083] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 7As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores processing data for infrared-vision cooperative line-following navigation. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements an infrared-vision cooperative line-following navigation method for mobile robots.

[0084] Those skilled in the art will understand that Figure 7 The structures shown are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0085] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0086] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0087] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0088] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0089] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0090] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0091] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An infrared-vision cooperative tracking navigation method applied to a mobile robot, characterized by, The application relates to an infrared-vision cooperative tracking navigation method for a mobile robot. An infrared sensor array is arranged at the bottom of the mobile robot, and each infrared sensor in the infrared sensor array collects a path surface reflection light intensity analog signal. Based on all the path surface reflection light intensity analog signals, an infrared tracking state is determined. When the infrared tracking state is the failure state, a path surface image collected by a camera is acquired, and a vision tracking motion control signal is determined based on the path surface image and a PID control algorithm; the vision tracking motion control signal is used for controlling the mobile robot to perform tracking motion; the camera is arranged at the top of the mobile robot. When the infrared tracking state is the normal state, an infrared tracking motion control signal is determined based on all the path surface reflection light intensity analog signals and a PID control algorithm; the infrared tracking motion control signal is used for controlling the mobile robot to perform tracking motion.

2. The infrared-vision cooperative tracking navigation method for mobile robots according to claim 1, wherein, The infrared sensor array comprises a left-turn infrared sensor array, a straight-ahead infrared sensor array and a right-turn infrared sensor array. Each path surface reflection light intensity analog signal is preprocessed to obtain a binary array corresponding to the left-turn infrared sensor array, a binary array corresponding to the straight-ahead infrared sensor array and a binary array corresponding to the right-turn infrared sensor array. The infrared tracking state is determined according to the binary array corresponding to the left-turn infrared sensor array, the binary array corresponding to the straight-ahead infrared sensor array and the binary array corresponding to the right-turn infrared sensor array.

3. The infrared-vision cooperative tracking navigation method for mobile robots according to claim 2, wherein, The infrared tracking state is determined according to the binary array corresponding to the left-turn infrared sensor array, the binary array corresponding to the straight-ahead infrared sensor array and the binary array corresponding to the right-turn infrared sensor array, specifically including: When the binary array corresponding to the left-turn infrared sensor array indicates that the left-turn direction is detected and the binary array corresponding to the right-turn infrared sensor array indicates that the right-turn direction is not detected, the infrared tracking state is determined as the normal state; When the binary array corresponding to the right-turn infrared sensor array indicates that the right-turn direction is detected and the binary array corresponding to the left-turn infrared sensor array indicates that the left-turn direction is not detected, the infrared tracking state is determined as the normal state; When the binary array corresponding to the straight-ahead infrared sensor array indicates that the straight-ahead direction is detected, the binary array corresponding to the right-turn infrared sensor array indicates that the right-turn direction is not detected and the binary array corresponding to the left-turn infrared sensor array indicates that the left-turn direction is not detected, the infrared tracking state is determined as the normal state; When the binary array corresponding to the right-turn infrared sensor array indicates that the right-turn direction is detected and the binary array corresponding to the left-turn infrared sensor array indicates that the left-turn direction is detected, the infrared tracking state is determined as the failure state; and When the binary array corresponding to the right-turn infrared sensor array indicates that the right-turn direction is detected and the binary array corresponding to the left-turn infrared sensor array indicates that the left-turn direction is detected, the infrared tracking state is determined as the failure state. When the straight-going infrared sensor array corresponding binary array indicates that the straight-going direction is not detected, the right-turn infrared sensor array corresponding binary array indicates that the right-turn direction is not detected, and the left-turn infrared sensor array corresponding binary array indicates that the left-turn direction is not detected, it is determined that the infrared tracking state is a failure state.

4. The infrared-vision cooperative tracking navigation method for mobile robots according to claim 1, wherein, Based on the path surface image and the PID control algorithm, a visual tracking motion control signal is determined, specifically including: The path surface image is preprocessed to obtain a preprocessed path surface image; The preprocessed path surface image is path positioned to obtain a final path positioning point; According to the final path positioning point and the preprocessed path surface image, a deviation angle is calculated; According to the deviation angle and the PID control algorithm, a visual tracking motion control signal is determined.

5. The infrared-vision cooperative tracking navigation method for mobile robots according to claim 4, wherein, The calculation formula of the deviation angle is: ； wherein, is the bias angle; H is the height of the pre-processed path surface image; W is the width of the pre-processed path surface image; is the horizontal coordinate of the final path positioning point; is the vertical coordinate of the final path positioning point.

6. The infrared-vision cooperative tracking navigation method for mobile robots according to claim 4, wherein, According to the deviation angle and the PID control algorithm, a visual tracking motion control signal is determined, specifically including: The deviation amount in the PID control algorithm is determined as the deviation angle; According to the visual tracking motion control signal determination formula and the deviation amount in the PID control algorithm, a visual tracking motion control signal is determined; The visual tracking motion control signal determination formula is: ； wherein, is the visual tracking motion control signal corresponding to the nth sampling time; is the proportional coefficient of the PID control algorithm; is the integral coefficient of the PID control algorithm; is the differential coefficient of the PID control algorithm; n is the sampling time; is the deviation of the mobile robot from the tracking line corresponding to the nth sampling time; is the deviation of the mobile robot from the tracking line corresponding to the n-1th sampling time.

7. A mobile robotic device, characterized by The mobile robot, the control module arranged on the mobile robot, the camera arranged on the top of the mobile robot, and the infrared sensor array arranged on the bottom of the mobile robot are included. The infrared sensor array includes a plurality of infrared emission tubes and an infrared sensor associated with each infrared emission tube. The infrared sensor is used to collect a path surface reflected light intensity analog signal. The camera is used to collect a path surface image. The control module is used to execute the infrared-visual collaborative tracking navigation method for a mobile robot according to any one of claims 1-6.

8. A computer device comprising: A memory, a processor, and a computer program stored on the memory and capable of running on the processor, characterized in that the processor executes the computer program to implement the infrared-visual collaborative tracking navigation method for a mobile robot according to any one of claims 1-6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the infrared-visual collaborative tracking navigation method for a mobile robot according to any one of claims 1-6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the infrared-visual collaborative tracking navigation method for a mobile robot according to any one of claims 1-6. The computer program is executed by the processor to implement the infrared-visual collaborative tracking navigation method for a mobile robot according to any one of claims 1-6.

Images