A ring robot environment perception method based on multi-sensor time sequence fusion

Abstract

The present application relates to the technical field of robot arena, and particularly relates to a kind of robot arena environment perception method based on multi-sensor time sequence fusion, mainly solve the technical problems that the low precision of perception, poor robustness and unreasonable scheduling exist in robot arena environment perception method.This method is based on robot arena equipped with vision sensor, IMU sensor, ultrasonic sensor and infrared reflective sensor and with preset perception cycle iterative update, in each perception cycle, the method executes the following steps: S1. generate synchronous observation sequence;S2. form feature sequence;S3. calculate credibility index;S4. construct dynamic attention network, and output fusion weight vector and scheduling instruction set;S5. calculate white line boundary risk degree;S6. carry out overlay correction by safety strategy layer;S7. multi-sensor time sequence fusion;S8. result feedback and update historical time sequence window.

Description

Technical Field

[0001] This invention relates to the field of arena robot technology, and in particular to an environmental perception method for arena robots based on multi-sensor temporal fusion. Background Technology

[0002] During the competition, the arena robot needs to simultaneously achieve three core tasks: opponent location, stable self-position control, and real-time boundary judgment to prevent cross-boundary. The reliability of the coordination of these three tasks directly determines the effectiveness of the arena robot's competition and the compliance of the competition, and is a key prerequisite for the stable operation of the arena robot.

[0003] Currently, solutions for boundary recognition and opponent detection mostly rely on visual sensors and / or distance sensors. However, in complex scenarios involving intense robot combat, these solutions are susceptible to interference from factors such as occlusion, reflection, high-speed turning, and impact vibrations, leading to a significant decrease in perception accuracy and stability. This makes it difficult to meet the demands of actual combat. Specific shortcomings are as follows: Visual sensors are prone to a sharp drop in confidence in recognizing white lines due to high ground reflection, glare from arena lights, or occlusion by opponent limbs, resulting in the inability to output reliable boundary perception signals in a timely manner and affecting the timeliness of boundary crossing control. Distance sensors are susceptible to multipath effects and echo interference, causing fluctuations in ranging data and making it difficult to stably support opponent positioning and boundary distance judgment, further exacerbating perception uncertainty. Regarding multi-sensor fusion strategies, fixed-weight fusion or simple weighted fusion modes are used, lacking the ability to dynamically adjust fusion weights based on the robot's real-time task objectives, motion state, and historical perception time-series information. This makes it difficult to adapt to the dynamic changes in the scenario during combat and to balance the priority requirements of different perception tasks, further reducing the robustness of the perception system.

[0004] Furthermore, existing sensor scheduling mechanisms have significant shortcomings. They typically only implement sensor on / off control or simple frequency reduction adjustment, lacking rigid constraints and verifiable safety scheduling strategies to address the risk of robots crossing the white line. This deficiency leads to the potential for boundary detection sensors to be erroneously degraded when system resources are strained, resulting in potential problems such as robots violating boundary rules and unstable posture control, seriously affecting the safety of the competition and the reliability of robot combat.

[0005] In summary, there is an urgent need for an environmental perception method for arena robots that is highly accurate, robust, and well-scheduled, in order to solve the perception and control challenges in complex adversarial scenarios. Summary of the Invention

[0006] To overcome the technical shortcomings of existing arena robot environmental perception methods, such as low perception accuracy, poor robustness, and unreasonable scheduling, this invention provides an arena robot environmental perception method based on multi-sensor temporal fusion.

[0007] The present invention provides an environmental perception method for arena robots based on multi-sensor temporal fusion. This method is based on an arena robot equipped with a visual sensor, an IMU sensor, an ultrasonic sensor, and an infrared reflective sensor, and iterates and updates at a preset perception cycle. In each perception cycle, the method performs the following steps:

[0008] S1. The visual sensor acquires images of the detection area in front of the arena robot and generates a visual perception data sequence. The IMU sensor acquires inertial data of the arena robot and generates an IMU perception data sequence. The ultrasonic sensor acquires distance signals between the arena robot and its opponent or obstacle and generates an ultrasonic perception data sequence. The infrared reflective sensor acquires white line reflection signals of the arena boundary and generates an infrared perception data sequence. Each perception data sequence is accompanied by a corresponding acquisition timestamp. A unified time axis is established based on the acquisition timestamps and time-domain alignment is performed to generate a synchronous observation sequence.

[0009] S2. Extract visual features, IMU features, ultrasonic features, and infrared features from the synchronous observation sequence, and form a feature sequence within the historical time window;

[0010] S3. Calculate the reliability index of each sensor to obtain the reliability of vision, IMU, ultrasound and infrared.

[0011] S4. The feature sequence, the credibility index, and the current state vector of the arena robot are input into a dynamic attention network, which outputs a fusion weight vector. and the set of scheduling instructions ;

[0012] in, , Represents the weights of the visual sensors. Indicates the weights of the IMU sensors. Indicates the weight of the ultrasonic sensor. The weights of the infrared reflective sensor are represented, and ;

[0013] S5. Calculate the risk level of the white line boundary. ;

[0014] in, For normalized quantity and ;

[0015] S6. Establish a security policy layer independent of the dynamic attention network. This security policy layer stores and executes a set of white-line security state machines to manage the fused weight vector. and the set of scheduling instructions Impose inviolable hard constraints;

[0016] S7. Based on the fusion weight vector corrected by the security policy layer overlay and the set of scheduling instructions It performs sensor scheduling and multi-sensor time-series fusion, and outputs environmental perception results.

[0017] S8. Feed back the environmental perception results to the arena robot and update the historical time sequence window for the next perception cycle.

[0018] Furthermore, the white-line safety state machine presets discrete states and a set of fixed parameters, the discrete states including the normal state. Alert status and dangerous situation The curing parameter set includes a first threshold. Second threshold hysteresis First Lock-up Period Second Locking Period , and The white-line safety state machine also presets the following state transition rules:

[0019] when Enter alert status ,when Entering a dangerous state ;

[0020] When in a dangerous situation And in the second locking period Internal continuous satisfaction At that time, it is permissible to leave the dangerous state. De-escalation to alert status ;

[0021] When in a state of alert And in the first locking period Internal continuous satisfaction At that time, it is permissible to leave the state of alert. Return to normal status .

[0022] Furthermore, the white line safety state machine also presets the following action rules:

[0023] The infrared reflective sensor has three preset levels for sampling frequency and quantization accuracy: low, medium, and high; when in alert mode... At that time, the sampling frequency and quantization accuracy of the infrared reflective sensor are both selected to be medium or high, and are maintained at least during the first locking period. The internal gear position is locked, and the weight of the infrared reflective sensor is maintained. When in a dangerous situation At that time, the sampling frequency and quantization accuracy of the infrared reflective sensor are both selected at a high level and are maintained at least during the second locking period. The internal gear position is locked, and the weight of the infrared reflective sensor is maintained. ;

[0024] When the action rule conflicts with the output of the dynamic attention network, the action rule shall prevail.

[0025] Furthermore, the security policy layer also stores and executes resource-limited judgment rules and resource priority downgrade rules.

[0026] The resource-limited determination rule is: when CPU utilization is... or bus bandwidth utilization or sensor power consumption utilization At that time, it is determined that resources are limited;

[0027] The resource priority degradation rule is as follows: Under resource-constrained conditions, priority is given to ensuring that the infrared reflective sensor meets the action rule, while the visual sensor and / or ultrasonic sensor are forcibly degraded. When the visual sensor is degraded, its frame rate is reduced to 30fps or below, or its resolution is reduced to 640×480 or below. When the ultrasonic sensor is degraded, its weight is reduced to... .

[0028] Furthermore, the security policy layer covers and corrects the fusion weight vector. Afterwards, a validity check is performed and the diagnostic quantity is recorded;

[0029] The diagnostic parameters include the weights of the infrared reflective sensors before and after coverage. Discrete states of the white line safety state machine, whether resources are limited, whether the weight cap is triggered, and the current white line boundary risk level. And the remaining cycle of the gear position and gear lock of the infrared reflective sensor.

[0030] Furthermore, the visual credibility is calculated based on the reflectivity anomaly and occlusion ratio indices;

[0031] When the visual credibility is below the visual threshold and the alert state is active. or dangerous situation In such cases, the security policy layer downgrades the visual sensor.

[0032] Furthermore, the ultrasonic reliability is calculated based on multipath anomaly and empty echo anomaly.

[0033] When the ultrasonic confidence level is below the ultrasonic threshold and the system is in a state of alert. or dangerous situation When using ultrasonic ranging results to reduce the risk level of white line boundaries, it is prohibited. The weights of the ultrasonic sensors are determined by the security policy layer. Apply upper limit constraints .

[0034] Furthermore, the infrared confidence level is calculated based on the white line reflection saturation discriminant and the ground material drift discriminant.

[0035] When the infrared confidence level is below the infrared threshold and the system is in a state of alert. or dangerous situation At that time, the security strategy layer maintains the constraint on the infrared reflective sensor and forces a retest action to be performed in the next sensing cycle. The retest action is to keep at least one of the sampling frequency and quantization accuracy of the infrared reflective sensor at the current level or increase it by one level.

[0036] Furthermore, the historical time series window includes a short window and a long window, with the short window used for assessing the risk level of the white line boundary. The calculation and state transition triggering are performed, and the long window is used for sensor stability assessment and drift trend analysis.

[0037] Furthermore, the risk level of the white line boundary. Including consistency risk components The consistency risk component The distance is calculated from the deviation between the infrared white line distance detected by the infrared reflective sensor and the visual white line distance detected by the visual sensor.

[0038] When the consistency risk component When the deviation exceeds the threshold, the risk level of the white line boundary will be adjusted. Raise to the second threshold and above, to enter a dangerous state. .

[0039] The technical solution provided by this invention has the following advantages compared with the prior art:

[0040] 1) The ring robot environment perception method provided by the present invention introduces visual credibility on the one hand, and reduces the weight of untrusted vision by working together with the dynamic attention network and the safety strategy layer to avoid misidentification under conditions such as glare and occlusion; on the other hand, it introduces ultrasonic credibility, which can avoid abnormal ultrasonic results such as abnormal echoes from affecting the risk of white lines, and suppress misleading boundary judgments from a mechanism perspective; the combination of the two aspects can improve perception accuracy.

[0041] 2) The arena robot environment perception method provided by the present invention adopts a learnable dynamic attention network and takes feature sequence, confidence index and state vector as input together, and outputs the fusion weight of each sensor in real time. It can achieve adaptive fusion against rhythm changes, sudden turns and sudden stops and other states, and has stronger robustness.

[0042] 3) The arena robot environment perception method provided by the present invention uses a dynamic attention network to output scheduling instructions, and the safety strategy layer performs hard constraints on the scheduling instructions by executing the white line safety state machine, which can effectively avoid boundary crossing and ensure the motion stability of the arena robot, making the scheduling more reasonable. Detailed Implementation

[0043] To better understand the above-mentioned objectives, features, and advantages of the present invention, the solutions of the present invention will be further described below. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0044] Many specific details are set forth in the following description in order to provide a full understanding of the invention, but the invention may also be practiced in other ways different from those described herein; obviously, the embodiments in the specification are only some embodiments of the invention, and not all embodiments.

[0045] The specific embodiments of the present invention will be described in detail below.

[0046] This embodiment provides a method for environmental perception of a ring robot based on multi-sensor temporal fusion. The method is based on a ring robot equipped with a visual sensor, an IMU sensor, an ultrasonic sensor and an infrared reflective sensor, and iterates and updates according to a preset perception cycle. In each perception cycle, the method executes steps S1 to S8.

[0047] Specifically, a vision sensor is installed on the top or front of the arena robot, facing the opponent and the arena floor in front, to output image frames; an IMU sensor, or inertial measurement unit, is used to output the angular velocity, linear acceleration, and attitude calculation values ​​of the arena robot; an ultrasonic sensor is used to output the distance value between the arena robot and the opponent or obstacle, as well as echo quality information, to assist in judging the distance to nearby obstacles or opponents; an infrared reflective sensor is installed facing the arena floor, and can be a single sensor or an array, to detect changes in the reflection of the white line at the arena boundary, and output infrared reflection intensity or reflection difference value.

[0048] It is easy to understand that the arena robot should also be equipped with a computing unit. The computing unit is embedded in the arena robot and can be a main control board or an industrial control board. It has the ability to perform online statistics on CPU usage, bus bandwidth usage, and sensor power consumption.

[0049] Specifically, the perception cycle can be set according to the platform's computing power and the speed requirements of the countermeasures, for example, it can be set to 10ms, 20ms or other fixed values.

[0050] It should be noted that each sensing data sequence retains its original acquisition timestamp. If a sensor can only provide a frame number or trigger sequence number, it is converted into a system timestamp and unified to the same time domain at the driver layer.

[0051] S1. Images of the detection area in front of the arena robot are acquired using a visual sensor and a visual perception data sequence is generated. Inertial data of the arena robot are acquired using an IMU sensor and an IMU perception data sequence is generated. Distance signals between the arena robot and its opponent or obstacle are acquired using an ultrasonic sensor and an ultrasonic perception data sequence is generated. Reflection signals of the white line at the arena boundary are acquired using an infrared reflective sensor and an infrared perception data sequence is generated. Each perception data sequence is accompanied by a corresponding acquisition timestamp. A unified time axis is established based on the acquisition timestamps and time-domain alignment is performed to generate a synchronous observation sequence.

[0052] Specifically, in this embodiment, the acquisition timestamp of the IMU sensing data sequence is used as the master clock source to generate the center moment of the current sensing cycle on a unified time axis. Then, the temporal alignment of the visual perception data sequence, ultrasonic perception data sequence, and infrared perception data sequence is completed through resampling or interpolation operations. IMU sensors have a high update frequency and good temporal continuity, making them suitable as an alignment reference.

[0053] More specifically, visual perception data sequences are discrete frames and are susceptible to frame rate fluctuations due to computing power or bandwidth limitations. Therefore, in this embodiment, when performing temporal alignment on the visual perception data sequence, a distance from the center time is selected. The two most recent frames, and perform the following operation: if the center time If it is located between two frames, then these two frames are related to the center time. The time intervals are all within the allowable range. Feature-level interpolation is performed between the two frames to complete the temporal alignment, thus obtaining a result closer to the center moment. The visual characteristics make the robot more stable when moving at high speeds; if the center moment If it is located outside two frames, then the time distance from the center is directly selected. Extract features from the most recent frame; if a sufficiently close visual frame cannot be obtained in several consecutive perception cycles, it is determined that the vision has lost frames or is severely delayed. In this case, the most recent effective visual features are used as temporary features.

[0054] More specifically, the IMU sensing data sequence has the characteristics of high frequency and continuity, and the update rate is much higher than the sensing cycle. Therefore, in this embodiment, when aligning the IMU sensing data sequence in the time domain, the following operation is adopted: IMU sampling points are collected in each sensing cycle to generate representative IMU features of that sensing cycle, such as representative values ​​of angular velocity, representative values ​​of acceleration, attitude stability index, etc.; if there is a missing IMU sample, the representative IMU features of the previous sensing cycle are used to fill it.

[0055] More specifically, ultrasonic sensing data sequences are typically sampled unevenly and are prone to multipath propagation and empty echoes. During alignment, it's crucial to avoid treating expired data as current data. Therefore, this embodiment employs the following operation when aligning ultrasonic sensing data sequences in the time domain: After each ultrasonic echo yields a distance result, an acceptable valid time window is established, which serves as the validity period. When the ultrasonic data is within the validity period, since the ultrasonic trigger interval may exceed the sensing cycle, the most recent valid echo result is directly used as the ultrasonic input for this sensing cycle. When the ultrasonic data is outside the validity period, i.e., there are no new ultrasonic echoes in the current sensing cycle and the most recent echo has exceeded its validity period, the ultrasonic input for this sensing cycle is marked as invalid. If an empty echo or multipath anomaly is detected, the echo is directly treated as invalid.

[0056] More specifically, infrared sensing data sequences are typically taken continuously, and infrared reflective sensors are used to detect white lines. Alignment rules tend to favor stability and security. Therefore, in this embodiment, the following operation is used when aligning the infrared sensing data sequence in the time domain: when the infrared sampling frequency is higher than or close to the sensing period, the center time... Typically, this occurs between two adjacent infrared samples. At this point, interpolation is performed on the infrared reflection intensity or an extracted infrared feature to obtain the result relative to the center time. Consistent infrared characteristics; if the infrared data for the current sensing cycle has not been updated, the most recent valid infrared data is retained as the infrared input for the current sensing cycle.

[0057] It is easy to understand that temporal alignment can ensure the comparability and fusion of multiple sensors within the same sensing cycle, avoiding the risk of misjudgment of white lines caused by time misalignment.

[0058] It is important to note that the synchronous observation sequence not only records the aligned features of each sensor, but also records the flags indicating whether each sensor has been updated in the current period and whether it has expired or is invalid.

[0059] S2. Extract visual features, IMU features, ultrasonic features, and infrared features from the synchronous observation sequence and form a feature sequence within the historical time window.

[0060] Specifically, the historical timeline window in this embodiment includes a short window and a long window: the short window emphasizes real-time performance and is used for assessing the risk level of the white line boundary. The calculation and state transition triggering (described in detail below); the long window emphasizes robustness and is used for sensor stability assessment and drift trend analysis.

[0061] Specifically, the visual features in this embodiment include opponent candidate features and white line candidate features. Opponent candidate features include the center, scale, and motion trend of the opponent target box, denoted as... The candidate features for the white line include the detection results of the white line region and the features of the edge line segments, denoted as... .

[0062] Specifically, the IMU features in this embodiment include the arena robot's angular velocity, linear acceleration, and attitude stability, denoted as... IMU features are cached in short and long windows for use in confidence determination, dynamic attention network, and white line boundary risk assessment, as described later. Used for calculations.

[0063] Specifically, the ultrasonic features in this embodiment include echo distance and echo stability features. The echo distance is denoted as... Echo stability characteristics include distance variance and number of abrupt changes within a short window, denoted as ; .

[0064] Specifically, the infrared features in this embodiment include white line reflection intensity, white line edge gradient, and white line continuity. White line reflection intensity is denoted as... The original infrared reflective sensor The value is obtained by normalization. The gradient along the edge of the white line is denoted as... It is obtained by the difference between adjacent channels or adjacent sampling points. The continuity of the white line is denoted as Characterizing short window The percentage of frames in which a white line is detected within a frame is defined as the short window. The threshold for determining the white line is Exponential function If the condition is true, take the value 1; if the condition is false, take the value 0. ,in, The larger the value, the more continuous the white line detection results are within the short window, and the higher the proportion of occurrence.

[0065] S3. Calculate the reliability index of each sensor to obtain the reliability of vision, IMU, ultrasound and infrared sensors.

[0066] It is easy to understand that the credibility index is an explicit quantification of whether a sensor is trustworthy, and it is uniformly normalized to the range of 0 to 1, with the larger the value, the more trustworthy it is.

[0067] Specifically, the visual credibility in this embodiment is denoted as . And includes the following discriminant features: reflectivity anomaly Examples include the ratio of highlighted and saturated pixels in an image, the ratio of locally strong reflective connected components, and occlusion ratio indicators. Examples include the visible pixel percentage of the opponent or white line candidate area, and the critical edge breakage rate. The formula for calculating visual credibility is:

[0068] ;

[0069] in, This represents a numerical truncation function. and This represents the weighting coefficient.

[0070] It should be noted that when a visual error is detected as frame dropping or severe delay, a time expiration penalty must be explicitly included in the visual credibility, and the visual credibility decreases as the duration of the time expiration penalty increases.

[0071] Specifically, the IMU reliability in this embodiment is denoted as... And it includes the following discriminative features: IMU self-test status A value of 1 indicates a pass in the self-test, otherwise a value of 0 indicates a noise level. It is obtained by normalizing statistics such as the variance of the angular velocity or acceleration sequence within the short window; saturation time proportion It is obtained by statistically analyzing the proportion of samples that reach the upper limit of the measurement range within a short window; drift index The reliability is obtained by normalizing the variation range of the zero-bias estimate within the long window. IMU reliability is primarily used for attitude stability assessment and time axis alignment reliability judgment. The formula for calculating IMU reliability is:

[0072] ;

[0073] in, , , and Indicates the weighting coefficient. , The larger the value, the more reliable the IMU data, and the more reliable the time axis alignment and attitude stability judgment.

[0074] It should be noted that when IMU sample loss occurs, the sample loss penalty should be included in the IMU reliability.

[0075] Specifically, the ultrasonic reliability in this embodiment is denoted as . It includes the following discriminant features: multipath anomaly degree Examples include the number of distance jumps within a short window and inconsistencies with IMU attitude changes; empty echo anomalies. For example, the proportion of invalid echoes. The formula for calculating the reliability of ultrasound is:

[0076] ;

[0077] in, and This represents the weighting coefficient.

[0078] It should be noted that when the ultrasonic input in this sensing cycle is marked as invalid, the proportion of invalid echoes or the number of consecutive invalid echoes should be included in the ultrasonic reliability.

[0079] Specifically, the infrared confidence level in this embodiment is denoted as... And includes the following discriminant features: white line reflectance saturation discriminant. For example, the intensity of white line reflection within a short window The percentage of items reaching the upper limit; ground material drift discrimination value, etc. For example, the drift of the background reflection baseline over time, the gradient of the white line edge. Long-term offsets, etc. Infrared reliability is mainly used to identify conditions such as overexposure of white line reflections, changes in ground material, and interference light. The formula for calculating infrared reliability is:

[0080] ;

[0081] in, and This represents the weighting coefficient.

[0082] It should be noted that if the infrared data for the current sensing cycle has not been updated, a missing sample penalty should be included in the infrared confidence level.

[0083] S4. Input the feature sequence, confidence index, and the current state vector of the arena robot into the dynamic attention network. The dynamic attention network outputs a fused weight vector. and the set of scheduling instructions ;in, , Represents the weights of the visual sensors. Indicates the weights of the IMU sensors. Indicates the weight of the ultrasonic sensor. The weights of the infrared reflective sensor are represented, and .

[0084] Specifically, the feature sequence in this embodiment includes visual features, IMU features, ultrasonic features, and infrared features in the short and long windows; the credibility index includes visual credibility, IMU credibility, ultrasonic credibility, and infrared credibility; the state vector includes the task objective (e.g., attack, defense, pursuit, risk avoidance), real-time linear velocity, real-time angular velocity, real-time acceleration, the white line risk level of the previous sensing cycle, and necessary pattern markers, etc.

[0085] It should be noted that the dynamic attention network can use a learnable model that can output attention weights based on temporal features and credibility, and there is no limit to the specific number of network layers; however, its output must satisfy the above-mentioned formal constraints of weights and three linkages.

[0086] The dynamic attention network used in this embodiment will be described in detail below.

[0087] 1) Network Input Definition

[0088] The feature sequence of a short window is defined as follows:

[0089] ;

[0090] in, Indicates the short window at the center time The feature sequence represents the length of the short window, for example, equal to 10 perception cycles.

[0091] The feature sequence of a long window is defined as follows:

[0092] ;

[0093] in, Indicates the long window at the center time The feature sequence represents the length of the long window, for example, equal to 50 perception cycles.

[0094] The credibility index is defined as:

[0095] ;

[0096] in, Indicates the center time Credibility metrics Indicates the center time Visual credibility Indicates the center time Infrared reliability, Indicates the center time The reliability of ultrasound. Indicates the center time The credibility of the IMU , , and All values ​​are normalized to the interval, and the larger the value, the more reliable it is.

[0097] The state vector is defined as:

[0098] Center moment The state vector is denoted as .

[0099] 2) Network Structure

[0100] The dynamic attention network consists of a temporal coding subnetwork, a state fusion subnetwork, a credibility gating subnetwork, and an attention weight generation subnetwork.

[0101] Temporal coding subnetwork: which respectively handles the short window at the center time Feature sequences With long window at center time Feature sequences Perform timing encoding to obtain the short window hidden state. With long window hidden state .

[0102] State fusion subnetwork: It hides the short window state Long window hidden state and center moment state vector The splicing and fusion representation are obtained through a fully connected layer. .

[0103] Trustworthy Gated Subnetwork: Central Moment Credibility index Input the confidence-gated subnetwork to obtain the gating coefficients. All gating coefficients are within the range and are used to suppress or enhance the sensor attention score.

[0104] Attention weight generation subnetwork: based on fusion representation Calculate the unnormalized attention score for each sensor. And combined with the gating coefficient Attention score is obtained by performing credibility modulation. .

[0105] 3) Fusion weight vector

[0106] Attention score Input to a normalization layer (Softmax), output fused weight vector ,Right now Thus making .

[0107] Fusion weight vector Used for temporal fusion of features from multiple sensors, and to increase the weight of infrared reflective sensors when the risk of white lines increases or the infrared confidence level is high. Increase the value to enhance the contribution of infrared white line detection to the final boundary determination.

[0108] 4) Set of scheduling instructions

[0109] Scheduling instruction set This is used to generate three-linkage scheduling instructions for each type of sensor: a switching instruction, a sampling frequency setting instruction, and a resolution or quantization accuracy setting instruction.

[0110] Specifically, each scheduling instruction uses a category header to output the gear index. For example, the switch instruction uses a two-category output. These correspond to off and on respectively; the sampling frequency level command uses a three-category output. These correspond to low, medium, and high settings, respectively; resolution or quantization precision settings are output using a three-category output. These correspond to low-end, mid-range, and high-end, respectively.

[0111] Taking an infrared reflective sensor as an example, its sampling frequency range is: Quantization accuracy level is The frequency range indices 0 / 1 / 2 correspond to 20Hz / 50Hz / 100Hz respectively, and the precision range indices 0 / 1 / 2 correspond to 8bit / 10bit / 12bit respectively. The three-linkage scheduling command for the infrared reflective sensor is represented as follows:

[0112] ;

[0113] in, , , .

[0114] The visual sensor, ultrasonic sensor, and IMU sensor also adopt the same structure to output their three-linkage scheduling commands.

[0115] 5) Training Objectives

[0116] The dynamic attention network is trained using multi-task loss, including loss for white line boundary crossing risk prediction or boundary state classification, opponent orientation / relative motion estimation loss (e.g., azimuth error, velocity error), and scheduling cost regularization terms (e.g., frequency and accuracy ramp-up penalties, on / off switching penalties, used to balance performance and resource consumption). Training is only used to obtain network parameters, enabling the network to adaptively output fused weight vectors under different adversarial scenarios. and the set of scheduling instructions .

[0117] 6) Reasoning process

[0118] Within each perception cycle, the dynamic attention network uses a short window-based feature sequence. Feature sequences of long windows State vector Credibility Indicators Output fusion weight vector and the set of scheduling instructions .

[0119] S5. Calculate the risk level of the white line boundary. ; where is the normalized quantity and .

[0120] Specifically, the risk level of the white line boundary in this embodiment Including white line near component , movement trend component and consistency risk components The white line is close to the component. It is estimated from the relative position or distance of the infrared white line; the closer to the white line, the better. The larger; the greater the trend component. It is obtained from the velocity component and angular velocity of the arena robot towards the white line. For example, when the velocity component of the arena robot towards the white line increases... Increase; Consistency risk component It is calculated from the deviation between the infrared white line distance detected by the infrared reflective sensor and the visual white line distance detected by the visual sensor.

[0121] The calculation process for the risk level of the white line boundary is explained in detail below.

[0122] 1) Definition of coordinates and basic quantities

[0123] The coordinate system of the arena robot is as follows: the center of mass of the arena robot is the origin, the forward direction is the positive X-axis, and the left side is the positive Y-axis.

[0124] Infrared white line distance: The relative distance or offset of the white line output by the infrared reflective sensor, denoted as... .

[0125] Visual white line distance: The relative distance or offset of the white line calculated by the visual sensor from the candidate features of the white line, denoted as .

[0126] Deviation threshold: The threshold used to determine whether the distance to the infrared white line is consistent with the distance to the visual white line, denoted as .

[0127] Maximum attention distance: Distance threshold, denoted as When the distance to the white line is greater than or equal to this distance threshold, the risk of the white line is very low, and the white line is close to the component. Approaching 0.

[0128] It should be noted that the distance to the white line refers to the shortest distance from the arena robot to the white line or the forward distance from the arena robot to the white line along its current direction of travel.

[0129] 2) Infrared white line distance Calculation

[0130] Specifically, in this embodiment, the infrared reflective sensor is a multi-channel array mounted on the bottom leading edge or side edge of the arena robot. The position of each channel in the arena robot's coordinate system is as follows: Take the gradient along the edge of the white line. The largest channel is used as the location of the white line edge, and linear interpolation is used to estimate the lateral position of the edge. Then the distance of the infrared white line Calculated using the following formula:

[0131] ;

[0132] in, The horizontal coordinates of the reference point of the arena robot are indicated. The reference point is the center of mass of the arena robot or the center of the multi-channel array.

[0133] 3) Distance from the visual white line Calculation

[0134] Edge extraction and line segment detection are performed on the candidate regions for the white line in the visual image to obtain the line segment representation of the white line in the visual image. Combined with calibration parameters such as camera focal length, pitch angle, and mounting height, the white line is projected onto the ground to obtain an approximate representation of the white line in the arena robot's coordinate system. The distance from the arena robot's reference point to the white line is then calculated as... .

[0135] It is important to note that when visual credibility... At lower levels, the visual white line distance The output will still be displayed, but it will be in the consistency risk component. It was suppressed during subsequent fusion.

[0136] 4) Risk level of the white line boundary Calculation of the three components

[0137] 4.1) White line is close to component

[0138] Based on infrared white line distance and maximum attention distance Calculate the white line proximity component :

[0139] ; This is a limiting function;

[0140] like (If it touches the edge of the white line), then ;

[0141] like ,but .

[0142] 4.2) Motion trend component

[0143] Define the velocity component of the arena robot toward the white line as: Calculated using the following formula:

[0144] ;

[0145] in, This represents the velocity vector of the arena robot in the ground plane. This represents the unit vector of the normal to the white line, estimated from the direction of the white line. , .

[0146] like If the white line is far away, the risk is 0; if... The symbol indicates that the closer you are to the white line, the greater the risk becomes with increasing speed.

[0147] Based on velocity components Calculate the motion trend components :

[0148] ;

[0149] in, This represents the preset maximum speed, used for normalization.

[0150] In addition, to account for the risk of lateral displacement caused by sharp turns, the angular velocity of the arena robot's yaw can also be introduced. Calculate the motion trend components :

[0151] ;

[0152] in, This indicates the preset maximum angular velocity. As a weighting factor, Preferred .

[0153] 4.3) Consistency Risk Components

[0154] Based on infrared white line distance Visual white line distance and deviation threshold Calculate the consistency risk component :

[0155] ;

[0156] like ,but The risk is relatively small; if ,but The risk is relatively high.

[0157] 5) Risk level of the white line boundary The weighted normalization method is used to obtain the following calculation formula:

[0158] ;

[0159] in, , and To preset weights, and Preferred .

[0160] To trigger the white line security state machine and achieve verifiable boundary protection, this embodiment calculates the white line boundary risk level in each sensing cycle. The larger the value, the closer it is to the boundary white line and the greater the risk of crossing the boundary.

[0161] S6. Set up a security policy layer independent of the dynamic attention network. The security policy layer stores and executes a set of white-line security state machines to manage the fused weight vectors. and the set of scheduling instructions Impose inviolable hard constraints.

[0162] It is easy to understand that the purpose of setting up the security policy layer is to ensure that even if the output of the dynamic attention network fluctuates or makes scheduling that is detrimental to whiteline protection when resources are limited, it must be "hard-covered" by the security policy layer.

[0163] Specifically, the white-line safety state machine in this embodiment has preset discrete states and a set of fixed parameters. The discrete states include the normal state. Alert status and dangerous situation The solidification parameter set includes a first threshold. Second threshold hysteresis First Lock-up Period Second Locking Period , and The white-line safety state machine also presets the following state transition rules:

[0164] when Enter alert status ,when Entering a dangerous state ;

[0165] When in a dangerous situation And in the second locking period Internal continuous satisfaction At that time, it is permissible to leave the dangerous state. De-escalation to alert status ;

[0166] When in a state of alert And in the first locking period Internal continuous satisfaction At that time, it is permissible to leave the state of alert. Return to normal status .

[0167] It should be noted that hysteresis First Lock-up Period Second Locking Period This is used to avoid jitter caused by frequent switching near the threshold.

[0168] More specifically, in this embodiment, , , , , .

[0169] To ensure verifiable security, this embodiment directly triggers a hazard level when there is a strong discrepancy between the distance estimates of the white line by the infrared reflective sensor and the visual sensor.

[0170] When satisfied ,but ;

[0171] in, This indicates the updated risk level of the white line boundary. This indicates the risk level of the white line before the update.

[0172] When consistency risk component Greater than the deviation threshold At that time, the risk level of the white line boundary will be determined. Raise to the second threshold and above, to enter a dangerous state. .

[0173] To reduce rapid fluctuations caused by measurement noise, this embodiment employs exponential smoothing when no consistency hard trigger occurs, i.e.:

[0174] ;

[0175] in, Preferred .

[0176] When a consistent hard trigger occurs, the hard trigger result should be taken as the standard, and exponential smoothing should not be performed to ensure that safety takes priority.

[0177] Furthermore, the white-line safety state machine in this embodiment also presets the following operating rules: the sampling frequency and quantization accuracy of the infrared reflective sensor are both preset to three levels: low, medium, and high; when in alert state... At that time, the sampling frequency and quantization accuracy of the infrared reflective sensor are both selected to be medium or high, and are maintained at least during the first locking period. Internal gear lock-up, weight of infrared reflective sensor When in a dangerous situation At that time, the sampling frequency and quantization accuracy of the infrared reflective sensor are both selected to be high-end and are maintained at least during the second locking period. Internal gear lock-up, weight of infrared reflective sensor When the action rule conflicts with the output of the dynamic attention network, the action rule shall prevail.

[0178] For example, the sampling frequency range set of the infrared reflective sensor in this embodiment is as follows: The set of quantization precision levels is When in a state of alert At that time, the sampling frequency of the infrared reflective sensor is selected or And the quantization accuracy is selected or When in a dangerous situation At that time, the sampling frequency of the infrared reflective sensor is selected And the quantization accuracy is selected .

[0179] It should be noted that downshifting is prohibited during gear lock. The gear lock sensing cycle is counted by a lock counter, and the count value decreases with each sensing cycle until it reaches 0.

[0180] Furthermore, the security policy layer in this embodiment also stores and executes resource-limited judgment rules and resource priority downgrade rules; the resource-limited judgment rule is: when CPU utilization is high... or bus bandwidth utilization or sensor power consumption utilization When resources are limited, the system is deemed to be in a resource-constrained state. The resource priority degradation rule is as follows: Under resource-constrained conditions, priority is given to ensuring that infrared reflective sensors meet the action rules, and visual sensors and / or ultrasonic sensors are forcibly degraded. When a visual sensor is degraded, its frame rate is reduced to 30fps or below, or its resolution is reduced to 640×480 or below. When an ultrasonic sensor is degraded, its weight is reduced to... .

[0181] Furthermore, the security policy layer in this embodiment also stores and executes trustworthiness rules; the trustworthiness rules are as follows: when the visual trustworthiness is below the visual threshold and the system is in an alert state... or dangerous situation When the security policy layer degrades the visual sensor, it does so when the ultrasonic confidence level is below the ultrasonic threshold and the system is in an alert state. or dangerous situation When using ultrasonic ranging results to reduce the risk level of white line boundaries, it is prohibited. The weights of the ultrasonic sensors are determined by the security policy layer. Apply upper limit constraints When the infrared confidence level is below the infrared threshold and the system is in a state of alert. or dangerous situation At that time, the security strategy layer maintains the constraint on the infrared reflective sensor and forces a retest action to be performed in the next sensing cycle. The retest action is to keep at least one of the sampling frequency and quantization accuracy of the infrared reflective sensor at the current level or increase it by one level.

[0182] It should be noted that the dynamic attention network in this embodiment outputs a fusion weight vector in each perception cycle. and the set of scheduling instructions However, since this embodiment also has an independent security policy layer, which executes a white-line security state machine, when the output of the dynamic attention network conflicts with the state transition rules, action rules, resource priority degradation rules, or trustworthiness rules, the security policy layer must overwrite and correct the output of the dynamic attention network.

[0183] Below is a coverage correction process that can be directly implemented in engineering and can be tested and accepted.

[0184] 1) Priority principle of overlay correction

[0185] Highest priority: State transition rule; Second highest priority: Resource priority downgrade rule; Third highest priority: Credibility rule; Lowest priority: Output of dynamic attention network.

[0186] When a low-priority output conflicts with a high-priority rule, the output must be overridden, and the overridden output must satisfy the constraints that "the total weight is 1, each weight is non-negative, and the level belongs to a preset set".

[0187] 2) Fusion weight vector Overlay correction

[0188] Fusion weight vector It consists of four components: the weights of the visual sensor. Weights of IMU sensors Weight of ultrasonic sensors Weight of infrared reflective sensors .

[0189] Fusion weight vector The overwrite correction is divided into the following steps.

[0190] Step 1: The priority is fixed.

[0191] Based on the current white line safety state machine judgment Minimum requirement: When in a state of alert hour, Not lower than the first lower limit (defined as 0.6 in this embodiment); when in a dangerous state hour, Not lower than the second lower limit (defined as 0.8 in this embodiment).

[0192] Compare the output of the dynamic attention network with the aforementioned lower bound: if the output of the dynamic attention network is greater than or equal to the aforementioned lower bound, then... The output value of the dynamic attention network is retained; if the output of the dynamic attention network is less than the aforementioned lower limit, then... The value is forcibly raised to the aforementioned lower limit, and this raising operation is a hard overwrite, which cannot be reversed by other rules.

[0193] Step 2: Reorganization of the remaining weights.

[0194] when Once determined, the remaining allocable weights are equal to During restructuring, it is necessary to ensure that the sum of all weights is 1.

[0195] The reorganization of the remaining weights is divided into two cases: regular reorganization and extreme reorganization.

[0196] Regular remodeling: If the output of the dynamic attention network... , and If at least one of the three values ​​is positive, then the relative proportions of the three remain unchanged, and the entire result is scaled up until the remaining assignable weights are filled. After scaling, if there is a slight error between the sum and 1 due to numerical precision, the error is uniformly compensated to the maximum value among the three. This ensures that the total weight is strictly equal to 1.

[0197] Extreme remodeling: If the output of the dynamic attention network... , and If all values ​​are 0, an extreme situation may occur where the remaining weights are left unassigned. In this case, a pre-set fallback allocation method must be used: for example, prioritizing allocation to... and For example, a fixed-ratio allocation can be adopted, while satisfying the aforementioned rules regarding the upper limit of weights. Once the fallback allocation scheme is determined, it should be solidified to ensure reproducibility during acceptance testing.

[0198] Step 3: Overlay " The second round of trimming involves "weight cap" and "resource-limited downgrade".

[0199] If the reorganization If the weight exceeds its weight limit, it will be pruned back to the weight limit; the excess weights will be redistributed. and This ensures that the total weight is 1.

[0200] If both are in a resource-constrained state, a further degradation tendency will be implemented: priority will be given to reducing [the resource level]. Or the visual setting; if necessary, it can be lowered simultaneously. To meet the upper limit.

[0201] Step 4: Legality verification and recording.

[0202] After the coverage correction is completed, the security policy layer merges the weight vector. Perform a validity check and record the diagnostic quantity.

[0203] The verification items include: Does the current white line safety state machine meet the lower bound constraints? Are all weights non-negative? Is the sum of weights 1? Does it meet the upper limit constraint? Diagnostic quantities include: pre-coverage... After covering Discrete states of the white line safety state machine, whether resources are limited, whether the weight cap is triggered, and the current white line boundary risk level. In addition, the remaining cycle of the gear position and gear lock of the infrared reflective sensor, diagnostic quantities are used for online adaptive or offline training in order to audit and reproduce experiments.

[0204] 3) Set of scheduling instructions Overlay correction

[0205] Scheduling instruction set A three-way scheduling command is executed on the output of each sensor: a switch command, a sampling frequency setting command, and a resolution or quantization accuracy setting command. (Set of scheduling commands) The coverage correction follows the order of "infrared first, then others", and includes the following steps.

[0206] Step 1: Hard overlay of infrared dispatch commands.

[0207] If the discrete state is a warning state If the infrared reflective sensor is forced to turn on, both the sampling frequency and quantization accuracy levels must be at least at the medium level; if the discrete state is a dangerous state... If the infrared reflective sensor is forced to turn on, both the sampling frequency and quantization accuracy levels will be set to high. If the infrared reflective sensor is within its lockout period, it will be prohibited from downgrading or turning off. Only if the hysteresis release condition is met and the lockout ends will the infrared reflective sensor be allowed to adjust according to the output of the dynamic attention network, but the discrete state correspondence rules must still be met.

[0208] Step 2: Non-infrared degradation when resources are limited.

[0209] When resource constraints are determined, the security policy layer prioritizes ensuring that the status and on / off state of infrared reflective sensors are not affected, and performs degradation on other sensors: firstly, the resolution or frame rate of visual sensors is reduced; secondly, the frequency of ultrasonic sensors can be reduced or kept at a low frequency, while the weight limit is combined to reduce their impact; generally, IMU sensors are not turned off, but their output rate or filtering parameters can be reduced to reduce the load.

[0210] Step 3: Targeted coverage triggered by credibility.

[0211] When visual credibility is low and the person is in a state of alert or dangerous situation When the visual sensor is in a downgraded state, it is forced to remain at a lower level to prevent it from upgrading and monopolizing resources; when the ultrasonic sensor has low reliability and is in a state of alert. or dangerous situation At that time, the use of ultrasonic structures to reduce the risk level of white line boundaries is prohibited. It can also force its frequency to decrease; when the infrared reliability is low but it is in an alert state or dangerous situation At the same time, the infrared reflective sensor remains on and at a level no lower than the current setting, and a retest is performed.

[0212] Step 4: Verify the validity of the instruction.

[0213] The overlay-corrected scheduling instructions must meet the following requirements: the switch can only have two values, on or off; the sampling frequency level can only fall within a preset set; and the resolution or quantization accuracy level can only fall within a preset set. If the dynamic attention network outputs an illegal level, the security policy layer directly corrects it according to the "most recent legal level" or the "nearest downward legal level," thereby ensuring that the system is operational and acceptable.

[0214] S7. Based on the fusion weight vector corrected by the security policy layer overlay and the set of scheduling instructions It performs sensor scheduling and multi-sensor time-series fusion, and outputs environmental perception results.

[0215] It is easy to understand that sensor scheduling refers to following the modified set of scheduling instructions. Each sensor is configured with its on / off status, sampling frequency level, and accuracy level (resolution or quantization accuracy level). Multi-sensor temporal fusion refers to feature fusion performed separately in short and long windows, such as weighted fusion of similar semantic features. Infrared is prioritized for white line-related output. Opponent information is fused from visual, IMU, and ultrasonic sensors to obtain more stable orientation and relative velocity estimates. Environmental perception results include opponent orientation and relative motion information, relative position and distance of white line boundaries, and boundary crossing risk level.

[0216] The following sections provide a detailed explanation of the three-linkage gear set of the vision sensor, IMU sensor, and ultrasonic sensor.

[0217] 1) Three-linkage gear set of vision sensor

[0218] Switching commands: The visual sensor employs binary control of "on / off". In ring combat, to ensure the continuous availability of opponent's positional information, the visual sensor is usually kept on; when resources are limited or visual reliability is low, its load can be reduced according to the security policy layer degradation rules, and it can only be temporarily turned off when necessary.

[0219] Sampling frequency level instructions: The sampling frequency (frame rate) of the vision sensor adopts a discrete set of levels, for example... , For low-end applications and intended for minimal load monitoring or rough target data. It is a mid-range solution for general combat, balancing real-time performance and load. It is designed for high-end applications and improves the refresh rate during high-speed combat, rapid movement, or turning.

[0220] Resolution level instructions: The resolution of the vision sensor uses a discrete set of levels, for example... , For lower-end applications and those using resource-constrained, highly reflective, or noisy environments, this reduces unwanted details. It is a medium speed and is used for balance. It is a high-end product used for fine-grained hand recognition or white line candidate feature enhancement.

[0221] Explanation of frame rate settings: A higher frame rate means faster image updates, but also requires more computation; a higher resolution means clearer details, but also requires more computation. Therefore, the sampling frequency and resolution settings of a visual sensor allow the system to make a controllable trade-off between fast viewing, clear viewing, and resource conservation.

[0222] Coordination rules with the security policy layer: When the white-line security state machine is in an alert state or dangerous situation If resources are limited, the frame rate or resolution of the visual sensor will be reduced first to ensure that the infrared reflective sensor still meets its hard constraint level. When the visual reliability is low, the security strategy layer can also force the visual sensor to downgrade to avoid the visual sensor consuming resources and interfering with the fusion when it is unreliable.

[0223] 2) Three-linkage speed setting of ultrasonic sensor

[0224] Switching commands: The ultrasonic sensor uses binary "on / off" control. It remains on during close-quarters combat; however, it will be switched off when multipath or void echoes are severe and the system is in alert mode. or dangerous situation In such cases, the impact can be downgraded or reduced according to the rules of the security policy layer.

[0225] Sampling frequency setting instructions: The sampling frequency of the ultrasonic sensor is also the trigger frequency, which uses a discrete set of settings, for example... , For low-end applications and use with low load and coarse distance sensing. It is a medium speed and is used for balance. It is designed for high-end applications and is used when rapid verification of distance changes is required.

[0226] Resolution level instructions: Since the ultrasonic sensor outputs distance values ​​rather than image resolution, this embodiment maps the resolution level to the echo processing accuracy level. The discrete level set can adopt any one or a combination of mapping scheme A, mapping scheme B and mapping scheme C.

[0227] Mapping scheme A represents the different lengths of the echo sampling window: the low setting is a short window, which saves more computing power but has weak noise resistance; the medium setting is a medium window; and the high setting is a long window, which is more noise-resistant and more stable but requires more computation and has a slightly slower response.

[0228] Mapping scheme B represents the filtering levels: low level is weak filtering, which has a fast response but large jitter; medium level is medium filtering; high level is strong filtering, which has a stable output but is not sensitive to transients.

[0229] Mapping scheme C represents the echo consistency check strength levels: low level means no or few consistency checks; medium level means basic consistency checks, such as accepting only after two consecutive consistent checks; high level means strict consistency checks, such as accepting only after multiple consecutive consistent checks.

[0230] Explanation of the setting: Ultrasonic sensors can sometimes misinterpret echoes: the echo may originate from a reflection off a wall or edge rather than from a target directly in front. Increasing the echo processing accuracy setting is equivalent to more rigorously determining the reliability of the echo, but at the cost of higher processing time and computational power.

[0231] Coordination rules with the security policy layer: When the white-line security state machine is in an alert state or dangerous situation Furthermore, when the reliability of the ultrasonic wave is low, the security strategy layer can force the ultrasonic sensor to switch to a low-frequency or mid-frequency setting and increase the echo processing accuracy setting to perform verification; at the same time, it can limit the impact of the ultrasonic sensor on the fusion and prevent its results from reducing the risk of the white line boundary. This avoids boundary misjudgment caused by ultrasonic sensors mishearing.

[0232] 3) Three-linkage mode set of IMU sensor

[0233] Switching commands: The IMU sensor uses binary "on / off" control. It is kept on for extended periods during intense competition; it is only considered to be turned off in extreme resource-constrained or faulty modes, but usually a better approach is to reduce the output rate or adjust the filter to alleviate the load.

[0234] Sampling frequency range command: The sampling frequency of the IMU sensor, i.e., the output rate, is set using a discrete range, for example... , For low-end applications and use with low load and basic attitude stabilization. It is a medium speed and is used for balance. It is a high-end product designed to improve dynamic response under conditions of sudden turns, sudden stops, and strong impacts.

[0235] Resolution level instructions: Since IMU sensors typically sample with fixed hardware precision, this embodiment maps resolution levels to IMU precision levels. The discrete level set can be any one or a combination of mapping schemes C, D, and F.

[0236] Mapping scheme A represents the low-pass filter cutoff levels: low level is low cutoff, resulting in a smoother output and stronger vibration resistance, but with a slightly slower response; medium level is medium cutoff; high level is high cutoff, resulting in a faster response, but with greater sensitivity to vibration.

[0237] Mapping scheme B is the range configuration level: low range is a large range mode, which is not easy to saturate and is suitable for strong impacts, but the fine resolution is lower; medium range is a medium range mode; high range is a small range mode, which has higher fine resolution, but is prone to saturation in strong impacts.

[0238] Explanation of gear settings: The output rate of the IMU sensor determines how fast the attitude update is; the IMU accuracy settings can be understood as the choice of "smoother / more sensitive": when the impact of the ring is large, it is necessary to prevent saturation and resist vibration; when pursuing fine control, higher resolution or faster response is required.

[0239] Coordination rules with the security policy layer: When the white-line security state machine is in an alert state or dangerous situation To ensure stable boundary decisions, IMU sensors are typically kept on. When resources are limited, the IMU sensor's output rate can be reduced or the low-pass filter cutoff setting adjusted to alleviate the load, but this should not come at the expense of the hard constraints of the infrared reflective sensor. If a strong impact is detected that causes an increase in IMU sensor noise, stability can be improved by increasing the low-pass filter cutoff setting or switching the range configuration setting, and this change should be reflected in the reliability index.

[0240] S8. Feed back the environmental perception results to the arena robot and update the historical timeline window for the next perception cycle.

[0241] It should be noted that the environmental perception results are fed back to the motion control module of the arena robot to form a closed loop: when the white line safety state machine is in a normal state... When the white line security state machine is in an alert state, it is permitted to execute conventional attack and defense strategies. When the white line defense line is in danger, the motion control module reduces the velocity component towards the white line defense line, performing a slight rotation or lateral movement; when the white line safety state machine is in a dangerous state... At this time, the motion control module performs forced avoidance actions, such as quickly reversing or turning away from the white line to avoid crossing the boundary.

[0242] It's easy to understand that updating the historical time-series window means writing the features and state of the current perception cycle into the short and long windows for calculation in the next perception cycle. Updating the historical time-series window requires overwriting diagnostic records: when the security policy layer undergoes overwrite correction, the system records the diagnostic data; this record facilitates debugging and reproduction during competitions and can also be used for improving the dynamic attention network in offline training, without affecting the verifiability of the security policy layer.

[0243] Specifically, the environmental perception results in this embodiment include reference orientation information, opponent orientation information, and white line orientation information. The following is a detailed description of each orientation information.

[0244] 1) Baseline Position Information: The arena robot uses its center of mass as the origin of the coordinate system. The forward direction is defined as forward, and the left side of the forward direction is defined as left. Relative positions on the ground plane are represented using a two-dimensional coordinate system of "forward-left". 0° is recorded when the opponent is directly in front; a positive angle is when the opponent is to the left front; and a negative angle is when the opponent is to the right front. Distances are expressed in mm or cm, angles in °, and speeds in m / s.

[0245] 2) Opponent's location information: Opponent's relative azimuth angle, that is, the angle of deviation of the opponent from the direction directly in front of the arena robot (e.g., 20° to the left, 15° to the right, etc.); Opponent's relative distance, that is, the distance of the opponent from the arena robot; Opponent's relative position components, which include at least the forward distance and leftward offset of the opponent relative to the arena robot; Opponent's tracking confidence, used to indicate whether the location results are reliable.

[0246] It should be noted that the opponent's location information is primarily obtained from visual sensors, supplemented by IMU sensors, and further supplemented by ultrasonic sensors. Details are as follows.

[0247] Detect adversary targets in visual images and obtain their position (e.g., target box center point) and size (e.g., target box area). If multiple target candidates are detected, the adversary target is selected based on target size, motion continuity, and adversarial scenario rules (e.g., prioritizing the largest or most stable target).

[0248] The vision system pre-calibrates the camera (e.g., focal length, mounting posture, etc.), and then calculates the opponent's relative azimuth angle based on the opponent's left and right offset in the image.

[0249] The distance to the opponent (including the relative distance and relative position component) can be obtained by considering the opponent's scale changes in the image or by combining ultrasonic ranging. The opponent's distance can be calculated using any of the following methods: visual scale estimation, i.e., the larger the target is in the image, the closer it is; visual + ultrasonic fusion, i.e., the ultrasonic sensor provides proximity assistance, and the visual sensor provides direction.

[0250] 3) Opponent's motion information: Approaching / moving trend, whether the opponent is approaching or moving away from the arena robot, can be output with discrete labels, such as approaching, moving away, not obvious, etc.; Left and right movement trend, whether the opponent is moving to the left, moving to the right, or basically not moving relative to the arena robot; Relative speed level, outputting three speed levels: low / medium / high, or outputting continuous speed values; Motion confidence, used to indicate whether the motion judgment is reliable, for example, it is reduced when visual frames are dropped.

[0251] When the robot in the arena makes a rapid turn or is subjected to impact vibration, relying solely on the visual sensor will result in image shift and orientation jitter. This embodiment uses the angular velocity and attitude changes of the IMU sensor to compensate for the output of the visual sensor: that is, correcting the directional change seen by the camera to the true direction after the robot rotates itself, so that the opponent's orientation remains continuous and stable even during rapid turns.

[0252] Short window smoothing and consistency check: The opponent's position is smoothed within the short window to avoid jumps caused by false detections in a single frame; if the visual confidence is low, the visual contribution of this perception cycle is reduced, the stable tracking results of the previous perception cycle are maintained first, and the opponent tracking confidence is reduced.

[0253] In a short window, continuously changing relative positions of the opponent are compared: if the forward distance continuously decreases, the opponent is judged to be approaching; if the forward distance continuously increases, the opponent is judged to be moving away; if the leftward offset continuously increases, the opponent is judged to be moving to the left, and vice versa. When visual frames are dropped or visual confidence decreases, a conservative strategy of maintaining the previous stable trend and reducing motion confidence can be adopted to avoid misleading the motion control module.

[0254] 4) White line boundary information: The relative distance of the white line, i.e. the distance from the arena robot to the white line boundary, is calculated by the infrared reflective white line detection sensor; the white line orientation label, i.e., whether the white line boundary is on the left, right, or in front of the arena robot, can output discrete labels for easy use of control strategies; the white line detection confidence is given based on infrared confidence and detection continuity; the white line consistency flag, i.e. whether the detection results of the infrared reflective sensor and the vision sensor are consistent, is used for diagnosis and hard trigger upgrades.

[0255] The relative distance to the white lines is estimated based on the distance from the center of the infrared array to the boundary of the white lines; when in alert status... or dangerous situation At this time, the infrared reflective sensor remains on and meets the high-level setting to ensure reliable output of the relative distance between the white lines.

[0256] The detection of white line boundaries by visual sensors is mainly used to enhance vigilance when the detection results of visual sensors and infrared reflective sensors are inconsistent.

[0257] Output stability: When the infrared reflective sensor experiences a short-term loss of sample, the most recent valid white line distance can be used, but the white line detection confidence must be reduced, and a retest and lockout strategy must be triggered to recover as soon as possible.

[0258] The output definition of the boundary risk level: In this implementation, the boundary risk level directly corresponds to the discrete states of the white-line safety state machine, so that the motion control module and acceptance personnel can understand: low risk corresponds to normal state. Medium risk indicates that the robot is far from the white line boundary or its movement trend will not approach the white line, allowing it to execute normal offensive and defensive strategies; medium risk corresponds to an alert state. This indicates that the robot in the arena has approached or is moving towards the white line, triggering a warning system. At this point, the motion control module should limit the velocity component moving towards the white line and prepare to avoid it. Simultaneously, the infrared reflective sensor is forcibly activated at a medium-high sampling frequency and quantization accuracy level. High risk corresponds to a dangerous state. This indicates that the robot may have crossed the boundary or that the infrared reflective sensor and the visual sensor may have mismatched their detection results of the white line, triggering a hazard. In this case, the motion control module should execute a forced avoidance maneuver, while the infrared reflective sensor is forced to operate at high frequency and high precision with a locked cycle. The risk level output in this way not only directly drives the control strategy but also serves as a core indicator for system logs and test acceptance.

[0259] Structured encapsulation and logging of output results: To facilitate debugging and verification, this embodiment also packages the environmental perception results into structured information, including at least: opponent location information, namely opponent relative azimuth angle, opponent relative distance, opponent relative position component, and opponent tracking confidence; opponent movement information, including approach / distance trend, left and right movement trend, relative speed level, and movement confidence; white line boundary information: white line relative distance, white line azimuth label, white line detection confidence, and white line consistency flag; boundary crossing risk level, namely low risk, medium risk, and high risk.

[0260] The above are merely specific embodiments of the present invention, enabling those skilled in the art to understand or implement the present invention. Although detailed descriptions have been provided with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments, and they should all be covered within the protection scope of the claims.

Claims

1. A method for environmental perception of a ring robot based on multi-sensor temporal fusion, characterized in that, This method is based on a ring robot equipped with a visual sensor, an IMU sensor, an ultrasonic sensor, and an infrared reflective sensor, and it updates iteratively at a preset perception cycle. Within each perception cycle, the method performs the following steps: S1. The visual sensor acquires images of the detection area in front of the arena robot and generates a visual perception data sequence. The IMU sensor acquires inertial data of the arena robot and generates an IMU perception data sequence. The ultrasonic sensor acquires distance signals between the arena robot and its opponent or obstacle and generates an ultrasonic perception data sequence. The infrared reflective sensor acquires white line reflection signals of the arena boundary and generates an infrared perception data sequence. Each perception data sequence is accompanied by a corresponding acquisition timestamp. A unified time axis is established based on the acquisition timestamps and time-domain alignment is performed to generate a synchronous observation sequence. S2. Extract visual features, IMU features, ultrasonic features, and infrared features from the synchronous observation sequence, and form a feature sequence within the historical time window; S3. Calculate the reliability index of each sensor to obtain the reliability of vision, IMU, ultrasound and infrared. S4. The feature sequence, the credibility index, and the current state vector of the arena robot are input into a dynamic attention network, which outputs a fusion weight vector. and the set of scheduling instructions ; in, , Represents the weights of the visual sensors. Indicates the weights of the IMU sensors. Indicates the weight of the ultrasonic sensor. The weights of the infrared reflective sensor are represented, and ; S5. Calculate the risk level of the white line boundary. ; in, For normalized quantity and ; Risk level of the white line boundary Including white line near component , movement trend component and consistency risk components ; S6. Establish a security policy layer independent of the dynamic attention network. This security policy layer stores and executes a set of white-line security state machines to manage the fused weight vector. and the set of scheduling instructions Impose inviolable hard constraints; S7. Based on the fusion weight vector corrected by the security policy layer overlay and the set of scheduling instructions It performs sensor scheduling and multi-sensor time-series fusion, and outputs environmental perception results. S8. Feed back the environmental perception results to the arena robot and update the historical time sequence window for the next perception cycle.

2. The method for environmental perception of a ring robot based on multi-sensor temporal fusion according to claim 1, characterized in that, The white line safety state machine has preset discrete states and a set of fixed parameters. The discrete states include the normal state. Alert status and dangerous situation The curing parameter set includes a first threshold. Second threshold hysteresis First Lock-up Period Second Locking Period , and The white-line safety state machine also presets the following state transition rules: when Enter alert status ,when Entering a dangerous state ; When in a dangerous situation And in the second locking period Internal continuous satisfaction At that time, it is permissible to leave the dangerous state. De-escalation to alert status ; When in a state of alert And in the first locking period Internal continuous satisfaction At that time, it is permissible to leave the state of alert. Return to normal status .

3. The method for environmental perception of a ring robot based on multi-sensor temporal fusion according to claim 2, characterized in that, The white line safety state machine also presets the following action rules: The infrared reflective sensor has three preset levels for sampling frequency and quantization accuracy: low, medium, and high; when in alert mode... At that time, the sampling frequency and quantization accuracy of the infrared reflective sensor are both selected to be medium or high, and are maintained at least during the first locking period. The internal gear position is locked, and the weight of the infrared reflective sensor is maintained. When in a dangerous situation At that time, the sampling frequency and quantization accuracy of the infrared reflective sensor are both selected at a high level and are maintained at least during the second locking period. The internal gear position is locked, and the weight of the infrared reflective sensor is maintained. ; When the action rule conflicts with the output of the dynamic attention network, the action rule shall prevail.

4. The method for environmental perception of a ring robot based on multi-sensor temporal fusion according to claim 3, characterized in that, The security policy layer also stores and executes resource-limited judgment rules and resource priority downgrade rules. The resource-limited determination rule is: when CPU utilization is... or bus bandwidth utilization or sensor power consumption utilization At that time, it is determined that resources are limited; The resource priority degradation rule is as follows: Under resource-constrained conditions, priority is given to ensuring that the infrared reflective sensor meets the action rule, while the visual sensor and / or ultrasonic sensor are forcibly degraded. When the visual sensor is degraded, its frame rate is reduced to 30fps or below, or its resolution is reduced to 640×480 or below. When the ultrasonic sensor is degraded, its weight is reduced to... .

5. The method for environmental perception of a ring robot based on multi-sensor temporal fusion according to claim 4, characterized in that, The security policy layer covers and corrects the fusion weight vector. Afterwards, a validity check is performed and the diagnostic quantity is recorded; The diagnostic parameters include the weights of the infrared reflective sensors before and after coverage. Discrete states of the white line safety state machine, whether resources are limited, whether the weight cap is triggered, and the current white line boundary risk level. And the remaining cycle of the gear position and gear lock of the infrared reflective sensor.

6. The method for environmental perception of a ring robot based on multi-sensor temporal fusion according to claim 5, characterized in that, The visual credibility is calculated based on the reflectivity anomaly and occlusion ratio indicators. When the visual credibility is below the visual threshold and the alert state is active. or dangerous situation In such cases, the security policy layer downgrades the visual sensor.

7. The method for environmental perception of a ring robot based on multi-sensor temporal fusion according to claim 5, characterized in that, The ultrasonic reliability is calculated based on multipath anomaly and echo anomaly. When the ultrasonic confidence level is below the ultrasonic threshold and the system is in a state of alert. or dangerous situation When using ultrasonic ranging results to reduce the risk level of white line boundaries, it is prohibited. The weights of the ultrasonic sensors are determined by the security policy layer. Apply upper limit constraints .

8. The method for environmental perception of a ring robot based on multi-sensor temporal fusion according to claim 5, characterized in that, The infrared reliability is calculated based on the white line reflection saturation discrimination value and the ground material drift discrimination value; When the infrared confidence level is below the infrared threshold and the system is in a state of alert. or dangerous situation At that time, the security strategy layer maintains the constraint on the infrared reflective sensor and forces a retest action to be performed in the next sensing cycle. The retest action is to keep at least one of the sampling frequency and quantization accuracy of the infrared reflective sensor at the current level or increase it by one level.

9. The method for environmental perception of a ring robot based on multi-sensor temporal fusion according to claim 5, characterized in that, The historical time series window includes a short window and a long window, with the short window used for assessing the risk level of the white line boundary. The calculation and state transition triggering are performed, and the long window is used for sensor stability assessment and drift trend analysis.

10. The method for environmental perception of a ring robot based on multi-sensor temporal fusion according to claim 2, characterized in that, The consistency risk component The distance is calculated from the deviation between the infrared white line distance detected by the infrared reflective sensor and the visual white line distance detected by the visual sensor. When the consistency risk component When the deviation exceeds the threshold, the risk level of the white line boundary will be adjusted. Raise to the second threshold and above, to enter a dangerous state. .