Robot dynamic obstacle avoidance method and system based on machine vision

Abstract

The application discloses a robot dynamic obstacle avoidance method and system based on machine vision, and relates to the technical field of robot obstacle avoidance.The method comprises the following steps: acquiring the appearance time period of a dynamic obstacle at each spatial position, and the mapping relationship between a non-vision signal and a vision detection result; if the robot runs to a preset range of a target position at a current time and falls within the appearance time period, calculating an existence probability according to the current non-vision signal and the mapping relationship; if the existence probability falls within a preset interval, controlling the robot to move to an observation pose with the maximum information gain to reacquire the non-vision signal to update the existence probability; otherwise, before the dynamic obstacle is detected by vision, generating a dynamic risk area in a cost map and mapping a cost value based on the existence probability, and guiding a planning to avoid a path; the method has the beneficial effects that the accuracy of robot dynamic obstacle avoidance and the environmental generalization capability can be improved without depending on specific environmental geometric conditions.

Description

Technical Field

[0001] This invention relates to the field of robot obstacle avoidance technology, and in particular to a method and system for dynamic obstacle avoidance of robots based on machine vision. Background Technology

[0002] Machine vision-based dynamic obstacle avoidance is one of the core technologies in the navigation and control systems of autonomous mobile robots. These robots often operate in environments such as hospitals, office buildings, and fixed production lines, where dynamic obstacles often exhibit certain spatiotemporal patterns. For these scenarios, existing technologies have mainly developed two types of obstacle avoidance enhancement schemes:

[0003] One type is to construct a spatiotemporal dynamic map, which involves recording the historical statistical frequency of dynamic targets appearing in a specific area through long-term observation, and raising the planning cost of the corresponding area in advance during high-frequency periods to achieve preventive obstacle avoidance;

[0004] Another type involves introducing non-line-of-sight (NLoS) sensing technologies such as radio frequency or acoustics, which rely on the diffraction or multipath reflection characteristics of signals to detect concealed dynamic targets in blind spots in advance.

[0005] However, the former relies on statistical fitting of historical data, which can easily lead to inaccurate predictions and redundant planning when faced with deviations in the dynamic laws of the environment or occasional changes; the latter is highly dependent on the specific geometric structures of the environment, such as wall materials and reflection angle layout, as well as high-cost sensor hardware, which limits its generalization ability and engineering implementation in complex unstructured environments. Summary of the Invention

[0006] In view of the above-mentioned prior art, this application is hereby filed. Embodiments of this application provide a machine vision-based method and system for dynamic obstacle avoidance in robots, which can improve the accuracy and environmental generalization ability of dynamic obstacle avoidance without relying on specific environmental geometry conditions.

[0007] According to one aspect of this application, a robot dynamic obstacle avoidance method based on machine vision is provided, comprising: acquiring the occurrence time of dynamic obstacles at various spatial locations within the robot's operating area, and the mapping relationship between non-visual signals collected by the robot during the occurrence time and visual detection results; taking any of the spatial locations as target locations, if the robot runs within a preset range of the target location at the current time, and the current time falls within the occurrence time corresponding to the target location, then calculating the probability of the existence of a dynamic obstacle at the target location based on the current non-visual signals and the mapping relationship; determining whether the probability of existence falls within a preset interval representing the inability to determine whether the dynamic obstacle exists or not: if so, then calculating... The information gain of non-visual signals acquired at multiple observation poses within the preset range on the existence probability is calculated. The robot is controlled to move to the observation pose with the largest information gain, and non-visual signals are reacquired to update the existence probability until the updated existence probability moves out of the preset range or reaches a preset termination condition. If not, or if the updated existence probability moves out of the preset range or reaches the termination condition, a dynamic risk region corresponding to the target position is generated in the robot's cost map based on the current existence probability before the dynamic obstacle is visually detected. The corresponding cost value is mapped to the dynamic risk region based on the current existence probability to guide the robot in planning an avoidance path.

[0008] According to another aspect of this application, a robot dynamic obstacle avoidance system based on machine vision is provided, comprising: a data acquisition module, configured to acquire the occurrence time of dynamic obstacles at various spatial locations within the robot's operating area, and the mapping relationship between non-visual signals collected by the robot during the occurrence time and visual detection results; a probability calculation module, configured to, taking any of the spatial locations as target locations, if the robot runs within a preset range of the target location at the current time, and the current time falls within the occurrence time corresponding to the target location, calculate the probability of the existence of a dynamic obstacle at the target location based on the current non-visual signals and the mapping relationship; and a decision module, configured to determine whether the probability of existence falls within the range where the existence of the dynamic obstacle cannot be determined. A preset range for whether or not: If yes, calculate the information gain of the presence probability to the non-visual signals collected at multiple observation poses within the preset range, control the robot to move to the observation pose with the largest information gain, and re-collect non-visual signals to update the presence probability until the updated presence probability moves out of the preset range or reaches the preset termination condition; if no, or the updated presence probability moves out of the preset range or reaches the termination condition, then based on the current presence probability, before visually detecting the dynamic obstacle, generate a dynamic risk area corresponding to the target position in the robot's cost map, and map the corresponding cost value to the dynamic risk area based on the current presence probability to guide the robot to plan an avoidance path.

[0009] According to another aspect of this application, an electronic device is provided, including a memory and a processor, the memory being used to store computer-executable instructions, and the processor being used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method described above.

[0010] According to another aspect of this application, a computer-readable storage medium is provided that stores computer-executable instructions thereon, which, when executed by a processor, implement the steps of the method described above.

[0011] Compared with existing technologies, the robot dynamic obstacle avoidance method and system based on machine vision according to the embodiments of this application can calculate the existence probability based on the occurrence time and mapping relationship of dynamic obstacles, and use real-time non-visual signals to correct historical statistical patterns to alleviate prediction inaccuracies. At the same time, when the existence probability falls into a preset range, the robot is controlled to move towards the observation pose with the maximum information gain and re-collect non-visual signals. By actively adjusting the pose, ambiguity is resolved, and dependence on fixed environmental geometry is eliminated. Finally, before visual detection occurs, a dynamic risk area is generated in the cost map and the cost value is mapped based on the existence probability to guide the robot to plan the avoidance path in advance, thereby improving the safety and environmental adaptability of blind spot obstacle avoidance. Attached Figure Description

[0012] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0013] Figure 1 This is a flowchart of the robot dynamic obstacle avoidance method based on machine vision according to the present invention.

[0014] Figure 2 This is a schematic diagram of the operating scenario of the robot dynamic obstacle avoidance method based on machine vision according to the present invention.

[0015] Figure 3 This is a block diagram of the robot dynamic obstacle avoidance system based on machine vision according to the present invention.

[0016] Figure 4 This is a block diagram of an electronic device according to the present invention. Detailed Implementation

[0017] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0018] Exemplary method:

[0019] This embodiment describes a specific implementation of a machine vision-based dynamic obstacle avoidance method for robots.

[0020] When autonomous mobile robots operate in environments such as hospitals, office buildings, and fixed production lines, their onboard visual sensors are limited by the field of view and occlusion. For dynamic obstacles that suddenly enter the field of view from blind spots such as corridor corners, doorways, and gaps between shelves, visual detection is often only completed when the obstacle is already close. This leaves insufficient reaction time for the robot, forcing it to passively respond by abruptly stopping or drastically slowing down, affecting operational continuity and safety. One approach is to pre-increase the planning cost of the corresponding area during high-frequency periods based on the historical statistical frequency of dynamic obstacles appearing in specific areas to achieve preventative avoidance. However, this approach essentially relies on statistical fitting of historical data, and is prone to inaccurate predictions when the dynamic patterns of the environment deviate or experience occasional changes. Another approach is to utilize the diffraction and multipath reflection characteristics of radio frequency or acoustic signals to perform non-line-of-sight detection of dynamic targets within blind spots. However, its detection effectiveness is highly dependent on specific environmental geometric conditions such as wall materials and reflective structures, as well as the corresponding sensor hardware configuration.

[0021] To address the limitations of the two approaches mentioned above, this embodiment provides a method that uses the time intervals in which dynamic obstacles appear at various spatial locations as a priori indicators of their presence or absence. It uses the mapping relationship between non-visual signals collected by the robot during its operation and historical visual detection results as the observation likelihood. Bayesian inference is used to estimate the probability of the dynamic obstacle's presence at the target location before visual contact occurs. When this probability falls into an ambiguous range where presence or absence cannot be determined, instead of relying on fixed environmental reflection structures for forced detection, the robot moves to the observation pose that best distinguishes the current non-visual signals, actively resolving ambiguity by changing the signal acquisition geometry. Finally, before visual detection of the dynamic obstacle, the probability of presence is hierarchically mapped to dynamic risk areas on a cost map, guiding the robot to plan avoidance paths in advance. Thus, on the one hand, real-time non-visual observation corrects purely historical statistical results, mitigating prediction inaccuracies when statistical patterns shift; on the other hand, the active adjustment of the robot's own pose replaces dependence on fixed environmental geometry, improving the method's adaptability in unstructured environments.

[0022] Figure 1 The illustration shows a machine vision-based robot dynamic obstacle avoidance method according to an embodiment of this application, including steps S1 to S3. Figure 2 This illustration shows the method of this embodiment in one operating scenario. The following will combine... Figure 1 and Figure 2 The implementation process of each step is explained.

[0023] like Figure 1 As shown, in step S1, the occurrence time of dynamic obstacles at various spatial locations within the robot's operating area is obtained, as well as the mapping relationship between non-visual signals collected by the robot during the occurrence time and visual detection results.

[0024] The operating area refers to the physical space covered by the robot when performing navigation tasks. Spatial location refers to a pre-defined discrete region within this operating area, used to statistically analyze the patterns of dynamic obstacle appearances. This region can be obtained by dividing the operating area into a grid or by selecting several key access nodes, such as corridor intersections or room entrances / exits. Figure 2 For example, positions A, B, and C are three spatial locations within the operating area, corresponding to the entrance and exit areas of different rooms, and are dynamic obstacles. Figure 2 (Illustration of pedestrians in China and Israel) Locations that frequently appear and are prone to creating blind spots.

[0025] The occurrence period refers to the regular time interval within which a dynamic obstacle appears at a certain spatial location. For example, at a certain entrance / exit, dynamic obstacles (such as pedestrians and trolleys) frequently pass through during certain time periods each day. Non-visual signals refer to physical signals collected by the robot during operation that do not rely on optical imaging. These signals are absorbed, scattered, or disturbed by obstacles during propagation, thus indirectly carrying information about the presence of obstacles. Visual detection results refer to the conclusions obtained by the robot through its visual sensors in detecting spatial locations, i.e., whether a dynamic obstacle has been detected at that location.

[0026] The mapping relationship refers to the correspondence between the statistical regularities exhibited by non-visual signals under the two scenarios of the presence and absence of dynamic obstacles. Mathematically, it is expressed as the conditional distribution of non-visual signals given the presence or absence of an obstacle, i.e., the observation likelihood. In this embodiment, the occurrence time provides prior information for calculating the probability of existence, and the mapping relationship provides the observation likelihood for calculating the probability of existence. Together, they constitute the two inputs for Bayesian inference in subsequent steps.

[0027] The aforementioned time periods and mapping relationships can be automatically constructed in a self-supervised manner during the robot's historical operations, without the need for manual annotation. This involves the following three steps:

[0028] The first step is to build a historical event database. The historical event database includes the temporal characteristics and visual detection results of the robot when it performs visual detection on various spatial locations during its historical operation, as well as historical non-visual signals collected within a preset time window before the visual detection results are generated. The visual detection results include two scenarios: detection of dynamic obstacles and no detection of dynamic obstacles.

[0029] Here, each time the robot completes a visual inspection of a spatial location, it uses the result of that visual inspection as a label to annotate the non-visual signals collected in the same area within a preset time window prior to that visual inspection. This automatically accumulates corresponding samples between non-visual signals and the presence or absence of obstacles without introducing additional manual annotation. The length of the preset time window can be set according to the typical moving speed of dynamic obstacles, for example, from several seconds to over ten seconds, to ensure that the non-visual signals within the window truly include signal changes caused by the obstacle's approach before visual contact.

[0030] The second step is to statistically analyze the time characteristics of the detected dynamic obstacles at each spatial location based on the visual detection results, and determine the corresponding time period of occurrence.

[0031] In essence, for a given spatial location, the temporal characteristics (e.g., times of day) of all detected dynamic obstacles at that location are collected from a historical event database and frequency statistics are performed. The time intervals where the detection frequency is significantly higher than the background level are defined as the occurrence periods for that spatial location. When a spatial location and a given time fall within its occurrence period, the prior probability of a dynamic obstacle being present at that location is high; conversely, the prior probability is low. Therefore, the occurrence period essentially provides a time-related prior to the presence or absence of dynamic obstacles at various spatial locations.

[0032] The third step is to extract the conditional distribution characteristics of historical non-visual signals under two visual detection results: dynamic obstacle detection and non-detection. The time advance between the time when the historical non-visual signals exhibit the conditional distribution characteristics and the time when the visual signals detect dynamic obstacles is calculated. An observation probability model based on the conditional distribution characteristics and the time advance is established as a mapping relationship.

[0033] Conditional distribution characteristics refer to the statistical features of non-visual signals under the conditions of obstacle presence and absence, such as differences in the mean, variance, or distribution pattern of a certain dimension of the signal. Timing advance refers to the time elapsed before the non-visual signal begins to exhibit the conditional distribution characteristics corresponding to the presence of an obstacle, relative to the time when the visual system finally detects the obstacle. Because non-visual signals can penetrate or bypass obstructions and perceive the approach of an obstacle before visual imaging, this timing advance is usually positive. It is this timing advance that gives this method the ability to predict before visual contact. The established observation probability model outputs the likelihood of the corresponding non-visual signal given the presence or absence of an obstacle, and uses the timing advance to characterize the temporal position of this likelihood before visual contact, thus serving as the mapping relationship used for subsequent inference.

[0034] The aforementioned non-visual signals can specifically employ a variety of physical signals that do not rely on optical imaging. In some embodiments, the non-visual signals include any one or more of the following combinations: received signal strength indication sequences of wireless local area networks, signal strength sequences of Bluetooth beacons, spectral characteristics of ambient acoustic signals, amplitude and phase characteristics of wireless channel state information, ultra-wideband signal characteristics, and millimeter-wave radar signal characteristics.

[0035] All of the aforementioned signals can reflect the presence of obstacles to varying degrees: when a human enters the propagation path of a wireless signal, it causes additional obstruction, attenuation, and multipath disturbance, resulting in observable changes in the amplitude and phase of the received signal strength indication of the wireless LAN, the signal strength of the Bluetooth beacon, and the wireless channel state information; human movement disturbs the ambient sound field, altering the spectral characteristics of the ambient acoustic signals; ultra-wideband signals and millimeter-wave radar signals can reflect the presence of moving targets on the propagation path through the time delay and Doppler characteristics of the echo. Using a combination of these signals can complement each other at different physical mechanisms, improving the ability to distinguish the presence or absence of obstacles. It should be noted that this embodiment does not require the aforementioned signal sources to be specifically geometrically deployed; the robot can collect these signals during operation using existing wireless access points, beacons, or its own onboard sensors.

[0036] like Figure 1 As shown, in step S2, taking any spatial location as the target location, if the robot runs to the preset range of the target location at the current time, and the current time falls within the occurrence period corresponding to the target location, then the probability of the existence of the dynamic obstacle at the target location is calculated based on the current non-visual signal and mapping relationship.

[0037] This step sets up dual trigger conditions in both space and time: the probability calculation for the target location is only triggered when the robot enters a preset range of the target location in space (e.g., a neighborhood centered on the target location and defined by a preset radius), and the current time falls within the time period corresponding to the occurrence of that target location. This setting ensures that the robot only performs subsequent inferences under spatiotemporal conditions where prediction is truly necessary, avoiding unnecessary overhead from continuous calculations throughout the entire process and at all locations. Combined with... Figure 2 When the robot travels along the corridor to the preset range of position B and the current time falls within the occurrence time period corresponding to position B, the existence probability calculation is triggered with position B as the target position.

[0038] The existence probability refers to the posterior probability of an event where a dynamic obstacle exists at the target location, inferred from currently acquired non-visual signals, before visual detection of the obstacle. Its calculation begins with a priori information given by the occurrence time period, corrects this priori information using the observational likelihood given by the mapping relationship, and is obtained according to Bayes' theorem. Let the event of a dynamic obstacle existing at the target location be denoted as... , does not exist If the currently acquired non-visual signal is s, then the probability of its existence is calculated using the following formula:

[0039] ;

[0040] in, This represents the prior probability of a dynamic obstacle existing at the target location, determined by the current time period in which the obstacle appears. and These represent the current non-visual signals given by the mapping relationship (i.e., the observation probability model) in the two cases of the presence and absence of obstacles. The likelihood, This is the probability of existence we are looking for.

[0041] For example, if the target location currently falls within the time period in which it occurs, the prior probability... If we take 0.5, and the likelihood of the currently collected non-visual signal in the case of the obstacle's presence is significantly higher than that in the case of its absence, then the existence probability calculated by the above formula will be higher than 0.5 and shift towards 1, indicating that the non-visual observation further supports the judgment of the obstacle's presence based on the prior; conversely, if the current signal is more consistent with the likelihood in the case of the obstacle's absence, then the existence probability will shift towards 0.

[0042] like Figure 1 As shown, in step S3, it is determined whether the existence probability falls within a preset interval representing the inability to determine the existence of a dynamic obstacle. If so, the information gain of the existence probability on the non-visual signals collected at multiple observation poses within the preset range is calculated, the robot is controlled to move to the observation pose with the largest information gain, and the non-visual signals are collected again to update the existence probability until the updated existence probability moves out of the preset interval or reaches the preset termination condition. If not, or the updated existence probability moves out of the preset interval or reaches the termination condition, a dynamic risk area corresponding to the target position is generated in the robot's cost map based on the current existence probability before the dynamic obstacle is visually detected. The corresponding cost value is mapped to the dynamic risk area based on the current existence probability to guide the robot to plan an avoidance path.

[0043] The preset interval is a probability range representing the inability to determine whether a dynamic obstacle exists. A probability close to 1 indicates a relatively certain determination of the obstacle's presence, while a probability close to 0 indicates a relatively certain determination of the obstacle's absence. When the probability falls within a certain range between these two values, the current non-visual signal alone is insufficient to reliably determine the obstacle's presence. This range is the preset interval, which can be, for example, between 0.4 and 0.6. This step branches the probability based on this range.

[0044] When the probability falls within a preset range, it indicates insufficient information under the current observation conditions. Instead of passively waiting, the method triggers active observation to resolve ambiguity. The underlying principle is that the propagation path of non-visual signals in space changes with the robot's observation pose. The distinguishability of the conditional distribution of signals collected by the robot at different observation poses varies depending on whether an obstacle is present or absent. At some observation poses, the influence of obstacles on the signal propagation path is masked by other environmental factors, resulting in highly overlapping signal conditional distributions that are difficult to distinguish. Conversely, at other observation poses, obstacles are precisely on the critical path affecting signal propagation, leading to significant separation of signal conditional distributions between the two scenarios, making them easily distinguishable. Therefore, by moving the robot to the observation pose that best separates the signal conditional distributions between the two scenarios, the ability to actively distinguish the presence or absence of obstacles can be improved without relying on fixed environmental reflection structures.

[0045] Taking received signal strength indication as an example: When the robot is in a certain observation pose, and the connection between it and a wireless access point is blocked by a wall, the received signal strength is mainly dominated by the fixed attenuation of the wall. At this time, the presence or absence of a dynamic obstacle at the suspected location has a relatively weak impact on the received signal strength, being submerged in the wall's attenuation. This results in a significant overlap in the received signal strength distributions depending on whether the obstacle is present or not. However, when the robot moves to another observation pose, so that the connection between it and the wireless access point just passes over the suspected location, the presence or absence of a dynamic obstacle will significantly change the obstruction along that connection, causing the received signal strength to show a clear difference between the two scenarios, with the corresponding distributions becoming distinct. The latter observation pose is the one more conducive to resolving ambiguity.

[0046] Combination Figure 2 The robot has multiple candidate observation poses within its preset range. Figure 2 China and Israel to (Illustrative diagram) Each candidate observation pose corresponds to different positions and orientations reachable by the robot within this range. The robot evaluates the information gain of acquiring non-visual signals at each candidate observation pose relative to the probability of their existence, and moves to the observation pose with the highest information gain. Figure 2 of to The one with the largest information gain is used to re-acquire non-visual signals at that pose, and the existence probability is updated according to the Bayesian formula described in step S2.

[0047] The aforementioned movement and re-acquisition can be performed iteratively until the updated existence probability moves out of the preset range, meaning the presence or absence of the obstacle can be determined, or a preset termination condition is met. The termination condition can be that the number of iterations reaches the upper limit, the available observation poses have been traversed, or the time budget allocated for active observation is exhausted, in order to prevent the robot from staying in active observation for too long and affecting the progress of its navigation task.

[0048] The aforementioned information gain can be quantitatively calculated based on the mapping relationship. In some embodiments, calculating the information gain of the probability of existence of non-visual signal pairs acquired at multiple observation poses within a preset range includes: calculating the expected divergence or expected decrease in posterior entropy of the conditional distribution of non-visual signals at each observation pose based on the mapping relationship, as the information gain corresponding to each observation pose.

[0049] One approach is to use the expected decrease in posterior entropy as the information gain. Let the binary entropy corresponding to the current probability of existence be the uncertainty in determining whether an obstacle exists, for a given candidate observation pose. Consider the non-visual signals that may be collected at this pose. (Its distribution is given by the mapping relationship), calculate the collected data. The expected value of the residual posterior entropy after updating the existence probability is then used; the difference between the two is the information gain of the observed pose. , denoted as:

[0050] ;

[0051] in, The binary entropy is calculated based on the current probability of the obstacle's presence or absence. In order to observe pose Signal collected at the location And update the corresponding posterior binary entropy, To collect signals that may be acquired at this pose Calculate the expected value. A larger information gain indicates a greater expected contribution of acquiring the signal at that pose to reducing the uncertainty of whether an obstacle exists.

[0052] Another approach is to use the expected divergence of the conditional distribution as the information gain, which measures the average difference between the signal conditional distribution and its mixture distribution at the observation pose o, considering the presence and absence of the obstacle. This is denoted as:

[0053] ;

[0054] in, Obstacles exist and non-existence Two scenarios, In order to observe pose Situation Lower signal The conditional distribution (given by the mapping relationship). For the corresponding mixed distribution, This measures the relative entropy (i.e., the divergence between two distributions) of the difference between them. The larger the divergence, the easier it is to distinguish the signal distributions under the two scenarios at the observation pose, and the greater the corresponding information gain. The robot then selects the observation pose with the maximum information gain, that is, the pose that is most conducive to distinguishing the presence or absence of obstacles.

[0055] When the existence probability does not fall within a preset range, or the existence probability updated after the aforementioned active observation has moved out of the preset range, or the termination condition is met, the method generates a dynamic risk region corresponding to the target location in the cost map based on the current existence probability, before visually detecting a dynamic obstacle, and maps a cost value to it for the robot to avoid during path planning. The cost map is a grid map used by the robot for navigation, representing the drivability of each area with a cost value; a higher cost value indicates that the area should be avoided. The dynamic risk region is the area in the cost map corresponding to the target location, used to bear the predictive cost derived from the existence probability. Figure 2 In the cost map, a1, b1, and c1 are dynamic risk areas generated corresponding to positions A, B, and C, respectively. Their range and cost are determined by the probability of their existence at the corresponding positions, thus guiding the robot to plan an avoidance path before it detects obstacles visually.

[0056] In some embodiments, mapping the corresponding cost value to a dynamic risk region based on the current existence probability includes: expanding the range of the dynamic risk region or increasing the cost value in the cost map at a first rate when the current existence probability increases; and shrinking the range of the dynamic risk region or reducing the cost value in the cost map at a second rate when the current existence probability decreases; wherein the first rate is not equal to the second rate.

[0057] The first rate and the second rate represent the speed at which the range or cost of the dynamic risk zone changes as the probability of existence increases and decreases, respectively. Setting them to be unequal is for obstacle avoidance safety considerations: when the probability of existence increases, the risk zone should be rapidly expanded or the cost increased at the faster first rate, allowing the robot to avoid obstacles as early as possible; conversely, when the probability of existence decreases, the risk zone should be gradually reduced or the cost decreased at the slower second rate to avoid safety hazards caused by the rapid removal of the risk zone due to short-term signal fluctuations, which could lead to the robot immediately entering the area.

[0058] Preferably, the first rate is greater than the second rate, so that the risk area exhibits an asymmetric evolutionary characteristic of "fast establishment and slow elimination": when the probability of existence increases, the faster establishment rate is used to prioritize obstacle avoidance safety, and when the probability of existence decreases, the slower elimination rate is used to prevent the risk area from being repeatedly established and removed in a short period of time due to the instantaneous jitter of non-visual signals. This avoids repeated oscillations in the robot's avoidance behavior or accidental entry into the risk area before the risk has been truly eliminated, thus ensuring obstacle avoidance safety in a conservative manner overall.

[0059] To ensure that the occurrence time and mapping relationship can be continuously corrected as the environment changes, in some embodiments, the method further includes: when the robot visually detects a dynamic obstacle within a preset range, recording the detection time and detection result of the dynamic obstacle being visually detected; and performing incremental updates on the occurrence time and mapping relationship corresponding to the target position based on the non-visual signals collected before the detection time and the detection result.

[0060] In other words, whenever the robot obtains a true visual detection result within a preset range, this visual detection result (including both detected and undetected dynamic obstacles) serves as a new label. This label, along with the non-visual signals collected before the detection time, forms a new sample. This sample is used to incrementally update the statistical analysis of the target location's occurrence time and the mapping relationship (observation probability model). Thus, the method continuously corrects the prior and likelihood online using true visual detection results during operation, ensuring that the occurrence time and mapping relationship consistently align with the actual dynamic patterns of the current environment, mitigating prediction inaccuracies when environmental dynamic patterns deviate.

[0061] Furthermore, after actual visual detection occurs, the probability of the dynamic obstacle's next possible location can be predicted based on the continuity of its movement. In some embodiments, the method further includes: extracting the local motion trajectory of the dynamic obstacle after visual detection; predicting the next spatial location the dynamic obstacle will reach based on the local motion trajectory; determining the transfer probability of the dynamic obstacle reaching the next spatial location based on a pre-constructed spatial location transfer model; and improving the probability of the dynamic obstacle's existence calculated for the next spatial location based on the transfer probability.

[0062] Local motion trajectory refers to the trajectory formed by the changing position of a dynamic obstacle over a period of time after it has been visually detected. The spatial position transfer model is a model pre-constructed based on historical data, characterizing the movement patterns of the dynamic obstacle between various spatial positions. It provides the probability of the dynamic obstacle moving from its current spatial position to each adjacent spatial position. Based on the local motion trajectory, the next spatial position the dynamic obstacle will reach can be predicted, and then the spatial position transfer model determines the probability of reaching that next spatial position. Combined with... Figure 2 When the robot visually detects a dynamic obstacle at location A ( Figure 2 (Illustrated by a pedestrian), and based on its local motion trajectory, predicting that the dynamic obstacle will move towards position B, that is, determining the probability of the dynamic obstacle moving from position A to position B according to the spatial position transfer model. Figure 2 The arrows between positions A, B, and C illustrate this transfer relationship.

[0063] Since it is known that the dynamic obstacle is moving to the next spatial location with a certain transition probability, the existence probability of that next spatial location can be increased accordingly, allowing the robot to predict that location earlier. It should be noted that the robot may not yet have reached the preset range of the next spatial location, and therefore may not have yet collected the current non-visual signal at that location. Regardless of whether the robot has reached the preset range of the location, the increase based on the transition probability is the prior probability of the next spatial location determined by its occurrence time, rather than directly modifying the Bayesian calculation result of step S2 itself. When the robot has not yet reached the location, the increased prior probability is retained. When the robot subsequently reaches the preset range of the location and collects the current non-visual signal, the increased prior probability replaces the original prior probability in the Bayesian calculation of step S2. When the robot has already reached the location, the increased prior probability directly participates in the Bayesian calculation of step S2. This increases the existence probability of the next spatial location. Specifically, the prior probability of the next spatial location can be increased using the following formula:

[0064] ;

[0065] in, The prior probability of the next spatial location being determined by its occurrence time period is used as the prior in the Bayesian calculation of step S2. , This represents the transition probability of a dynamic obstacle reaching its next spatial position. The preset fusion coefficient is a value between 0 and 1. This represents the adjusted prior probability. The above formula uses... The margin for adjustment is such that the higher the transition probability and the lower the original prior probability, the larger the adjustment will be, and the adjusted prior probability will not exceed 1. Thus, before the dynamic obstacle reaches the next spatial position, the robot has already raised the prior for that position, and when it subsequently reaches that position, it can more quickly establish risk prediction, further enhancing the foresight of the method.

[0066] In summary, the machine vision-based robot dynamic obstacle avoidance method provided in this embodiment has the following technical advantages over existing technologies:

[0067] First, by using the occurrence time of dynamic obstacles as a prior and the mapping relationship between non-visual signals and visual detection results as the observation likelihood, the probability of the existence of dynamic obstacles is estimated before visual contact using Bayesian inference. The real-time non-visual observations collected during robot operation are used to correct the pure historical statistical results online, which alleviates the prediction inaccuracy caused by the deviation of environmental dynamic laws when relying solely on historical statistical fitting.

[0068] Second, by setting a preset interval where the presence or absence of an obstacle cannot be determined, active observation is triggered only when the probability of existence falls into this interval, i.e., when there is indeed ambiguity in the judgment. Based on information gain, the robot is driven to move to the observation pose that is most conducive to distinguishing the presence or absence of obstacles. Ambiguity is actively resolved by changing the signal acquisition geometry. The robot replaces the dependence on fixed environmental reflection structures and dedicated sensor deployment with the active adjustment of its own pose. Moreover, the robot only pays for maneuvering costs when necessary, thus achieving a balance between distinguishing ability and maneuvering costs.

[0069] Third, by mapping the existing probability hierarchy into dynamic risk areas in the cost map before visual detection of obstacles, and making the risk areas exhibit the asymmetric evolution characteristics of "fast establishment and slow elimination", the robot can plan the avoidance path in advance before visual contact and conservatively ensure obstacle avoidance safety. At the same time, through incremental updates of real visual detection results and downstream prediction based on spatial position transfer model, the method continuously adapts to the actual environment and has cross-position forward-looking capabilities during operation.

[0070] Exemplary system:

[0071] Figure 3The illustration shows a machine vision-based robot dynamic obstacle avoidance system according to an embodiment of this application, including: a data acquisition module, used to acquire the occurrence time of dynamic obstacles at various spatial locations within the robot's operating area, and the mapping relationship between non-visual signals collected by the robot during the occurrence time and visual detection results; a probability calculation module, used to take any spatial location as the target location, and if the robot runs within a preset range of the target location at the current time, and the current time falls within the occurrence time corresponding to the target location, calculate the probability of the existence of the dynamic obstacle at the target location based on the current non-visual signals and the mapping relationship; and a decision module, used to determine whether the probability of existence falls within the range where the dynamic obstacle cannot be determined. Preset range for object presence / absence: If yes, calculate the information gain of the presence probability to the non-visual signals collected at multiple observation poses within the preset range, control the robot to move to the observation pose with the largest information gain, and re-collect non-visual signals to update the presence probability until the updated presence probability moves out of the preset range or reaches the preset termination condition; if no, or the updated presence probability moves out of the preset range or reaches the termination condition, generate a dynamic risk area corresponding to the target position in the robot's cost map based on the current presence probability before visual detection of the dynamic obstacle, and map the corresponding cost value to the dynamic risk area based on the current presence probability to guide the robot to plan an avoidance path.

[0072] In one example, the process involves obtaining the occurrence time periods of dynamic obstacles at various spatial locations within the robot's operating area, as well as the mapping relationship between non-visual signals collected by the robot during these occurrence time periods and visual detection results. This includes: constructing a historical event database, which includes the temporal characteristics and visual detection results of the robot's visual detection of various spatial locations during historical operations, as well as historical non-visual signals collected within a preset time window before the visual detection results were generated. The visual detection results include two scenarios: detection of dynamic obstacles and non-detection of dynamic obstacles. For each spatial location, statistics are performed based on the temporal characteristics of the visual detection results indicating the detection of dynamic obstacles to determine the corresponding occurrence time periods. The conditional distribution characteristics of historical non-visual signals under both visual detection results (detection of dynamic obstacles and non-detection of dynamic obstacles) are extracted. The time advance between the moment when the historical non-visual signals exhibit the conditional distribution characteristics and the moment when the visual detection of dynamic obstacles occurs is calculated. An observation probability model based on the conditional distribution characteristics and the time advance is established as the mapping relationship.

[0073] In one example, non-visual signals include any one or more of the following: received signal strength indication sequences of wireless LANs, signal strength sequences of Bluetooth beacons, spectral characteristics of ambient acoustic signals, amplitude and phase characteristics of wireless channel state information, ultra-wideband signal characteristics, and millimeter-wave radar signal characteristics.

[0074] In one example, the information gain of the probability of the existence of non-visual signals collected at multiple observation poses within a preset range is calculated, including: based on the mapping relationship, calculating the expected divergence or expected decrease of the posterior entropy of the conditional distribution of non-visual signals at each observation pose, as the information gain corresponding to each observation pose.

[0075] In one example, mapping the cost value corresponding to the dynamic risk area based on the current existence probability includes: expanding the range of the dynamic risk area or increasing the cost value in the cost map at a first rate when the current existence probability increases; and shrinking the range of the dynamic risk area or reducing the cost value in the cost map at a second rate when the current existence probability decreases; wherein the first rate is not equal to the second rate.

[0076] In one example, the system functions also include: when the robot visually detects a dynamic obstacle within a preset range, recording the detection time and result of the visual detection of the dynamic obstacle; and performing incremental updates on the occurrence time period and mapping relationship corresponding to the target position based on the non-visual signals collected before the detection time and the detection result.

[0077] In one example, the system functionality also includes: after visually detecting a dynamic obstacle, extracting the local motion trajectory of the dynamic obstacle; predicting the next spatial location the dynamic obstacle will reach based on the local motion trajectory; determining the transfer probability of the dynamic obstacle reaching the next spatial location based on a pre-built spatial location transfer model; and improving the probability of the existence of the dynamic obstacle calculated for the next spatial location based on the transfer probability.

[0078] Exemplary electronic device:

[0079] Figure 4 A block diagram of an electronic device according to an embodiment of this application is illustrated.

[0080] like Figure 4 As shown, the electronic device includes one or more processors and memory.

[0081] A processor can be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and can control other components in an electronic device to perform desired functions.

[0082] The memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0083] In one example, the electronic device may also include input devices and output devices, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0084] Of course, for the sake of simplicity, Figure 3 Only some of the components of the electronic device relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device may include any other suitable components depending on the specific application.

[0085] Exemplary computer-readable medium:

[0086] Embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps described in the "Exemplary Methods" section above according to the various embodiments of this application.

[0087] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0088] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not restrict the application from being implemented using the specific details described above.

[0089] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0090] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0091] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0092] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A robot dynamic obstacle avoidance method based on machine vision, characterized in that, include: The robot obtains the time periods during which dynamic obstacles appear at various spatial locations within its operating area, and the mapping relationship between non-visual signals collected by the robot during the time periods and visual detection results. Taking any of the aforementioned spatial locations as the target location, if the robot moves to a preset range of the target location at the current time, and the current time falls within the occurrence period corresponding to the target location, then the probability of the existence of a dynamic obstacle at the target location is calculated based on the current non-visual signal and the mapping relationship. Determine whether the probability of existence falls within a preset range that indicates the inability to determine the existence of the dynamic obstacle. If so, calculate the information gain of the non-visual signals acquired at multiple observation poses within the preset range on the existence probability, control the robot to move to the observation pose with the largest information gain, and reacquire non-visual signals to update the existence probability until the updated existence probability moves out of the preset interval or reaches the preset termination condition. If not, or if the updated existence probability moves out of the preset interval, or if the termination condition is reached, then based on the current existence probability, before the dynamic obstacle is visually detected, a dynamic risk area corresponding to the target location is generated in the robot's cost map, and the corresponding cost value is mapped to the dynamic risk area based on the current existence probability, so as to guide the robot to plan an avoidance path.

2. The robot dynamic obstacle avoidance method based on machine vision according to claim 1, characterized in that, The acquisition of the occurrence time of dynamic obstacles at various spatial locations within the robot's operating area, and the mapping relationship between non-visual signals collected by the robot during the occurrence time and visual detection results, includes: A historical event database is constructed, which includes the temporal characteristics and visual detection results of the robot when it performs visual detection on various spatial locations during its historical operation, as well as historical non-visual signals collected within a preset time window before the visual detection results are generated. The visual detection results include two scenarios: detection of the dynamic obstacle and non-detection of the dynamic obstacle. For each of the aforementioned spatial locations, the time characteristics of the detected dynamic obstacles are statistically analyzed based on the visual detection results to determine the corresponding occurrence time period; Extract the conditional distribution features of the historical non-visual signal under two visual detection results: detection of the dynamic obstacle and non-detection of the dynamic obstacle. Calculate the time advance between the moment when the historical non-visual signal exhibits the conditional distribution features and the moment when the dynamic obstacle is detected visually. Establish an observation probability model based on the conditional distribution features and the time advance as the mapping relationship.

3. The robot dynamic obstacle avoidance method based on machine vision according to claim 1, characterized in that, The non-visual signals include any one or more of the following combinations: received signal strength indication sequence of wireless local area network, signal strength sequence of Bluetooth beacon, spectral characteristics of ambient acoustic signals, amplitude and phase characteristics of wireless channel state information, ultra-wideband signal characteristics, and millimeter-wave radar signal characteristics.

4. The robot dynamic obstacle avoidance method based on machine vision according to claim 1, characterized in that, The calculation of the information gain of the presence probability to non-visual signals acquired at multiple observation poses within the preset range includes: Based on the mapping relationship, the expected divergence or expected decrease of the posterior entropy of the non-visual signal conditional distribution under each observation pose is calculated, and used as the information gain corresponding to each observation pose.

5. The robot dynamic obstacle avoidance method based on machine vision according to claim 1, characterized in that, The cost value corresponding to the mapping of the current existence probability to the dynamic risk area includes: When the current probability of existence increases, the range of the dynamic risk area or the cost value is increased in the cost map at a first rate. When the current probability of existence decreases, the range of the dynamic risk area or the cost value is reduced in the cost map at a second rate. Wherein, the first rate is not equal to the second rate.

6. The robot dynamic obstacle avoidance method based on machine vision according to claim 1, characterized in that, The method further includes: When the robot visually detects the dynamic obstacle within the preset range, it records the detection time and result of the visual detection of the dynamic obstacle. Based on the non-visual signals collected before the detection time and the detection results, incremental updates are performed on the occurrence time period corresponding to the target location and the mapping relationship.

7. The robot dynamic obstacle avoidance method based on machine vision according to claim 6, characterized in that, The method further includes: After visually detecting the dynamic obstacle, the local motion trajectory of the dynamic obstacle is extracted; Based on the local motion trajectory, predict the next spatial location that the dynamic obstacle will reach; Based on a pre-built spatial location transfer model, the transfer probability of the dynamic obstacle reaching the next spatial location is determined; Based on the transition probability, the probability of the existence of the dynamic obstacle calculated for the next spatial location is increased.

8. A robot dynamic obstacle avoidance system based on machine vision, characterized in that, include: The data acquisition module is used to acquire the time periods of occurrence of dynamic obstacles at various spatial locations within the robot's operating area, as well as the mapping relationship between non-visual signals collected by the robot during the occurrence periods and visual detection results. The probability calculation module is used to calculate the probability of the existence of a dynamic obstacle at the target location based on the mapping relationship between the current non-visual signal and the target location, if the robot runs within a preset range of the target location at the current time and the current time falls within the occurrence period corresponding to the target location. The decision module is used to determine whether the probability of existence falls within a preset range that indicates the inability to determine the existence of the dynamic obstacle. If so, calculate the information gain of the non-visual signals acquired at multiple observation poses within the preset range on the existence probability, control the robot to move to the observation pose with the largest information gain, and reacquire non-visual signals to update the existence probability until the updated existence probability moves out of the preset interval or reaches the preset termination condition. If not, or if the updated existence probability moves out of the preset interval, or if the termination condition is reached, then based on the current existence probability, before the dynamic obstacle is visually detected, a dynamic risk area corresponding to the target location is generated in the robot's cost map, and the corresponding cost value is mapped to the dynamic risk area based on the current existence probability, so as to guide the robot to plan an avoidance path.

9. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer-executable instructions stored thereon, characterized in that: When the computer-executable instructions are executed by a processor, they implement the steps of the method as described in any one of claims 1 to 7.

Images