Machine vision navigation optimization method for complex environment

Abstract

The invention relates to the technical field of machine vision and autonomous navigation, and discloses a complex environment-oriented machine vision navigation optimization method, which comprises the following steps of: acquiring a visual image, and calculating multi-dimensional original entropy of semantic uncertainty, dynamic instability and task association degree of the visual image; performing modulation fusion on the original entropy to generate a cognitive entropy graph representing comprehensive uncertainty; arbitrating a region of interest based on the graph and calling resources to carry out fine solution perception; planning a risk avoidance path in combination with the cognitive entropy graph and the refined solution result; and finally, performing tracing self-calibration on the generation process of the cognitive entropy graph according to a refined solution result. According to the method, a set of brand-new environment uncertainty quantification framework is established, semantic ambiguity, dynamic chaos degree and task association degree are taken into consideration together, and nonlinear fusion is performed on the multi-source information by using a cross modulation means, so that a cognitive entropy graph capable of comprehensively reflecting potential risks is generated.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine vision and autonomous navigation, in particular to a machine vision navigation optimization method for complex environments. BACKGROUND

[0002] Autonomous mobile systems, such as unmanned vehicles, delivery robots and service robots, are gradually moving from structured environments to complex open scenarios coexisting with humans. In these scenarios, the system must continuously interact with various dynamic, occasional and ambiguous events. In order to ensure the safety and smoothness of operation, the system needs to have a capability far beyond traditional path planning, that is, to accurately predict and intelligently respond to potential and unclear risks in the environment. Therefore, it has important application value to develop a technical solution that enables autonomous systems to achieve robust perception, efficient decision-making and adaptive optimization in complex dynamic environments.

[0003] The existing mainstream machine vision navigation technology is based on target detection and semantic segmentation based on deep learning. These technologies process visual information through large-scale neural networks and exhibit high accuracy in identifying and locating predefined classes of objects. They can generate a clear environmental geometry and semantic model, accurately depicting roads, buildings and other static elements, and identifying vehicles, pedestrians and other known dynamic objects. On this basis, the planning module calculates a collision-free path based on this deterministic environmental model, which enables the system to achieve reliable navigation in scenarios composed of familiar elements.

[0004] However, when these systems are deployed in real-world complex environments, their inherent technical limitations become apparent. First, the risk assessment model has a dimensional defect. When faced with an object that has never appeared in the training set or a known object exhibiting abnormal motion patterns, existing methods cannot distinguish the nature of the risk. Their handling of semantic ambiguity and dynamic instability is mixed and rough, which leads to insufficient perception robustness of the system under non-ideal conditions. Second, the allocation of system computing resources is fixed. In order to cope with occasional extreme situations, high-performance perception models are continuously running, which causes serious redundancy of computing resources in most regular driving scenarios, directly restricting the application efficiency of platforms with limited endurance. Finally, the system architecture has inherent open-loop characteristics. The information transfer between perception and planning is one-way, and the planning module cannot know the degree of uncertainty at the perception level. More critically, the system lacks a feedback loop and cannot use subsequent accurate analysis results to trace and correct the uncertainty assessment model at the front end, thus losing the ability to optimize itself from task experience. Therefore, the technical personnel in the field propose a machine vision navigation optimization method for complex environments to solve the above problems. SUMMARY

[0005] In view of the deficiencies of the prior art, the machine vision navigation optimization method for complex environment is provided, which solves the technical problems that the navigation robustness and resource utilization efficiency are difficult to be considered in the prior art due to the single risk assessment dimension, the fixed allocation of computing resources and the open-loop of perception decision-making process.

[0006] To achieve the above object, the technical scheme is adopted in the present application. The first aspect of the present application provides a machine vision navigation optimization method for complex environment, which realizes the quantitative evaluation of environmental uncertainty, the dynamic on-demand allocation of computing resources and the risk containment of navigation path through a multi-step closed-loop processing flow, and has self-adaptive calibration capability. The method comprises: Firstly, the visual image of the current scene is obtained, and a plurality of original entropy components respectively representing semantic uncertainty, dynamic instability and task correlation degree are calculated based on the visual image; Then, the plurality of original entropy components are cross-modulated and fused to generate a cognitive entropy map representing the comprehensive uncertainty degree of each region in the scene; Then, the arbitration is performed based on the cognitive entropy map to determine one or more interest regions in the scene, and the computing resources are dynamically allocated accordingly; Then, the computing resources are called to perform fine perception on the one or more interest regions to generate fine perception results; Subsequently, the cognitive entropy map and the fine perception results are combined to perform risk-averse path planning; Finally, according to the comprehensive uncertainty degree represented by the fine perception results and the cognitive entropy map, the process of generating the cognitive entropy map is traced and self-calibrated.

[0007] Preferably, the step of calculating the plurality of original entropy components comprises: For each image block in the visual image, a preset neural network model is used to calculate the probability distribution of the image block in a plurality of known semantic categories and a dedicated unknown category, and the semantic uncertainty entropy is determined according to the Shannon entropy formula of the probability distribution; For the image block, the dense optical flow field inside the image block is calculated, and the Jacobian matrix corresponding to the spatial gradient of the image block is constructed based on the optical flow field, and then the variance of the eigenvalues of the Jacobian matrix is calculated to determine the dynamic instability entropy; For the image block, the shortest Euclidean distance between the projection point of the image block in the three-dimensional space and the currently planned navigation path is calculated, and the distance is substituted into a preset exponential decay function to determine the task correlation degree entropy.

[0008] In one embodiment, the step of cross-modulating and fusing the plurality of original entropy components comprises: setting a semantic threshold and a dynamic threshold, and when the calculation results of the semantic uncertainty entropy and the dynamic instability entropy of a region are both higher than the respective corresponding thresholds, nonlinearly increasing the uncertainty representation value of the region by a preset enhancement function; setting a task association threshold, and when the calculation result of the task association degree entropy of a region is lower than the task association threshold, reducing the weight of the semantic uncertainty entropy and the dynamic instability entropy in the subsequent fusion calculation by a preset suppression function.

[0009] Further, the step of cross-modulating and fusing the plurality of original entropy components further comprises: performing weighted summation operation on the plurality of original entropy components after cross-modulation and a set of weight coefficients to obtain the final cognitive entropy value; wherein the weight coefficients are dynamically adjusted by the output of the traceable self-calibration process.

[0010] In one embodiment, the step of arbitrating based on the cognitive entropy map comprises: performing numerical comparison between the entropy values of each pixel point in the cognitive entropy map and a preset low-risk threshold and a preset high-risk threshold; if the entropy values of all points are lower than the low-risk threshold, the control system is maintained in the alert cruise mode of only performing basic perception and obstacle avoidance; if there is any point whose entropy value is higher than the high-risk threshold, the control system switches to the detailed analysis mode, and the image region corresponding to the point is determined as the region of interest.

[0011] Preferably, the step of calling the computing resource to perform detailed perception on the one or more regions of interest comprises: activating a preset high-precision perception model; the data input of the high-precision perception model is limited to the image part determined as the region of interest, and the high-precision perception model processes the part to output the detailed perception result; using the accurate semantic or geometric information contained in the detailed perception result to update a global world model used for navigation.

[0012] In one embodiment, the step of performing risk-averse path planning comprises: stacking the cognitive entropy map as an additional risk cost map into the cost space of the navigation algorithm; constructing a comprehensive cost function including a path length cost term, a collision cost term with known obstacles, and a cognitive entropy integral cost term obtained by line integrating the cognitive entropy map along a path; performing a search algorithm based on the comprehensive cost function to determine an optimal navigation path.

[0013] Further, the step of tracing back the process of generating the cognitive entropy map to perform self-calibration includes: regarding the fine-resolution perception result as an objective description of the real situation of the region of interest; constructing a loss function for quantifying the numerical difference between the comprehensive uncertainty degree predicted by the cognitive entropy map and the real situation description; based on the loss function, performing an optimization algorithm to adjust the weight coefficients and internal parameters of the cross-modulation module used to generate the cognitive entropy map.

[0014] Preferably, the construction of the loss function is for: when the numerical value of the comprehensive uncertainty degree predicted by the cognitive entropy map is high, but the fine-resolution perception result indicates a low-risk situation, determining it as a false alarm event and accordingly generating a first type of loss value; when the numerical value of the comprehensive uncertainty degree predicted by the cognitive entropy map is low, but the fine-resolution perception result indicates a high-risk situation, determining it as a missed alarm event and accordingly generating a second type of loss value.

[0015] The second aspect of the present application provides a machine vision navigation optimization system for complex environments, which is applied to any of the above-mentioned methods and includes: a raw entropy calculation module for obtaining a visual image of a current scene and calculating a plurality of raw entropy components representing semantic uncertainty, dynamic instability, and task relevance degree based on the visual image; a cognitive entropy generation module connected to the output end of the raw entropy calculation module for cross-modulating and fusing the plurality of raw entropy components to generate a cognitive entropy map representing the comprehensive uncertainty degree of each region in the scene; an arbitration and resource allocation module connected to the output end of the cognitive entropy generation module for arbitrating based on the cognitive entropy map to determine one or more regions of interest in the scene and accordingly generating a calculation resource allocation instruction; a fine-resolution perception module for performing fine-resolution perception on the one or more regions of interest after receiving the calculation resource allocation instruction to generate a fine-resolution perception result; a path planning module having input ends connected to the output ends of the cognitive entropy generation module and the fine-resolution perception module for jointly using the cognitive entropy map and the fine-resolution perception result to perform risk-averse path planning. A self-calibration module, an input end of which is connected with output ends of the fine solution perception module and the cognitive entropy generation module, is used for traceable self-calibration of the cognitive entropy generation module according to the fine solution perception result and the comprehensive uncertainty degree represented by the cognitive entropy graph.

[0016] The application provides a machine vision navigation optimization method for a complex environment. 1. The application establishes a new environment uncertainty quantification framework, considers semantic ambiguity, dynamic chaos and task correlation together, and uses cross modulation to nonlinearly fuse these multi-source information, so that a cognitive entropy graph reflecting potential risks is generated, compared with the prior art which only depends on classification confidence or a geometric model to judge threats, the application solves the fundamental defects of poor perception robustness and easy misjudgment caused by insufficient risk assessment dimension when facing novel objects or scenes with non-rigid complex motion patterns.

[0017] 2. The application provides an adaptive computing resource allocation mechanism based on cognitive entropy arbitration, a lightweight warning perception network is used to continuously monitor the global, and high-power fine solution perception resources are accurately focused on the interest area with real uncertainty according to the quantification result of the cognitive entropy graph, compared with the prior art which uses a single high-performance model to run fixedly in the whole perception cycle, the application solves the problems of serious algorithm redundancy in most conventional scenes and difficulty in deployment on platforms with limited endurance, and realizes the optimal configuration of perception performance and running efficiency.

[0018] 3. The application constructs a closed-loop system of perception, decision and learning, the cognitive entropy graph is directly introduced into the cost function of path planning, so that the navigation decision has the ability to actively avoid unknown risks, and through the designed traceable self-calibration loop, the analysis result of the high-precision model can be fed back to the front-end perception module, realizing the online iterative optimization of the uncertainty evaluation model, compared with the prior art which separates planning and perception and has fixed model parameters, the application solves the core bottleneck that the behavior is passive when facing potential risks and cannot accumulate cognition from task experience to realize self-evolution. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 Fig. 1 is a system structure schematic diagram of the application; Figure 2 Fig. 2 is a method flow schematic diagram of the application; Figure 3 Fig. 3 is a cognitive entropy graph generation process schematic diagram of the application; Figure 4 Fig. 4 is a traceable self-calibration working principle schematic diagram of the application.

[0020] Wherein, 10, original entropy calculation module; 20, cognitive entropy generation module; 30, arbitration and resource allocation module; 40, fine solution perception module; 50, path planning module; 60, self-calibration module. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the specification of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0022] Referring to the drawings Figure 1 , Figure 1 It is a structural schematic diagram of a machine vision navigation optimization system for complex environment according to an embodiment of the present application. The present application provides a machine vision navigation optimization system for complex environment, which can be deployed on a mobile platform requiring autonomous navigation. The system can include: an original entropy calculation module 10, a cognitive entropy generation module 20, an arbitration and resource allocation module 30, a fine solution perception module 40, a path planning module 50, and a self-calibration module 60.

[0023] The original entropy calculation module 10 is used to receive visual image data acquired by external sensors. The module internally contains a lightweight processing unit, which performs grid segmentation on the input visual image, and calculates three independent original entropy components in parallel for each image block. The three components are: semantic uncertainty entropy representing the classification confidence of image content, dynamic instability entropy representing the complexity of the motion pattern of image content, and task relevance entropy representing the relevance of image content to the current navigation task space. After calculation, the original entropy calculation module 10 transmits the vector group containing the three entropy components output by it to the cognitive entropy generation module 20.

[0024] The cognitive entropy generation module 20 is connected to the output end of the original entropy calculation module 10. The module internally contains a cross-modulation submodule and a weighted fusion submodule. The cross-modulation submodule performs a set of preset nonlinear transformation rules on the received original entropy components. The weighted fusion submodule then performs weighted summation operation on the transformed entropy components and a set of weight coefficients, thereby generating a final cognitive entropy value for each image block. The module combines all cognitive entropy values into a two-dimensional cognitive entropy map aligned with the original image space, and transmits the map to the arbitration and resource allocation module 30 and the path planning module 50.

[0025] An arbitration and resource allocation module 30, whose input is connected to the output of the cognitive entropy generation module 20. Inside this module, there is a decision logic unit, which compares the value in the cognitive entropy map with a pre-set low risk threshold and a pre-set high risk threshold. Based on the comparison result, this module generates system operation mode switching instructions and computing resource allocation instructions. Specifically, when the value in the cognitive entropy map is higher than the high risk threshold, the module determines that the corresponding region is an area of interest, and transmits the coordinate information of this region together with the mode switching instructions to the fine-grained perception module 40.

[0026] The fine-grained perception module 40, whose input is used to receive raw visual image data, and whose output is connected to the arbitration and resource allocation module 30 to receive instructions. Inside this module, there is a high-precision perception model. After receiving the mode switching instructions, the module is activated, and its calculation processing range is limited to the image part defined by the received interest region coordinates. The module analyzes the interest region to output fine-grained perception results containing accurate semantics, geometry and motion state. The fine-grained perception results are used to update a global world model, and are simultaneously transmitted to the path planning module 50 and the self-calibration module 60.

[0027] The path planning module 50, whose input is connected to the outputs of the cognitive entropy generation module 20 and the fine-grained perception module 40. Inside this module, there is a path search algorithm unit, whose cost function takes into account three parts: the length cost of the path itself, the collision cost of the path with the known obstacles in the global world model, and the cognitive risk cost obtained by performing a line integral operation on the cognitive entropy map along the path. Based on this comprehensive cost function, the module performs calculations and outputs an optimal navigation path to the control unit of the platform.

[0028] The self-calibration module 60, whose input is connected to the fine-grained perception module 40 and the cognitive entropy generation module 20. This module is used to receive the fine-grained perception results of the interest region, as well as the original entropy component information of this region before the cognitive entropy map is generated. Inside this module, there is a loss function calculation unit and a parameter optimization unit. It takes the fine-grained perception results as an objective reference, calculates the difference between them and the cognitive entropy prediction value, and generates a loss value. The parameter optimization unit calculates the adjustment amount of the weight coefficients and the cross-modulation parameters inside the cognitive entropy generation module 20 based on the loss value, and feeds back the adjustment amount to the cognitive entropy generation module 20 to complete a closed-loop calibration.

[0029] Referring to the accompanying drawings Figure 2 , Figure 2 is a flowchart of a complex environment-oriented machine vision navigation optimization method according to an embodiment of the present application. The method can be performed by the system shown in Figure 1 , and its specific steps can include: S1. Perform the original entropy component calculation step. Acquire a visual image frame and process the image to calculate three original entropy components that represent semantic uncertainty, dynamic instability, and task relevance, respectively.

[0030] S2. Perform the cognitive entropy map generation step. The three original entropy components calculated in step S1 are nonlinearly cross-modulated and then weighted and fused to generate a cognitive entropy map that can comprehensively characterize the degree of uncertainty in each region of the scene.

[0031] S3. Execute arbitration and resource allocation steps. Analyze the cognitive entropy map generated in step S2, and based on the comparison results of the values ​​of each point in the map with preset thresholds, determine one or more regions of interest in the current scene, and generate computing resource allocation instructions accordingly.

[0032] S4. Perform the refined perception step. Based on the allocation instructions generated in step S3, call the high-precision perception model and analyze only the image portion identified as the region of interest to generate a refined perception result containing accurate physical and semantic attributes.

[0033] S5. Perform the risk avoidance path planning step. Combine the cognitive entropy map generated in step S2 with the refined perception results generated in step S4 to perform path search under a comprehensive cost function to plan an optimal navigation path.

[0034] S6. Perform the abductive self-calibration step. Using the refined perception results generated in step S4 as a reference, evaluate and adjust the process of generating the cognitive entropy map in step S2 to achieve closed-loop optimization of the parameters of the uncertainty assessment model.

[0035] The specific implementation methods of the above steps will be explained in detail below.

[0036] In step S1, the system processes each frame of the input visual image to calculate the raw entropy components in three dimensions. This step is performed by the raw entropy calculation module 10. First, the input visual image... Spatially divided into one The grid forms a series of non-overlapping image patches. A lightweight convolutional neural network (CNN) model performs a forward pass through all image patches, and the network's output is designed to simultaneously support the computation of the following three entropy components: First, calculate the semantic uncertainty entropy. The network's classification head classifies each image patch. Output a dimensional probability vector .in, Another dimension corresponds to the "unknown" class. Semantic uncertainty entropy is calculated by the following formula: ; where, is the predicted probability that the image patch belongs to the th class. This formula is in the standard Shannon information entropy form.

[0037] When the network's classification prediction for an image patch presents a highly dispersed probability distribution (i.e., multiple class probabilities are close to each other), the value will be correspondingly higher, which objectively reflects the degree of ambiguity of the system's understanding of the content of this region.

[0038] Second, the dynamic instability entropy is calculated. The system first uses a computationally efficient optical flow estimation algorithm (e.g., the TV-L1 algorithm or a miniaturized optical flow neural network) to calculate the two-dimensional velocity vector of each pixel point within each image patch , forming a dense optical flow field . Then, the spatial gradient of this optical flow field is calculated, i.e., the Jacobian matrix . For an image patch, this matrix can be calculated at its center point or be the average of the Jacobian matrices of all points within the block: ; The dynamic instability entropy is defined as the variance of the two real eigenvalues and of the matrix , and the calculation formula is as follows: ; where, is the arithmetic mean of the two eigenvalues.

[0039] This index is used to measure the non-uniformity of the local motion pattern. For example, a rigid motion with overall translation or uniform rotation has a smooth gradient change in the optical flow field, the eigenvalues are close to , and the value is low; while a non-rigid complex motion containing vortex, tearing or random shaking has a sharp gradient change in the optical flow field, the eigenvalues differ significantly, resulting in a higher value.

[0040] Third, the task relevance entropy is calculated. This entropy value reflects the potential impact of the uncertainty of a region on the current navigation task. First, the system needs to map the two-dimensional image patch Mapping to the three-dimensional space under the vehicle coordinate system, this can be achieved by image depth information (e.g. provided by binocular vision or radar) and camera internal and external parameters. The three-dimensional space point corresponding to the center point of the image block is obtained Meanwhile, the current planning path is obtained from the path planning module 50 Task correlation entropy is calculated by the following exponential decay function: ; wherein, represents the shortest Euclidean distance between the three-dimensional point and all points on the path ; is a positive constant, which is a distance attenuation factor, and its value determines the range of the area of interest; the closer the area to the path, the higher the value, indicating that the uncertainty of the area has a greater potential impact on the execution of the task.

[0041] Referring to the accompanying Figure 3 , Figure 3 is the cognitive entropy map generation process diagram of the embodiment of the present application. In the S2 step, the system fuses the three original entropy components output in the S1 step to generate the final cognitive entropy map. This step is performed by the cognitive entropy generation module 20, and the core is a two-stage processing flow: nonlinear cross modulation and weighted fusion.

[0042] For each image block , its original entropy components constitute a three-dimensional vector .

[0043] The first stage is nonlinear cross modulation. The goal of this stage is to simulate a more complex judgment logic, that is, there is mutual influence between different types of uncertainty, rather than a simple linear superposition relationship. The cross modulation submodule in the cognitive entropy generation module 20 applies a set of preset rules to transform the original entropy vector into a modulated entropy vector . In a specific embodiment, the modulation process is implemented by the following logic: First, define the synergistic enhancement rule. This rule is used to handle the case where the semantic ambiguity region and the dynamic instability region appear simultaneously, and the potential risk of this case is considered to be higher than the sum of the risks when the two regions appear independently. The system presets a semantic uncertainty threshold and a dynamic instability threshold . When the entropy components of an image block satisfy and at the same time, its modulated semantic uncertainty entropy will be nonlinearly amplified, and the calculation formula is: ; wherein, is a positive enhancement factor. In other cases, .

[0044] Secondly, task inhibition rule is defined. This rule is used to reduce the attention of high uncertainty areas far away from the current navigation task, so as to concentrate the computing resources on the areas directly related to the task. The system presets a task correlation threshold When the task correlation entropy of an image block is lower than the threshold, it indicates that the area is far away from the current path, and its uncertainty is less urgent. At this time, its modulated semantic and dynamic entropy will be inhibited: ; ; wherein, is an inhibition factor with a value between [0, 1). The priority of this inhibition rule is higher than that of the synergy enhancement rule. For the task correlation entropy itself, its modulated value remains the same as the original value, i.e. .

[0045] The second stage is weighted fusion. After cross modulation, the system uses a set of weight coefficients to linearly weight and sum the modulated entropy vector to obtain the final cognitive entropy value of the image block. This calculation is completed by the weighted fusion sub-module: ; wherein, the weight coefficients represent the importance of semantic, dynamic and task correlation in the comprehensive risk assessment, and their sum is 1. These weight coefficients are not fixed, but are dynamically adjusted by the subsequent self-calibration module 60 according to the actual performance of the system.

[0046] Arrange the cognitive entropy values of all image blocks in their corresponding spatial positions, and a complete two-dimensional cognitive entropy map that is aligned with the original input image in size and space can be generated. Each pixel value of this map intuitively reflects the comprehensive uncertainty or potential risk level of the corresponding scene area.

[0047] In step S3, the system analyzes the cognitive entropy map generated in step S2 to make a decision on the allocation of computational resources. This step is performed by the arbitration and resource allocation module 30, which is centered on a double-threshold-based decision mechanism for switching between different system operating modes and precisely specifying the areas that need further analysis.

[0048] First, the arbitration and resource allocation module 30 receives the complete cognitive entropy map. This map can be represented as a two-dimensional matrix where each element corresponds to the cognitive entropy value of an image block. The primary task of this module is to calculate the maximum value of the cognitive entropy map of the current frame ; This maximum value is a quick assessment indicator of the overall risk level of the current scene. The system has two key numerical threshold values preset internally: a low-risk threshold and a high-risk threshold , and .

[0049] The decision logic unit within the arbitration and resource allocation module 30 generates instructions based on the comparison results of and these two thresholds. The decision flow is as follows: Maintenance of alert cruise mode: If the maximum value of the current cognitive entropy map is less than or equal to the low-risk threshold , i.e. This indicates that there are no significant sources of uncertainty in the current scene, and the environment is judged to be safe and predictable. At this time, the module outputs a maintenance instruction, causing the entire system to remain in the low-power "alert cruise mode". In this mode, the refined perception module 40 remains dormant, and the system relies only on the basic perception information provided by the raw entropy calculation module 10 and simple obstacle avoidance strategies for navigation, thereby maximizing the conservation of computational resources.

[0050] Activation of refined analysis mode: If the maximum value of the current cognitive entropy map is greater than the high-risk threshold , i.e. This indicates that there is at least one area in the scene whose overall uncertainty level has exceeded the range that the system can tolerate, and immediate in-depth analysis is required. At this time, the arbitration and resource allocation module 30 performs two operations: First, it generates a mode switching instruction to switch the system from "alert cruise mode" to "refined analysis mode", which is used to activate the refined perception module 40.

[0051] Second, the module iterates through the entire cognitive entropy map Identify all entropy values ​​exceeding the high-risk threshold. Image patches. The set of these image patches is defined as "Regions of Interest" (ROIs). Specifically, the set of ROIs... The definition of is: ; The module will use this collection containing the coordinates of all high-risk image patches. As a computing resource allocation instruction, it is sent together with the fine-grained perception module 40.

[0052] Intermediate state processing: If the current cognitive entropy graph has reached its maximum value... Located between two thresholds, i.e. This indicates a degree of uncertainty in the scenario, but not yet reaching a level of urgency requiring immediate detailed analysis. In this situation, the system can adopt several strategies. In one embodiment, the system chooses to maintain the "alert cruise mode," but increases the frequency of perception or tracks the area more closely, constituting a third intermediate state or "alert mode." In another embodiment, to simplify processing, this interval is merged into the alert cruise mode, meaning that mode switching is only triggered when the entropy value clearly exceeds a high-risk threshold.

[0053] Through this arbitration mechanism, the system achieves on-demand allocation of computing resources, only calling upon heavyweight computing resources for targeted analysis when a clear, high-uncertainty region is identified, thus achieving a balance between ensuring security and maintaining high efficiency.

[0054] In step S4, the system performs high-precision analysis on the region of interest identified in step S3. This step is executed by the refined perception module 40, whose core function is to call a large-capacity, high-precision deep learning model after receiving the activation instruction from the arbitration and resource allocation module 30 to perform in-depth analysis of the specified image region to eliminate uncertainty.

[0055] When the system switches to "detailed analysis mode", the detailed perception module 40 is activated. This module receives two key inputs: raw, unprocessed visual images. and the set of region of interest coordinates sent by the arbitration module. .

[0056] First, the module is based on the set of coordinates of the region of interest. From the original high-resolution image The corresponding image sub-regions are cropped or extracted from the image. These sub-regions constitutes the only input to a high-precision perception model. In this way, computationally intensive analysis is strictly confined to the parts of the scene that most require attention, avoiding unnecessary processing of the entire image.

[0057] These image sub-regions are then fed into a pre-defined high-precision perception model. This model can be a large-scale Vision Transformer (ViT) model, a convolutional neural network with attention mechanisms (such as ResNeXt-w / SE), or a multi-task learning model designed to provide a depth and accuracy of analysis far beyond the original entropy computation module 10. The model processes each region of interest input, outputting a structured refined perception result .

[0058] In a specific embodiment, the refined perception result is a data structure containing multiple fields, the specific content of which can include: High-confidence semantic class : The model outputs a semantic label with very high confidence, for example, accurately identifying a region labeled as "unknown" in the S1 step as "plastic bag blown by the wind" or "child hiding behind the shrubs". Its confidence is much higher than the output of the original entropy computation module.

[0059] Three-dimensional geometric properties : For regions identified as objects, the model estimates their precise three-dimensional bounding box (3D Bounding Box), including their position , size (length, width, height) and orientation angle in the vehicle coordinate system.

[0060] Velocity vector : For dynamic objects, the model estimates their velocity vector in three-dimensional space, providing more complete motion information than the two-dimensional optical flow field calculated in the S1 step.

[0061] Internal state or intention prediction : For specific categories of objects (such as pedestrians or vehicles), the model can further predict their future behavior intentions, such as the probability of "crossing the road soon".

[0062] After generating the refined perception result , these high-deterministic information is used to update the system's global world model. The global world model is a dynamic database that stores the properties and states of all known objects in the environment. For example, if the refined perception module identifies a new dynamic obstacle, its and Information is added or updated to the world model for use by the path planning module 50. If a high-entropy region is confirmed to be harmless (e.g., a shadow or water stain), the corresponding region in the world model is marked as traversable.

[0063] Finally, the refined perception module 40 generates refined perception results along with their corresponding region-of-interest coordinates, which are transmitted to the self-calibration module 60 to provide a supervisory signal for subsequent closed-loop learning. This step does not involve complex mathematical formulas, and its core is to use the reasoning ability of large models to convert low-quality uncertain information into high-quality structured deterministic knowledge.

[0064] In the S5 step, the system plans a path that is both efficient and safe based on the comprehensive understanding of the environment. This step is performed by the path planning module 50, and its core feature is that the generation of the path not only considers traditional geometric constraints but also actively avoids potential unknown risks quantified by the cognitive entropy map.

[0065] The input of the path planning module 50 is multi-source: it receives the global world model updated by the refined perception module 40 in the S4 step, which contains the precise location, size, and motion state of known obstacles in the environment; at the same time, it also receives the cognitive entropy map generated by the cognitive entropy generation module 20 in the S2 step. The path planning module uses a search-based algorithm, such as the A algorithm, DLite algorithm, or Hybrid A* algorithm, to find the optimal path from the current location to the target point under a comprehensive cost function.

[0066] The key of the invention lies in the construction of the comprehensive cost function . For any candidate path , its total cost is defined as the weighted sum of multiple cost terms. In one embodiment, the function has the following form: ; where , , are the weight coefficients of each cost term, used to balance the relationship between path length, collision risk, and unknown risk. These weights can be preset according to different driving styles or scenarios.

[0067] The specific definitions of each cost term are as follows: Path length cost : This is the most basic cost term, used to ensure the economy of the path. It is defined as the total length of the path : ; where is the arc length infinitesimal on the path. In a discretized implementation, this is typically the sum of the lengths of all line segments on the path.

[0068] Obstacle collision cost This cost term is used to penalize paths that come close to or cross known obstacles. The global world model provides an obstacle map. For each point on the path , the distance to the nearest known obstacle is . The collision cost can be defined as the integral of a penalty function over the parts of the path where the distance to the obstacle is less than a safety threshold : ; where the penalty function is 0 when , and a sharply increasing value, e.g. linear or exponential, as .

[0069] Cognitive risk cost This is a unique cost term of the present invention, used to quantify the potential risk of traversing uncertain regions of the environment. This cost is directly derived from the cognitive entropy map . For each point on the path , one can find its corresponding entropy value on the cognitive entropy map. The cognitive risk cost is defined as the line integral of the cognitive entropy values along the path: ; This cost term makes the path planner naturally inclined to avoid regions of the cognitive entropy map that appear highlighted, even if these regions are not yet marked as obstacles in the current global world model. It embodies a kind of active avoidance of the “unknown”.

[0070] The path planning module 50 executes a search algorithm to find a path that minimizes the integrated cost function , taking into account kinematic constraints.

[0071] It is worth noting that the role of the cognitive entropy map is dynamic. When the system is in “alert cruise mode”, the cognitive entropy map is used to guide the path planning process. ​​High-value regions in the map will dominate path planning, causing the vehicle to make early, conservative evasive maneuvers. Once the system switches to the "fine resolution mode" and completes step S4, if a high-entropy region is confirmed to be harmless (e.g., a shadow of the ground) by the fine resolution perception module 40, the global world model will be updated, and the path planning module 50 can use this new deterministic information to plan a more aggressive, efficient path. Conversely, if it is confirmed to be a high-risk obstacle, its evasive behavior will be confirmed and strengthened. This mechanism enables the navigation behavior to be dynamically adjusted according to the depth of environmental cognition.

[0072] Referring to the accompanying drawings Figure 4 , Figure 4 is a schematic diagram of the traceability self-calibration working principle of an embodiment of the present application. In step S6, the system uses the results of high-precision perception to inversely optimize its own uncertainty evaluation model, forming a complete learning closed loop. This step is performed by the self-calibration module 60, and the core is to establish an optimization path from "fine resolution results" to "model parameters", so that the system can learn from experience and continuously improve the accuracy of its judgment of environmental uncertainty.

[0073] This step is only triggered after step S4 is performed, i.e., after the system has processed one or more regions of interest. The self-calibration module 60 receives two key inputs: fine resolution perception results for the regions of interest from the fine resolution perception module 40 , and the original cognitive entropy values of these regions calculated in step S2 .

[0074] First, the module needs to convert the fine resolution perception results into a "real risk label" that can be compared with the cognitive entropy value. This conversion process relies on a set of pre-set rules. For example: If indicates that the region is a high-risk dynamic obstacle (such as a pedestrian crossing), then is assigned a value of 1.0.

[0075] If indicates that the region is a non-threatening artifact (such as a shadow of the ground or a water stain), then is assigned a value of 0.0.

[0076] For objects between the two, such as static obstacles on the roadside, can be assigned an intermediate value between 0 and 1.

[0077] Next, the loss function calculation unit within the self-calibration module 60 calculates the loss function for each processed region of interest , compute its predicted cognitive entropy value from the true risk label . This difference is quantified by a specially designed loss function . To impose different penalties on different types of errors, the loss function is designed to be asymmetric: ; where and are the false positive and false negative penalty weights, respectively. In safety-critical applications, it is common to set because it is much more serious to miss a real danger than to issue a false alarm on a harmless object. Here, we denote .

[0078] The total loss of the system is the sum or average of the losses of all processed regions of interest. The goal of the self-calibration module 60 is to adjust the internal parameters of the cognitive entropy generation module 20 to minimize this total loss. These tunable parameters include the weight coefficients of the weighted fusion stage and the coefficients of the cross-modulation stage (e.g., the enhancement coefficient and the suppression factor ).

[0079] The optimization process of the parameters is performed by a parameter optimization unit within the module, typically using a gradient-based optimization algorithm such as stochastic gradient descent (SGD) or its variants (e.g., Adam). The iterative update rule for optimization is as follows: ; where is the current parameter value, is the updated parameter value, is the learning rate, and is the gradient of the total loss function with respect to the parameter .

[0080] Since the computation of the cognitive entropy (including cross-modulation and weighted fusion) is differentiable with respect to the parameters , this gradient can be computed by the backpropagation algorithm. Specifically, the gradient will be propagated back from the loss function to the cognitive entropy , then to the modulated entropy components , and finally to the parameters .

[0081] By repeatedly executing the complete cycle of S1 to S6, the system continuously discovers uncertainty, in-depth analyzes, and modifies its own uncertainty evaluation model according to the analysis results in the continuous interaction with the environment. This enables the cognitive entropy generation module 20 to gradually adapt to the specific environment type and task requirements it is in, thereby making more and more accurate risk predictions.

[0082] In order to further illustrate the synergistic working process of the technical solutions of the present application, a specific working scene example will be described below.

[0083] This embodiment sets an autonomous delivery robot as the bearing platform of the system of the present application. The robot is executing a delivery task in a semi-open urban park, and its task path needs to pass through a boulevard. At this time, it is in the afternoon, and the sunlight passing through the leaves forms a patchy and moving light and shadow on the ground.

[0084] Stage one: regular cruising and preliminary perception: The robot is driving on a road that is open and has a clear road surface. In the image input by the visual sensor, most of the area is road, grass, and distant buildings.

[0085] The original entropy calculation module 10 processes each frame of image. For the road and grass areas, the lightweight CNN model can classify with high confidence, so the semantic uncertainty entropy of these areas is extremely low. Since there are no fast-moving objects in the scene, the optical flow field is gentle, and the dynamic instability entropy is also low. According to the planned path, the task relevance entropy of these areas is normal. After the cognitive entropy generation module 20 fuses these low-value original entropy components, the generated cognitive entropy map is overall low in value.

[0086] The arbitration and resource allocation module 30 detects that the maximum value of the cognitive entropy map is far below the high-risk threshold . Therefore, the system maintains the "alert cruising mode", and the in-depth perception module 40 remains dormant. The path planning module 50 only plans a smooth path along the centerline of the road according to the basic geometric obstacle avoidance and the extremely low risk cost in the cognitive entropy map.

[0087] Stage two: encountering uncertainty events and upgrading processing: The robot enters the boulevard. At this time, a gust of wind blows, causing the tree shadow in front to quickly sway on the road surface, and a discarded, semi-transparent plastic bag is also blown up and irregularly tumbles in the air.

[0088] The original entropy computation module 10 experiences a significant change in its output when processing the image region containing the swaying tree shadow and the plastic bag. For the tree shadow region, the dynamic instability entropy rises sharply due to its fast morphological and illumination changes. For the plastic bag region, the lightweight CNN model fails to classify it into any known category due to its irregular shape and high transparency, resulting in a semantic uncertainty entropy that skyrockets; meanwhile, its irregular motion also makes its value very high.

[0089] The cognitive entropy generation module 20 receives these high-value entropy components. For the plastic bag region, since its and both exceed the threshold values, the cross-modulation submodule applies the synergistic amplification rule, nonlinearly amplifying the risk representation of this region. Finally, the generated cognitive entropy map forms two distinct high-light patches at the locations corresponding to the swaying tree shadow and the plastic bag.

[0090] The arbitration and resource allocation module 30 detects that the maximum value of the cognitive entropy map (originating from the plastic bag region) has exceeded the high-risk threshold . The system immediately switches from the "alert cruise mode" to the "fine-resolution analysis mode" and defines the image coordinates corresponding to the two high-light patches as the regions of interest, instructing the fine-resolution perception module 40 to analyze them.

[0091] Before the fine-resolution perception module 40 completes the analysis, the path planning module 50 has already received this highlighted cognitive entropy map. The cognitive risk cost in its comprehensive cost function makes the planner immediately generate a temporary avoidance path that actively and significantly avoids the two high-entropy regions.

[0092] The fine-resolution perception module 40 is activated and calls a large-scale visual model to process only the two input regions of interest. After a few hundred milliseconds, the model outputs the fine-resolution perception results: the first region is identified with high confidence as "ground light shadow, no entity, no threat"; the second region is identified as "plastic bag, extremely light in mass, non-rigid, low collision risk."

[0093] Stage Three: Decision Correction and Self-Calibration Learning: The fine-resolution perception results are sent to the path planning module 50 and the self-calibration module 60.

[0094] Upon receiving the certainty information that "the ground shadow is not a threat", the path planning module 50 updates its internal global world model, marking the area as traversable. It re-plans the path, which no longer needs to avoid the shadow area, but still keeps a safe distance from the area identified as "plastic bag", resulting in a final path that is more optimal than the temporary path, but more cautious than the regular path.

[0095] The self-calibration module 60 receives the complete records of the two events. For the shadow area, its original prediction was high, but the true risk label was determined to be 0.0, constituting a "false positive". For the plastic bag area, its original prediction was extremely high, while the refined result determined its risk to be low, e.g. , also constituting a "false positive" but of a lesser degree.

[0096] The self-calibration module 60 calculates the loss value based on the two events, and fine-tunes the internal parameters of the cognitive entropy generation module 20 through the backpropagation algorithm. For example, it might slightly reduce the weight of the dynamic instability entropy , or adjust the enhancement coefficient of cross modulation , so that in the future when the system encounters similar "high-frequency dynamic disturbance without entity", its response amplitude of cognitive entropy can be more accurate, reducing unnecessary emergency avoidance.

[0097] Through this complete process, the system not only safely handles complex unknown situations, but also learns from the experience and improves its ability to handle similar scenarios in the future.

Claims

1. A method for optimizing machine vision navigation in complex environments, characterized in that, The method comprises the following steps: acquiring a visual image of a current scene, and calculating a plurality of original entropy components respectively representing semantic uncertainty, dynamic instability and task relevance based on the visual image; cross-modulating and fusing the plurality of original entropy components to generate a cognitive entropy map representing the comprehensive uncertainty degree of each region in the scene; arbitrating based on the cognitive entropy map to determine one or more regions of interest in the scene, and dynamically allocating computing resources accordingly; invoking the computing resources to perform fine-grained perception on the one or more regions of interest to generate fine-grained perception results; jointly using the cognitive entropy map and the fine-grained perception results to perform risk-averse path planning; according to the comprehensive uncertainty degree represented by the fine-grained perception results and the cognitive entropy map, performing traceable self-calibration on the process of generating the cognitive entropy map.

2. The method of claim 1, wherein, The calculation of the plurality of original entropy components comprises: for an image block in the visual image, calculating the probability distribution over a plurality of known semantic categories and an unknown category, and determining the semantic uncertainty entropy based on the Shannon entropy of the probability distribution; calculating the optical flow field within the image block, and determining the dynamic instability entropy based on the variance of the eigenvalues of the Jacobian matrix of the optical flow field; calculating the shortest distance between the projection point of the image block in the three-dimensional space and the current navigation path, and determining the task relevance entropy based on the exponential decay function of the shortest distance.

3. The method of claim 1, wherein, The cross-modulation and fusion of the plurality of original entropy components comprises: when the semantic uncertainty entropy and the dynamic instability entropy of a region are both higher than a preset threshold, nonlinearly enhancing the uncertainty representation; when the task relevance entropy of a region is lower than a preset threshold, actively suppressing the contribution of the semantic uncertainty entropy and the dynamic instability entropy to subsequent calculations.

4. The method of claim 3, wherein, The cross-modulation and fusion of the plurality of original entropy components further comprises: weighting and summing the plurality of original entropy components after cross-modulation by a set of weight coefficients to obtain the final cognitive entropy value; wherein the weight coefficients are dynamically adjusted by the traceable self-calibration process.

5. The method of claim 1, wherein, The arbitration based on the cognitive entropy map comprises: comparing the entropy values of each point in the cognitive entropy map with a preset low-risk threshold and a high-risk threshold; if the entropy values of all points are lower than the low-risk threshold, the system is maintained in a alert cruise mode; if the entropy value of any point is higher than the high-risk threshold, the system is switched to a fine-grained analysis mode, and the region where the point is located is determined as the region of interest.

6. The method of claim 1, wherein, The invocation of the computing resources to perform fine-grained perception on the one or more regions of interest comprises: invoking a preset high-precision perception model; the high-precision perception model only processes the image part determined as the region of interest to output the fine-grained perception results; updating a global world model used for navigation using the fine-grained perception results.

7. The method of claim 1, wherein, The risk-averse path planning comprises: using the cognitive entropy map as an additional risk cost map; constructing a comprehensive cost function including path length cost, obstacle collision cost and integral cost of cognitive entropy along the path; Search and determine the optimal navigation path based on the comprehensive cost function.

8. The method of claim 1, wherein, The process of generating the cognitive entropy map includes: The refined perception result is regarded as a true situation description of the region of interest; A loss function is defined to quantify the difference between the comprehensive uncertainty degree predicted by the cognitive entropy map and the true situation; Based on the loss function, an optimization algorithm is used to adjust the weight coefficients and internal parameters of the cross-modulation module used to generate the cognitive entropy map.

9. The method of claim 8, wherein, The loss function is used to: When the comprehensive uncertainty degree predicted by the cognitive entropy map is high, but the refined perception result indicates a low-risk situation, it is determined as a false alarm and a first type of loss is generated; When the comprehensive uncertainty degree predicted by the cognitive entropy map is low, but the refined perception result indicates a high-risk situation, it is determined as a missed alarm and a second type of loss is generated.

10. The system for optimizing machine vision navigation in complex environments according to any one of claims 1 to 9, wherein, It includes: An original entropy calculation module is used to obtain a visual image of the current scene, and based on the visual image, multiple original entropy components representing semantic uncertainty, dynamic instability, and task relevance are calculated; A cognitive entropy generation module is used to cross-modulate and fuse the multiple original entropy components to generate a cognitive entropy map representing the comprehensive uncertainty degree of each region in the scene; An arbitration and resource allocation module is used to arbitrate based on the cognitive entropy map to determine one or more regions of interest in the scene, and accordingly dynamically allocate computing resources; A refined perception module is used to perform refined perception on the one or more regions of interest after receiving the allocated computing resources to generate a refined perception result; A path planning module is used to jointly use the cognitive entropy map and the refined perception result for risk-averse path planning; A self-calibration module is used to perform retroactive self-calibration on the cognitive entropy generation module according to the refined perception result and the comprehensive uncertainty degree represented by the cognitive entropy map.

Images