Vehicle adaptive navigation path planning method and system based on multi-view vision

Abstract

The application discloses a kind of vehicle adaptive navigation path planning method and system based on multi-vision, and the multi-angle environmental image is collected by deploying multiple visual sensors with different installation positions and orientations, the feature point matching of adjacent field of view overlap region is used to complete panoramic stitching, combined with static road element extraction and dynamic target detection and identification to generate semantic topology graph, and a four-dimensional space-time environment model is constructed accordingly, the initial planning path is mapped into the model for space-time scanning to identify static geometric incompatibility conflict section and dynamic target space-time overlap conflict section, and then the static geometric parameter is used as boundary constraint, and the dynamic motion state sequence is used as motion obstacle constraint to generate obstacle avoidance path curve, the curvature continuity is checked to screen safe and feasible path, and the adaptive navigation path is formed by smooth splicing with reserved section, which improves the vehicle circumferential panoramic environment perception coverage capability and real-time adaptability of path planning in complex scene.

Description

Technical Field

[0001] This invention relates to the field of intelligent vehicle equipment and autonomous driving navigation technology, and in particular to a vehicle adaptive navigation path planning method and system based on multi-view vision. Background Technology

[0002] In the fields of autonomous driving and intelligent transportation, vehicle navigation path planning is a core component for achieving safe driving. Current mainstream solutions primarily rely on monocular or binocular vision sensors to collect road environment information and combine it with pre-set map data to generate paths. However, monocular vision systems are limited by their single-view observation range, easily creating blind spots when dealing with complex intersections, obstructed road sections, and multi-lane intersections, resulting in incomplete environmental perception information. While binocular vision possesses depth measurement capabilities, it still falls short in long-distance target recognition and large field-of-view coverage, failing to meet the accuracy requirements of high-level autonomous driving for comprehensive environmental awareness. Simultaneously, existing path planning algorithms are mostly based on static road models. When faced with unexpected situations such as construction zones, temporary obstacles, and dynamic changes in traffic flow, they lack adaptive feedback to real-time visual information and dynamic path replanning capabilities, easily leading to a disconnect between the planned results and actual road conditions, impacting driving safety and traffic efficiency.

[0003] Therefore, there is an urgent need to provide a technical solution to address the above problems. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a vehicle adaptive navigation path planning method and system based on multi-view vision.

[0005] In a first aspect, the present invention provides a vehicle adaptive navigation path planning method based on multi-view vision, the technical solution of which is as follows: Multiple visual sensors deployed on a vehicle acquire multi-view environmental images simultaneously at the same time. The multiple visual sensors are installed at different positions and orientations on the vehicle, and the fields of view of adjacent visual sensors have a preset overlap rate, together forming a panoramic field of view covering the circumference of the vehicle. The feature points in the overlapping areas corresponding to the preset overlap rate between the fields of view of the adjacent visual sensors in the multi-view environmental image are matched. Based on the matching results, the multi-view environmental images are stitched into a panoramic environmental image. The geometric parameters and semantic attributes of static road elements are extracted from the panoramic environmental image, and the position, velocity vector and category information of dynamic targets are detected and identified. The semantic topology map representing the complete state of the environment around the vehicle at the current moment is fused and generated. Based on the semantic topology graph generated at multiple consecutive time points, a four-dimensional spatiotemporal environment model is constructed, which includes the motion state sequence of the dynamic target at consecutive time steps and the change history of the static road elements. Obtain the initial planned path, map the initial planned path to the four-dimensional spatiotemporal environment model, perform spatiotemporal scanning along the mileage direction of the initial planned path, and identify the first conflict segment where the geometric parameters of the initial planned path are incompatible with those of the static road elements, and the second conflict segment where the initial planned path and the motion state sequence of the dynamic target generate spatiotemporal overlap. Using the geometric parameters of the static road elements involved in the first conflict segment as boundary constraints and the motion state sequence of the dynamic target involved in the second conflict segment as motion obstacle constraints, an obstacle avoidance path curve is generated in the four-dimensional spatiotemporal environment model from the current position of the vehicle to the return point after the conflict segment. The curvature continuity of the obstacle avoidance path curve is checked, and candidate paths whose curvature does not meet the kinematic constraints of the vehicle are eliminated. A safe and feasible path is determined, and the safe and feasible path is smoothly spliced ​​with the reserved segments in the initial planned path that do not conflict, to form an adaptive navigation path for controlling the vehicle's movement.

[0006] Secondly, the present invention provides a vehicle adaptive navigation path planning system based on multi-view vision, the technical solution of which is as follows: The acquisition module is used to acquire multi-view environmental images synchronously collected by multiple vision sensors deployed on the vehicle at the same time. The multiple vision sensors are installed at different positions and orientations on the vehicle, and the fields of view of adjacent vision sensors have a preset overlap rate, which together constitute a panoramic field of view covering the circumference of the vehicle. The fusion module is used to match feature points in the overlapping areas corresponding to the preset overlap rate between the fields of view of the adjacent visual sensors in the multi-view environmental image, stitch the multi-view environmental image into a panoramic environmental image based on the matching result, extract the geometric parameters and semantic attributes of static road elements from the panoramic environmental image, detect and identify the position, velocity vector and category information of dynamic targets, and fuse to generate a semantic topology map that represents the complete state of the environment around the vehicle at the current moment. The construction module is used to construct a four-dimensional spatiotemporal environment model containing the motion state sequence of the dynamic target in consecutive time steps and the change history of the static road elements based on the semantic topology graph generated at multiple consecutive time steps. The identification module is used to obtain the initial planned path, map the initial planned path to the four-dimensional spatiotemporal environment model, perform spatiotemporal scanning along the mileage direction of the initial planned path, and identify the first conflict segment where the geometric parameters of the initial planned path are incompatible with those of the static road elements, and the second conflict segment where the initial planned path and the motion state sequence of the dynamic target generate spatiotemporal overlap. The generation module is used to generate an obstacle avoidance path curve from the current position of the vehicle to the return point after the conflict segment in the four-dimensional spatiotemporal environment model, using the geometric parameters of the static road elements involved in the first conflict segment as boundary constraints and the motion state sequence of the dynamic target involved in the second conflict segment as motion obstacle constraints. The planning module is used to check the curvature continuity of the obstacle avoidance path curve, eliminate candidate paths whose curvature does not meet the kinematic constraints of the vehicle, determine a safe and feasible path, and smoothly connect the safe and feasible path with the reserved segments in the initial planned path that do not conflict, to form an adaptive navigation path for controlling the vehicle's driving.

[0007] The technical solution of this invention simultaneously acquires multi-view environmental images by deploying multiple visual sensors at different installation locations and orientations. It completes panoramic stitching by matching feature points in adjacent overlapping fields of view. Combined with static road element extraction and dynamic target detection and recognition, it generates a semantic topology map and constructs a four-dimensional spatiotemporal environment model. The initial planned path is mapped to the model for spatiotemporal scanning to identify static geometrically incompatible conflict segments and dynamic target spatiotemporally overlapping conflict segments. Then, using static geometric parameters as boundary constraints and dynamic motion state sequences as motion obstacle constraints, it generates obstacle avoidance path curves. After curvature continuity checks, safe and feasible paths are selected and smoothly stitched with the retained segments to form an adaptive navigation path. This solves the problems of limited field of view, insufficient long-distance recognition capability, and difficulty of static path planning algorithms in coping with real-time dynamic changes in road conditions in monocular or binocular vision systems. It improves the vehicle's circumferential panoramic environment perception coverage capability and the real-time adaptability of path planning in complex scenarios.

[0008] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0010] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a flowchart illustrating an embodiment of a vehicle adaptive navigation path planning method based on multi-view vision according to the present invention. Detailed Implementation

[0011] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0012] Figure 1 The diagram illustrates a flowchart of an embodiment of a vehicle adaptive navigation path planning method based on multi-view vision provided by the present invention, executed by a control terminal. Figure 1 As shown, the vehicle adaptive navigation path planning method based on multi-view vision includes the following steps: S1. Acquire multi-view environmental images simultaneously collected by multiple vision sensors deployed on the vehicle at the same time. The multiple vision sensors are installed at different positions and orientations on the vehicle, and the fields of view of adjacent vision sensors have a preset overlap rate, together forming a panoramic field of view covering the circumference of the vehicle.

[0013] Among them, visual sensors refer to optical sensing devices installed at different locations on the vehicle body to collect images of the surrounding environment. For example, in a test on January 1, 2026, vehicle A had a wide-angle camera installed on the front, rear, left, and right sides of the roof. The front-view camera was responsible for collecting images of the road in front of the vehicle, the rear-view camera collected images of vehicles following behind, and the left and right cameras covered the adjacent lanes to the left and right, respectively. Multi-view environmental images refer to a set of environmental images simultaneously collected by multiple visual sensors deployed on the vehicle from different installation positions and orientations at the same time. For example, when vehicle A passed through an intersection at 10:00 AM, the four cameras simultaneously triggered the shutters, obtaining four environmental images from different perspectives: a front-view image of the intersection, a rear-view image of the bus stop, an image of the construction site barrier on the left, and an image of the non-motorized vehicle lane on the right.

[0014] The preset overlap rate refers to the proportion of the overlapping area between two adjacent visual sensor fields of view to the field of view of a single sensor. For example, if vehicle A sets the field of view overlap rate between the front-view camera and the right-side camera to 20%, then the right-side one-fifth of the front-view camera's image and the left-side one-fifth of the right-side camera's image will capture the same piece of ground at the intersection.

[0015] S2. Match feature points in the overlapping areas corresponding to the preset overlap rate between the fields of view of the adjacent visual sensors in the multi-view environmental image. Based on the matching results, stitch the multi-view environmental images into a panoramic environmental image. Extract the geometric parameters and semantic attributes of static road elements from the panoramic environmental image, and detect and identify the position, velocity vector and category information of dynamic targets. Fuse and generate a semantic topology map that represents the complete state of the environment around the vehicle at the current moment.

[0016] The matching result refers to the set of corresponding feature point pairs found by extracting feature points from the overlapping areas of adjacent visual sensors and using nearest neighbor search. For example, pairing traffic marking corner points and road texture points extracted from the front-view camera and the right-side camera within a 20% overlap area yields 325 successfully matched feature point pairs, which constitute the matching result. The panoramic environment image refers to a continuous environmental image covering the vehicle's circumferential field of view, generated by matching feature points and stitching together multi-view environmental images acquired at the same time. For example, using 325 pairs of matched feature points to calculate the transformation matrix, the four images (front-view, rear-view, left-side, and right-side) are stitched together into a panoramic environment image with a resolution of 4096×1024, fully displaying a 360-degree surround view of vehicle A.

[0017] Static road elements refer to traffic infrastructure components in a road scene whose positions are fixed or change slowly; for example, the straight-ahead directional arrow marking in the center of an intersection, the traffic light pillars at the intersection, and the temporary guardrail in the construction area on the right side, all identified from a panoramic environmental image, are static road elements. Geometric parameters refer to quantitative indicators that numerically describe the spatial shape and position of static road elements; for example, the straight-ahead directional arrow marking measured from a panoramic environmental image is 4.5m long and 0.9m wide, with the arrow's vertex having three-dimensional coordinates of 125.3m east and 38.7m north in the world coordinate system—these are geometric parameters. Semantic attributes refer to the functional meaning and traffic rule category labels assigned to static road elements; for example, the straight-ahead directional arrow marking is assigned the type "lane instruction marking" and the driving restriction attribute of "straight only," while the temporary guardrail on the right is assigned the passable attribute of "cannot be crossed."

[0018] The semantic topology graph refers to a graph structure formed by constructing a static layer with the geometric parameters and semantic attributes of static road elements, constructing a dynamic layer with the position and velocity vectors and category information of dynamic targets, and connecting the two layers with spatial association edges in a unified coordinate system. For example, in the semantic topology graph corresponding to a panoramic environment image, the static layer contains straight-ahead directional arrow nodes and right-turn lane marking nodes, while the dynamic layer contains a bus node and an electric bicycle node. The straight-ahead directional arrow node and the bus node are connected by a spatial association edge indicating that the bus is currently on top of the straight-ahead directional arrow.

[0019] S3. Based on the semantic topology graph generated at multiple consecutive time points, construct a four-dimensional spatiotemporal environment model that includes the motion state sequence of the dynamic target at consecutive time steps and the change history of the static road elements.

[0020] Among them, the motion state sequence refers to the state trajectory formed by arranging the position and velocity vectors of the same dynamic target in a time sequence at multiple consecutive moments. For example, in the dynamic layer, the positions of the bus at three consecutive moments of 0s, 0.5s, and 1.0s are 125.3m east and 38.7m north, 130.1m east and 39.1m north, and 134.8m east and 39.5m north, respectively, and the velocity vectors are 9.6m / s east and 0.8m / s north, 9.4m / s east and 0.8m / s north, and 9.4m / s east and 0.8m / s north. These three consecutive states constitute the motion state sequence of the bus.

[0021] The four-dimensional spatiotemporal environment model refers to an environment model that includes three-dimensional space and one-dimensional time, formed by stacking three-dimensional occupancy grid maps generated at multiple consecutive time points along the time axis and associating the same dynamic target between consecutive time layers. For example, by stacking three-dimensional occupancy grid maps at three time points of 0s, 0.5s, and 1.0s along the time axis and identifying the occupancy grid of the same bus by comparing adjacent time layers, a four-dimensional spatiotemporal environment model is obtained that includes both the semantics of the straight-ahead guide arrow grid maintaining "passable road surface" from 0s to 1.0s and the motion information of the bus grid moving from 125.3m to 134.8m eastward.

[0022] S4. Obtain the initial planned path, map the initial planned path to the four-dimensional spatiotemporal environment model, perform spatiotemporal scanning along the mileage direction of the initial planned path, identify the first conflict segment where the geometric parameters of the initial planned path are incompatible with those of the static road elements, and the second conflict segment where the initial planned path and the motion state sequence of the dynamic target generate spatiotemporal overlap.

[0023] The initial planned route refers to a route from the starting point to the destination calculated in advance based on the global positioning system and preset map data when the vehicle starts. For example, if vehicle A sets its destination to the entrance of the shopping mall parking lot 800m ahead, the navigation system will calculate an initial planned route when it starts, which is to first go straight along the current road for 200m and then turn left into the commercial street and drive for 600m.

[0024] The first conflict segment refers to a continuous mileage segment on the initial planned path where the geometric parameters of the static road elements are spatially incompatible. For example, the four-dimensional spatiotemporal environment model shows that the straight lane on the initial planned path between mileage coordinates 150m and 165m is occupied by temporary guardrails from the construction area. The geometric parameters of the guardrails overlap with the space required for vehicle passage on the initial planned path, and the mileage segment between 150m and 165m is identified as the first conflict segment. The second conflict segment refers to a continuous mileage segment on the initial planned path where the spatiotemporal trajectory tube of the vehicle and the spatiotemporal region probability distribution of the dynamic target overlap simultaneously in time and space, and the collision risk index exceeds a preset threshold. For example, on the initial planned path between mileage 180m and 195m, the expected time interval for vehicle A to pass through at a preset speed profile overlaps with the expected time interval for the bus to occupy the mileage segment between mileage 180m and 195m obtained by extrapolating the bus motion state sequence. The collision risk index reaches 0.72 at mileage 188m, exceeding the preset threshold of 0.5, and the mileage segment between mileage 180m and 195m is identified as the second conflict segment.

[0025] S5. Using the geometric parameters of the static road elements involved in the first conflict segment as boundary constraints and the motion state sequence of the dynamic target involved in the second conflict segment as motion obstacle constraints, generate an obstacle avoidance path curve in the four-dimensional spatiotemporal environment model from the current position of the vehicle to the return point after the conflict segment.

[0026] Boundary constraints refer to the boundary restrictions of the feasible domain defined by the geometric parameters of static road elements when generating obstacle avoidance path curves. For example, the maximum lateral movement range of the obstacle avoidance path curve is defined by the passable lane edges on both sides of the first conflict section that are not occupied by temporary construction barriers, and the distance between any path point on the obstacle avoidance path curve and the lane edge must be greater than 1m half the width of vehicle A. Motion obstacle constraints refer to the path prohibition restrictions imposed by the spatiotemporally occupied area predicted by the dynamic target motion state sequence when generating obstacle avoidance path curves. For example, based on the bus motion state sequence, it is predicted that the bus will continuously occupy a rectangular area with eastward coordinates of 130m to 145m and northward coordinates of 38m to 41m after 1.5s to 3.0s, and the rectangular area is set as a motion obstacle constraint that path points cannot enter when generating obstacle avoidance path curves.

[0027] The obstacle avoidance path curve refers to a continuous path curve that starts from the vehicle's current position, bypasses the conflict section, and returns to the return point on the initial planned path. For example, vehicle A starts from its current position, changes lanes to the right, and bypasses the construction area and the area occupied by the bus along an arc with a minimum radius of curvature of 25m. At a distance of 210m, it returns to the center line of the straight lane of the initial planned path. This arc constitutes the obstacle avoidance path curve.

[0028] S6. Perform a curvature continuity check on the obstacle avoidance path curve, eliminate candidate paths whose curvature does not meet the vehicle's kinematic constraints, determine a safe and feasible path, and smoothly connect the safe and feasible path with the reserved segments in the initial planned path that do not conflict, to form an adaptive navigation path for controlling the vehicle's movement.

[0029] A safe and feasible path refers to an obstacle avoidance path curve whose curvature, after a curvature continuity check, satisfies the vehicle's kinematic constraints throughout its entire length. For example, if the curvature of the obstacle avoidance path curve changes continuously from 0 to 0.04m⁻¹ along the entire path, and the maximum curvature of 0.04m⁻¹ corresponds to a turning radius of 25m, which is greater than the minimum turning radius of vehicle A (6m) and the rate of curvature change does not exceed the maximum rate of change of the steering wheel angle, then the obstacle avoidance path curve is determined to be a safe and feasible path. An adaptive navigation path refers to the final path used to control vehicle movement, formed by smoothly splicing the safe and feasible path with the unconflicted reserved segments of the initial planned path. For example, if the endpoint of the safe and feasible path at mileage 210m is smoothly spliced ​​with the unconflicted reserved segment of the initial planned path from mileage 210m to 800m through a cubic spline transition curve that satisfies positional continuity, tangent vector direction continuity, and curvature continuity, the resulting complete path from 0m to 800m constitutes the adaptive navigation path.

[0030] The technical solution of this embodiment synchronously acquires multi-view environmental images by deploying multiple visual sensors at different installation locations and orientations. It completes panoramic stitching by matching feature points in adjacent overlapping fields of view. Combined with static road element extraction and dynamic target detection and recognition, it generates a semantic topology map and constructs a four-dimensional spatiotemporal environment model. The initial planned path is mapped to the model for spatiotemporal scanning to identify static geometrically incompatible conflict segments and dynamic target spatiotemporally overlapping conflict segments. Then, using static geometric parameters as boundary constraints and dynamic motion state sequences as motion obstacle constraints, it generates obstacle avoidance path curves. After curvature continuity checks, safe and feasible paths are selected and smoothly stitched with the retained segments to form an adaptive navigation path. This solves the problems of limited field of view, insufficient long-distance recognition capability, and difficulty of static path planning algorithms in coping with real-time dynamic changes in road conditions in monocular or binocular vision systems. It improves the vehicle's circumferential panoramic environment perception coverage capability and the real-time adaptability of path planning in complex scenarios.

[0031] In one alternative approach, the step of fusing and generating a semantic topological graph representing the complete state of the vehicle's surrounding environment at the current moment includes: The geometric parameters and semantic attributes of the static road elements are used to construct a static layer containing node coordinates and semantic labels.

[0032] In this context, a static layer refers to a layer in a semantic topology map that contains node coordinates and semantic labels, constructed from the geometric parameters and semantic attributes of static road elements. For example, in a semantic topology map, a straight-ahead directional arrow marking is constructed as a static layer node with node coordinates of 125.3m east and 38.7m north and a semantic label of "straight-ahead directional arrow," and a right-turn lane marking is constructed as a static layer node with node coordinates of 127.1m east and 37.9m north and a semantic label of "right-turn lane line." All static road element nodes together constitute a static layer.

[0033] The position, velocity vector, and category information of the dynamic target are used to construct a dynamic layer containing node motion attributes and category labels.

[0034] In this context, a dynamic layer refers to a layer in a semantic topology graph that contains node motion attributes and category labels, constructed from the position, velocity vectors, and category information of dynamic targets. For example, in a semantic topology graph, a bus is constructed as a dynamic layer node with position coordinates of 125.3m east and 38.7m north, velocity vectors of 9.6m / s east and 0.8m / s north, and a category label of "large passenger vehicle." An electric bicycle is constructed as a dynamic layer node with position coordinates of 127.5m east and 36.2m north, velocity vectors of 5.2m / s east and 0.1m / s north, and a category label of "non-motorized vehicle."

[0035] The static layer and the dynamic layer are spatiotemporally aligned in a unified world coordinate system, and the aligned static layer nodes and dynamic layer nodes are connected by spatial association edges to form the semantic topology graph.

[0036] In the above-mentioned optional methods, static road elements and dynamic targets are further constructed as static layers and dynamic layers respectively, and spatiotemporal alignment is completed under a unified world coordinate system. A semantic topology graph is formed by connecting spatially related edges, so that the road environment structure and moving target attributes can be hierarchically organized and relatedly expressed, thereby improving the clarity of environmental state representation and the rationality of information integration.

[0037] In one alternative approach, the geometric parameters of the static road element include cubic polynomial coefficients characterizing the lane centerline and the lane width, and the semantic attributes include lane line type and driving direction restrictions.

[0038] The coefficients of the cubic polynomial refer to the coefficients used to fit the lane centerline curve equation y=a0+a1x+a2x. 2 +a3x 3 The four parameters a0, a1, a2, and a3 are used in the model. For example, fitting the centerline of the lane where vehicle A is located in a panoramic environment image yields cubic polynomial coefficients a0=2.35, a1=0.12, a2=-0.003, and a3=0.0001. This set of coefficients describes the spatial shape of a lane centerline that curves slightly to the right ahead. Lane width refers to the lateral distance between the left and right boundary lines of a single lane. For example, measuring the lateral distance between the left and right boundary lines of the lane currently being traveled by vehicle A in a panoramic environment image yields a lane width of 3.5m.

[0039] Lane type refers to the functional classification label assigned to lane markings; for example, the lane marking to the left of vehicle A's lane is identified as a "double solid yellow line," indicating that crossing is prohibited and that the lane is separated from oncoming traffic, while the lane marking to the right is identified as a "white dashed line," indicating that lane changing is permitted. Driving direction restriction refers to the constraints on the permitted driving direction of vehicles as specified by the lane markings; for example, the driving direction restriction for the lane corresponding to the straight-ahead directional arrow is "straight-ahead only," while the driving direction restriction for the adjacent lane to the right corresponding to the right-turn directional arrow is "right-turn only."

[0040] Among the above-mentioned optional methods, the cubic polynomial coefficients of the lane centerline, lane width and lane line type, and driving direction restrictions are further incorporated into the geometric parameters and semantic attributes of static road elements. This allows the semantic topology map to provide a refined description of the road's geometric boundaries and traffic rules, thereby enhancing the depth of understanding of road structural features and traffic constraints in path planning.

[0041] In one optional approach, the step of matching feature points within the overlapping regions corresponding to a preset overlap rate between the fields of view of adjacent visual sensors in the multi-view environmental image includes: The feature point descriptors with scale invariance and rotation invariance are extracted from the overlapping region. The relative pose between adjacent visual sensors provided by the inertial measurement unit is used as the initial transformation matrix. The feature point descriptors from different viewpoints are matched for nearest neighbor, and the random sampling consensus algorithm is applied to remove erroneous matching point pairs.

[0042] Scale invariance refers to the property that feature point descriptors remain numerically unchanged or approximately unchanged when an image is scaled. For example, if a camera takes pictures at distances of 10m and 15m from a straight directional arrow marking, the pixel area occupied by the marking in the two images may differ, but the magnitude difference of the extracted feature point descriptor vectors is less than 3%, demonstrating scale invariance. Rotation invariance refers to the property that feature point descriptors remain numerically unchanged or approximately unchanged when an image is rotated. For example, if a front-view camera and a right-side camera capture the same road surface texture at angles approximately 75 degrees apart, the cosine of the angle between the corresponding feature point descriptor vectors extracted from the two images is greater than 0.92, demonstrating rotation invariance.

[0043] The feature point descriptor refers to a mathematical expression that represents the local neighborhood gray-level distribution or gradient direction distribution of a feature point using a fixed-dimensional numerical vector. For example, by using an accelerated segmentation test feature algorithm, a 128-dimensional floating-point feature vector is extracted from the neighborhood of the corner point of the straight guide arrow marking in the overlapping region. The feature vector value is the feature point descriptor of the corner point. The initial transformation matrix refers to a rigid body transformation matrix used to roughly align two viewpoint images, calculated using the relative pose between adjacent visual sensors provided by the inertial measurement unit. For example, if the inertial measurement unit outputs a relative rotation angle of 82 degrees around the Z-axis and a relative translation of 0.48m along the X-axis between the front-view camera and the right-side camera, a 3×4 initial transformation matrix is ​​calculated based on this set of pose parameters to initially map the right-side camera image onto the front-view camera coordinate system.

[0044] In the above-mentioned optional methods, feature point descriptors with scale invariance and rotation invariance are further extracted from the overlapping areas. The relative pose provided by the inertial measurement unit is used as the initial transformation matrix for nearest neighbor matching. The random sampling consensus algorithm is applied to remove erroneous matching point pairs, thereby improving the accuracy of feature correspondence and stitching stability in the multi-view image stitching process.

[0045] In one alternative approach, the step of constructing a four-dimensional spatiotemporal environment model containing the motion state sequence of the dynamic target over consecutive time steps and the change history of the static road elements, based on the semantic topology graph generated at multiple consecutive time steps, includes: By setting a fixed time step, the semantic topology map generated at each time step is mapped to a three-dimensional spatial raster layer to obtain a three-dimensional occupancy raster map.

[0046] The fixed time step refers to the constant time interval between adjacent 3D occupancy raster maps when constructing a 4D spatiotemporal environment model. For example, if the fixed time step is set to 0.5s, the 3D occupancy raster map sequence is generated sequentially at time intervals of 0s, 0.5s, 1.0s, and 1.5s and arranged along the time axis. The 3D occupancy raster map is a 3D grid obtained by projecting static road elements and dynamic targets from a single moment's semantic topology map onto a 3D spatial raster layer, where raster cells record occupancy status and semantic categories. For example, projecting the semantic topology map at 0.5s onto a 3D grid covering 50m horizontally, 80m vertically, and 5m high with a raster resolution of 0.2m, the raster record corresponding to the straight directional arrow is "passable road surface," and the raster record for a bus is "occupied by a large passenger vehicle," generating the 3D occupancy raster map at 0.5s.

[0047] The three-dimensional occupancy raster is stacked along the time axis to obtain a four-dimensional raster.

[0048] Among them, a four-dimensional grid refers to a four-dimensional array structure with three spatial dimensions and one time dimension formed by stacking three-dimensional occupancy grid images of multiple consecutive time moments along the time axis; for example, three-dimensional occupancy grid images of three time moments of 0s, 0.5s, and 1.0s are arranged and stacked sequentially along the time axis to form a four-dimensional grid with a size of 250×400×25×3, where the first three dimensions correspond to spatial coordinates and the fourth dimension corresponds to the time index.

[0049] By associating the same dynamic target in consecutive time layers of the four-dimensional grid, a motion state sequence of the dynamic target in consecutive time steps is obtained.

[0050] The semantic category of each spatial grid in the four-dimensional grid is encoded along the time axis to obtain the change history of the static road element.

[0051] In the above-mentioned optional methods, the semantic topology map is further mapped into a three-dimensional occupancy grid map by a fixed time step and stacked along the time axis to form a four-dimensional grid. The dynamic targets in the continuous time layer are associated to obtain the motion state sequence, and the time-varying state of the semantic category is encoded to obtain the change history of static road elements, so that the environmental temporal evolution information can be stored in a structured manner within a unified spatiotemporal framework.

[0052] In one alternative approach, the step of identifying a second conflict segment in which the initial planned path and the motion state sequence of the dynamic target overlap in time and space includes: Based on the four-dimensional spatiotemporal environment model, the initial planned path is expanded into a spatiotemporal trajectory tube of the vehicle according to a preset speed profile, and the spatiotemporal trajectory tube assigns a time interval to each mileage point.

[0053] The spatiotemporal trajectory tube refers to a spatial-temporal tubular region formed by expanding the initial planned path along the mileage direction according to a preset speed profile and adding the expected arrival time interval of the vehicle at each mileage point. For example, if vehicle A sets the preset speed profile to a constant speed of 10m / s and considers a time margin of ±1s for acceleration and deceleration fluctuations, the expected arrival interval of the initial planned path at mileage 180m is 17.5s to 19.5s. The spatial coordinates and time intervals of each mileage point along the entire path are continuously arranged to form a cylindrical spatiotemporal trajectory tube with a diameter equal to the width of the vehicle body.

[0054] Based on the motion state sequence of the dynamic target in continuous time steps, the motion state of the dynamic target is extrapolated to obtain the probability distribution of the spatiotemporal region occupied by the dynamic target in the future time period.

[0055] Among them, the spatiotemporal probability distribution refers to the joint distribution of the probability density of a dynamic target appearing at various locations in space during a future time period as spatial coordinates and time change. For example, based on the extrapolation of the bus's motion state sequence from 0.5s to 1.5s, it is predicted that the bus will appear at 2.0s with a probability of 0.85 around the center position 144.5m to the east and 40.3m to the north, and the probability density follows a two-dimensional Gaussian distribution with the center as the mean and a standard deviation of 0.5m to the east and 0.3m to the north. The probability density values ​​of all spatiotemporal points constitute the spatiotemporal probability distribution of the bus.

[0056] By performing an overlap integral on the probability distribution of the spatiotemporal region occupied by the spatiotemporal trajectory tube and the dynamic target, a collision risk index describing the collision risk distribution along the mileage is calculated. .

[0057] The collision risk index A continuous mileage segment exceeding a preset threshold is identified as the second conflict segment.

[0058] The expression for the collision risk index is: , In the formula, The mileage coordinates along the initially planned path, For mileage The area where the road is passable. For mileage The vehicle appears in a spatial location as a condition. and time The probability density function, For the dynamic target at time Located in spatial position The probability density function, For probability truncation threshold, This is an indicator function that takes the value 1 when the condition inside the parentheses is true, and 0 otherwise.

[0059] It should be noted that the collision risk index expression performs a probability overlap analysis between the spatiotemporal trajectory tube of the vehicle along the initial planned path and the spatiotemporal probability distribution of the dynamic target in the future time period. The path position is parameterized by mileage coordinates. Within the road passable area corresponding to each mileage point, the product of the conditional probability density of the vehicle appearing at a specific spatial location and time and the probability density of the dynamic target appearing at the same spatiotemporal point is used as the basic collision risk measure. A probability cutoff threshold is introduced to filter the area occupied by the dynamic target with low probability. The binary screening effect of the indicator function is used to exclude the spatiotemporal points where the probability of the dynamic target appearing is lower than the threshold. The product of the probability densities in the effective overlap area is subjected to spatial double integration and temporal single integration to obtain the collision risk quantification value continuously distributed along the path mileage. The collision risk index expression transforms the probability of a vehicle colliding with a dynamic target at each mileage point along the path into a continuous risk value distribution curve through probabilistic spatiotemporal overlap integral calculation. This provides an objective and quantitative risk assessment basis for determining dynamic conflict segments, elevating the conflict identification process from deterministic geometric intersection judgment to probabilistic risk assessment that considers prediction uncertainty. This avoids misjudgment or omission of the degree of conflict due to dynamic target motion prediction errors or low probability of occupying areas.

[0060] Wherein, the probability density function of the dynamic target The distribution pattern is adaptively determined based on the category information of the dynamic target and the rate of change of the velocity vector. When the rate of change of the velocity direction of the dynamic target is lower than the preset direction change threshold, the probability density function adopts a two-dimensional Gaussian distribution with the extrapolated position of the motion state sequence as the mean and the standard deviation of the velocity magnitude as the broadening parameter. When the rate of change of the velocity direction is higher than or equal to the preset direction change threshold, the probability density function adopts a uniform distribution centered on the extrapolated position of the motion state sequence. The radius of the uniform distribution is determined jointly based on the velocity magnitude and the rate of change of direction.

[0061] In the above-mentioned optional methods, the initial planned path is further expanded into a spatiotemporal trajectory tube according to a preset speed profile. The future spatiotemporal occupancy probability distribution is extrapolated based on the dynamic target motion state sequence. The collision risk index is calculated by integrating the two overlaps. The continuous mileage segment exceeding the preset threshold is determined as the second conflict segment, so that the dynamic obstacle conflict identification is based on probabilistic risk assessment.

[0062] In one alternative approach, the step of generating an obstacle avoidance path curve in the four-dimensional spatiotemporal environment model, using the geometric parameters of the static road elements involved in the first conflict segment as boundary constraints and the motion state sequence of the dynamic target involved in the second conflict segment as motion obstacle constraints, includes: Construct an objective function with the path curvature change rate, the lateral deviation relative to the initial planned path, and the collision risk index as penalty terms. Solving under the boundary constraints and the motion obstacle constraints makes The obstacle avoidance path curve is obtained by minimizing the path parameters.

[0063] The expression for the objective function is: , In the formula, The starting mileage of the obstacle avoidance path. The endpoint mileage of the obstacle avoidance path. The collision risk index as described in claim 6. For path curvature, This represents the lateral offset of the path point relative to the initially planned path. These are preset weighting coefficients.

[0064] It should be noted that the objective function expression models the obstacle avoidance path curve optimization problem as a cost function minimization problem in the form of path arc length integral. The cost function consists of a weighted superposition of three penalty terms. The first term couples the collision risk index and lateral offset through an exponential function. As the lateral offset increases, the risk penalty increases exponentially, forcing the obstacle avoidance path to avoid high-risk areas as much as possible while deviating from the initial planned path. The second term uses the square of the rate of change of path curvature with respect to arc length as a measure of path smoothness, limiting drastic changes in path curvature to ensure driving comfort. The third term uses the square of the lateral offset as a measure of the degree of path deviation, preventing the obstacle avoidance path from deviating too far from the initial planned path and increasing unnecessary driving distance. The three terms are multiplied by preset weight coefficients and integrated within the arc length interval from the starting mileage to the ending mileage of the obstacle avoidance path. Within the feasible path domain jointly defined by the boundary constraints delineated by static road geometry parameters and the motion obstacle constraints imposed by the dynamic target motion state sequence, the objective function expression searches for the path parameter combination that minimizes the objective function through numerical optimization methods. This transforms the generation of the obstacle avoidance path curve into a weighted balance optimization process among three mutually restrictive factors: safety, driving comfort, and path deviation. This ensures that the generated obstacle avoidance path curve has a smooth curvature change and a reasonable lateral deviation range while avoiding multiple constraint obstacles.

[0065] After obtaining the obstacle avoidance path curve, the obstacle avoidance path curve is compared and verified with the semantic labels of the static road element nodes in the semantic topology graph. If any path point on the obstacle avoidance path curve falls into a semantic area where the driving direction is restricted to no passage or crosses the boundary of a static road element whose lane line type is prohibited from crossing, a penalty term for violating traffic rules is added to the objective function, and the path parameters that minimize the corrected objective function are solved again until the obtained obstacle avoidance path curve simultaneously satisfies boundary constraints, motion obstacle constraints, curvature continuity constraints, and traffic rule constraints.

[0066] In the above-mentioned optional methods, an objective function is further constructed using the path curvature change rate, the lateral deviation relative to the initial planned path, and the collision risk index as penalty terms. The optimal path parameters are solved under static geometric boundary constraints and dynamic motion obstacle constraints, so that the generated obstacle avoidance path curve achieves comprehensive optimization and balance between safety, driving comfort, and path deviation.

[0067] In one alternative approach, the step of smoothly joining the safe and feasible path with the reserved segments in the initial planned path that do not conflict includes: A transition spline curve is constructed between the end point of the safe and feasible path and the start point of the reserved segment, satisfying the requirements of positional continuity, tangent vector direction continuity, and curvature continuity.

[0068] In the above-mentioned optional methods, a transition spline curve that satisfies positional continuity, tangent vector direction continuity, and curvature continuity is further constructed between the end point of the safe and feasible path and the starting point of the retained segment. This enables the spliced ​​adaptive navigation path to achieve a high-order smooth connection in geometry, avoiding steering impact or driving posture fluctuation caused by sudden changes in path curvature.

[0069] In one alternative approach, the vehicle's kinematic constraints include the vehicle's minimum turning radius, maximum rate of change of steering wheel angle, and tire slip angle limit; the road geometric constraints include lane boundaries, curb locations, and traversable area boundaries; and the spatiotemporal alignment fusion is jointly calibrated using a high-precision GPS timestamp and an inertial measurement unit angular velocity integral.

[0070] Among the above-mentioned optional methods, the minimum turning radius of the vehicle, the maximum rate of change of the steering wheel angle, and the tire slip angle limit are further defined as kinematic constraints. The lane boundary, curb position, and crossable area boundary are defined as road geometric constraints. Spatiotemporal alignment and fusion are achieved by using high-precision global positioning system timestamps and inertial measurement unit angular velocity integrals to ensure that the path planning meets physical performance and spatial constraints.

[0071] This invention provides a vehicle adaptive navigation path planning system based on multi-view vision, which includes: The acquisition module is used to acquire multi-view environmental images synchronously collected by multiple vision sensors deployed on the vehicle at the same time. The multiple vision sensors are installed at different positions and orientations on the vehicle, and the fields of view of adjacent vision sensors have a preset overlap rate, which together constitute a panoramic field of view covering the circumference of the vehicle. The fusion module is used to match feature points in the overlapping areas corresponding to the preset overlap rate between the fields of view of the adjacent visual sensors in the multi-view environmental image, stitch the multi-view environmental image into a panoramic environmental image based on the matching result, extract the geometric parameters and semantic attributes of static road elements from the panoramic environmental image, detect and identify the position, velocity vector and category information of dynamic targets, and fuse to generate a semantic topology map that represents the complete state of the environment around the vehicle at the current moment. The construction module is used to construct a four-dimensional spatiotemporal environment model containing the motion state sequence of the dynamic target in consecutive time steps and the change history of the static road elements based on the semantic topology graph generated at multiple consecutive time steps. The identification module is used to obtain the initial planned path, map the initial planned path to the four-dimensional spatiotemporal environment model, perform spatiotemporal scanning along the mileage direction of the initial planned path, and identify the first conflict segment where the geometric parameters of the initial planned path are incompatible with those of the static road elements, and the second conflict segment where the initial planned path and the motion state sequence of the dynamic target generate spatiotemporal overlap. The generation module is used to generate an obstacle avoidance path curve from the current position of the vehicle to the return point after the conflict segment in the four-dimensional spatiotemporal environment model, using the geometric parameters of the static road elements involved in the first conflict segment as boundary constraints and the motion state sequence of the dynamic target involved in the second conflict segment as motion obstacle constraints. The planning module is used to check the curvature continuity of the obstacle avoidance path curve, eliminate candidate paths whose curvature does not meet the kinematic constraints of the vehicle, determine a safe and feasible path, and smoothly connect the safe and feasible path with the reserved segments in the initial planned path that do not conflict, to form an adaptive navigation path for controlling the vehicle's driving.

[0072] The technical solution of this embodiment synchronously acquires multi-view environmental images by deploying multiple visual sensors at different installation locations and orientations. It completes panoramic stitching by matching feature points in adjacent overlapping fields of view. Combined with static road element extraction and dynamic target detection and recognition, it generates a semantic topology map and constructs a four-dimensional spatiotemporal environment model. The initial planned path is mapped to the model for spatiotemporal scanning to identify static geometrically incompatible conflict segments and dynamic target spatiotemporally overlapping conflict segments. Then, using static geometric parameters as boundary constraints and dynamic motion state sequences as motion obstacle constraints, it generates obstacle avoidance path curves. After curvature continuity checks, safe and feasible paths are selected and smoothly stitched with the retained segments to form an adaptive navigation path. This solves the problems of limited field of view, insufficient long-distance recognition capability, and difficulty of static path planning algorithms in coping with real-time dynamic changes in road conditions in monocular or binocular vision systems. It improves the vehicle's circumferential panoramic environment perception coverage capability and the real-time adaptability of path planning in complex scenarios.

[0073] Furthermore, the system provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to the actual situation to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0074] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A vehicle adaptive navigation path planning method based on multi-view vision, characterized in that, The method includes: Multiple visual sensors deployed on a vehicle acquire multi-view environmental images simultaneously at the same time. The multiple visual sensors are installed at different positions and orientations on the vehicle, and the fields of view of adjacent visual sensors have a preset overlap rate, together forming a panoramic field of view covering the circumference of the vehicle. The feature points in the overlapping areas corresponding to the preset overlap rate between the fields of view of the adjacent visual sensors in the multi-view environmental image are matched. Based on the matching results, the multi-view environmental images are stitched into a panoramic environmental image. The geometric parameters and semantic attributes of static road elements are extracted from the panoramic environmental image, and the position, velocity vector and category information of dynamic targets are detected and identified. The semantic topology map representing the complete state of the environment around the vehicle at the current moment is fused and generated. Based on the semantic topology graph generated at multiple consecutive time points, a four-dimensional spatiotemporal environment model is constructed, which includes the motion state sequence of the dynamic target at consecutive time steps and the change history of the static road elements. Obtain the initial planned path, map the initial planned path to the four-dimensional spatiotemporal environment model, perform spatiotemporal scanning along the mileage direction of the initial planned path, and identify the first conflict segment where the geometric parameters of the initial planned path are incompatible with those of the static road elements, and the second conflict segment where the initial planned path and the motion state sequence of the dynamic target generate spatiotemporal overlap. Using the geometric parameters of the static road elements involved in the first conflict segment as boundary constraints and the motion state sequence of the dynamic target involved in the second conflict segment as motion obstacle constraints, an obstacle avoidance path curve is generated in the four-dimensional spatiotemporal environment model from the current position of the vehicle to the return point after the conflict segment. The curvature continuity of the obstacle avoidance path curve is checked, and candidate paths whose curvature does not meet the kinematic constraints of the vehicle are eliminated. A safe and feasible path is determined, and the safe and feasible path is smoothly spliced ​​with the reserved segments in the initial planned path that do not conflict, to form an adaptive navigation path for controlling the vehicle's movement.

2. The vehicle adaptive navigation path planning method based on multi-view vision according to claim 1, characterized in that, The steps of fusing and generating a semantic topology graph representing the complete state of the vehicle's surrounding environment at the current moment include: The geometric parameters and semantic attributes of the static road elements are used to construct a static layer containing node coordinates and semantic labels; The position, velocity vector, and category information of the dynamic target are used to construct a dynamic layer containing node motion attributes and category labels; The static layer and the dynamic layer are spatiotemporally aligned in a unified world coordinate system, and the aligned static layer nodes and dynamic layer nodes are connected by spatial association edges to form the semantic topology graph.

3. The vehicle adaptive navigation path planning method based on multi-view vision according to claim 2, characterized in that, The geometric parameters of the static road element include the cubic polynomial coefficients representing the lane centerline and the lane width, and the semantic attributes include lane line type and driving direction restrictions.

4. The vehicle adaptive navigation path planning method based on multi-view vision according to claim 1, characterized in that, The step of matching feature points within the overlapping regions corresponding to a preset overlap rate between the fields of view of adjacent visual sensors in the multi-view environmental image includes: The feature point descriptors with scale invariance and rotation invariance are extracted from the overlapping region. The relative pose between adjacent visual sensors provided by the inertial measurement unit is used as the initial transformation matrix. The feature point descriptors from different viewpoints are matched for nearest neighbor, and the random sampling consensus algorithm is applied to remove erroneous matching point pairs.

5. The vehicle adaptive navigation path planning method based on multi-view vision according to claim 1, characterized in that, The steps of constructing a four-dimensional spatiotemporal environment model, which includes the motion state sequence of the dynamic target at consecutive time steps and the change history of the static road elements, based on the semantic topology graph generated at multiple consecutive time steps, include: A fixed time step is set, and the semantic topology map generated at each time step is mapped to a three-dimensional spatial grid layer to obtain a three-dimensional occupancy grid map; The three-dimensional occupancy raster is stacked along the time axis to obtain a four-dimensional raster; By associating the same dynamic target in consecutive time layers of the four-dimensional grid, a motion state sequence of the dynamic target in consecutive time steps is obtained; The semantic category of each spatial grid in the four-dimensional grid is encoded along the time axis to obtain the change history of the static road element.

6. The vehicle adaptive navigation path planning method based on multi-view vision according to claim 5, characterized in that, The step of identifying the second conflict segment in which the initial planned path and the motion state sequence of the dynamic target overlap in time and space includes: Based on the four-dimensional spatiotemporal environment model, the initial planned path is expanded into a spatiotemporal trajectory tube of the vehicle according to a preset speed profile, and the spatiotemporal trajectory tube assigns a time interval to each mileage point. Based on the motion state sequence of the dynamic target in consecutive time steps, the motion state of the dynamic target is extrapolated to obtain the probability distribution of the spatiotemporal region occupied by the dynamic target in the future time period; By performing an overlap integral on the probability distribution of the spatiotemporal region occupied by the spatiotemporal trajectory tube and the dynamic target, a collision risk index describing the collision risk distribution along the mileage is calculated. ; The collision risk index A continuous mileage segment exceeding a preset threshold is identified as the second conflict segment; The expression for the collision risk index is: ， In the formula, The mileage coordinates along the initially planned path, For mileage The area where the road is passable. For mileage The vehicle appears in a spatial location as a condition. and time The probability density function, For the dynamic target at time Located in spatial position The probability density function, For probability truncation threshold, This is an indicator function that takes the value 1 when the condition inside the parentheses is true, and 0 otherwise.

7. The vehicle adaptive navigation path planning method based on multi-view vision according to claim 6, characterized in that, The step of generating an obstacle avoidance path curve from the vehicle's current position to the return point after the conflict segment in the four-dimensional spatiotemporal environment model, using the geometric parameters of the static road elements involved in the first conflict segment as boundary constraints and the motion state sequence of the dynamic targets involved in the second conflict segment as motion obstacle constraints, includes: Construct an objective function with the path curvature change rate, the lateral deviation relative to the initial planned path, and the collision risk index as penalty terms. Solving under the boundary constraints and the motion obstacle constraints makes The obstacle avoidance path curve is obtained by minimizing the path parameters; The expression for the objective function is: ， In the formula, The starting mileage of the obstacle avoidance path. The endpoint mileage of the obstacle avoidance path. The collision risk index as described in claim 6. For path curvature, This represents the lateral offset of the path point relative to the initially planned path. These are preset weighting coefficients.

8. The vehicle adaptive navigation path planning method based on multi-view vision according to claim 7, characterized in that, The step of smoothly joining the safe and feasible path with the reserved segments in the initial planned path that do not conflict includes: A transition spline curve is constructed between the end point of the safe and feasible path and the start point of the reserved segment, satisfying the requirements of positional continuity, tangent vector direction continuity, and curvature continuity.

9. The vehicle adaptive navigation path planning method based on multi-view vision according to claim 1, characterized in that, The vehicle's kinematic constraints include the vehicle's minimum turning radius, maximum rate of change of steering wheel angle, and tire slip angle limit. The road geometric constraints include lane boundaries, curb positions, and traversable area boundaries. The spatiotemporal alignment fusion is jointly calibrated using a high-precision global positioning system timestamp and an inertial measurement unit angular velocity integral.

10. A vehicle adaptive navigation path planning system based on multi-view vision, characterized in that, The system includes: The acquisition module is used to acquire multi-view environmental images synchronously collected by multiple vision sensors deployed on the vehicle at the same time. The multiple vision sensors are installed at different positions and orientations on the vehicle, and the fields of view of adjacent vision sensors have a preset overlap rate, which together constitute a panoramic field of view covering the circumference of the vehicle. The fusion module is used to match feature points in the overlapping areas corresponding to the preset overlap rate between the fields of view of the adjacent visual sensors in the multi-view environmental image, stitch the multi-view environmental image into a panoramic environmental image based on the matching result, extract the geometric parameters and semantic attributes of static road elements from the panoramic environmental image, detect and identify the position, velocity vector and category information of dynamic targets, and fuse to generate a semantic topology map that represents the complete state of the environment around the vehicle at the current moment. The construction module is used to construct a four-dimensional spatiotemporal environment model containing the motion state sequence of the dynamic target in consecutive time steps and the change history of the static road elements based on the semantic topology graph generated at multiple consecutive time steps. The identification module is used to obtain the initial planned path, map the initial planned path to the four-dimensional spatiotemporal environment model, perform spatiotemporal scanning along the mileage direction of the initial planned path, and identify the first conflict segment where the geometric parameters of the initial planned path are incompatible with those of the static road elements, and the second conflict segment where the initial planned path and the motion state sequence of the dynamic target generate spatiotemporal overlap. The generation module is used to generate an obstacle avoidance path curve from the current position of the vehicle to the return point after the conflict segment in the four-dimensional spatiotemporal environment model, using the geometric parameters of the static road elements involved in the first conflict segment as boundary constraints and the motion state sequence of the dynamic target involved in the second conflict segment as motion obstacle constraints. The planning module is used to check the curvature continuity of the obstacle avoidance path curve, eliminate candidate paths whose curvature does not meet the kinematic constraints of the vehicle, determine a safe and feasible path, and smoothly connect the safe and feasible path with the reserved segments in the initial planned path that do not conflict, to form an adaptive navigation path for controlling the vehicle's driving.

Images