Limit perception system complementary to fixed perception system and control method thereof

Abstract

The application relates to the technical field of vehicle environment sensing, and discloses a limit sensing system complementary to a fixed sensing system and a control method thereof. The limit sensing system comprises: a plurality of sensing modules; each sensing module comprises a serpentine multi-joint bending mechanism and a sensing component; the sensing component comprises a camera and a laser radar; a plurality of driving mechanisms; a limit controller used for acquiring vehicle state information, navigation information and external environment information, determining a driving scene, generating an initial activation scheme, and controlling corresponding sensing modules to stretch out to a target posture through the driving mechanisms; the limit controller is also used for fusing data information and displaying the data information on a central control screen. The limit sensing system and the control method thereof introduce the serpentine multi-joint bending mechanism to realize omnidirectional limit accurate sensing on the basis of the fixed sensing system, have high sensing flexibility and freedom, and solve the technical problems of a visual field blind area and insufficient flexibility in the prior art.

Description

Technical Field

[0001] This invention relates to the field of vehicle environmental perception technology, specifically to an extreme perception system and its control method that complements a fixed perception system. Background Technology

[0002] With the acceleration of urbanization, the driving environment is becoming increasingly complex. Drivers frequently face various extreme operating scenarios: height restrictions at underground parking garage entrances, passing other vehicles in narrow alleys, and bottoming out when crossing bumps. When a vehicle is close to an obstacle, the panoramic images of existing fixed sensing systems often only capture a portion of the obstacle, failing to provide a precise relative position between the wheels and the obstacle. More importantly, the ultrasonic sensors of existing fixed sensing systems have significant detection blind spots; when an obstacle is less than 20 centimeters from the sensor, the radar often fails to detect it or experiences severe data fluctuations—this distance is precisely the range where scrapes are most likely to occur. When passing other vehicles in narrow alleys in older residential areas, in extreme situations where the actual distance between the vehicle's sides and obstacles is less than 20 centimeters, drivers rely heavily on experience to judge, making rearview mirrors and vehicle sides prone to scraping. Simultaneously, at underground parking garage entrances, underpasses, and tree-lined roads, drivers struggle to accurately judge the gap between the vehicle's roof and overhead obstacles, easily leading to roof scrapes.

[0003] Among the related technologies, the latest technologies include some vehicles that, in addition to traditional fixed sensing systems, have added upward and downward auxiliary sensing. Specifically, some vehicles have cameras installed under the vehicle to monitor the bottom of the vehicle in real time, reducing blind spots and improving driving safety; in other vehicles, retractable camera devices are set on the top of the vehicle, which can greatly increase the monitoring range on the top of the vehicle and increase safety during use.

[0004] However, even with the addition of upper and lower auxiliary sensors, fixed sensing systems still have the following drawbacks: ① In the front and rear directions, there are problems with poor perception accuracy or severe jumps at the extreme positions in the front and rear directions (the extreme case of less than 20cm); in the left and right directions, fixed perception systems usually only display images and rely on the driver to visually estimate the distance by looking at the fixed auxiliary lines on the screen, lacking real-time, dynamic, numerical and accurate distance feedback. ② In the vertical direction, these cameras installed under and on the roof of the vehicle contribute very little to improving passability, because the field of view of the under-vehicle camera is perpendicular to the ground and can only capture a part of the obstacle (such as ground bumps), and cannot provide a view of the limit gap or the limit gap size; similarly, the roof camera only captures a general gap view at an angle and does not provide the limit gap size. ③ Retractable cameras can only achieve linear movement or single-axis rotation, which is insufficient in degree of freedom. They cannot flexibly bypass obstacles, cannot penetrate into narrow spaces for detection, and cannot accurately provide a field of view in extreme gaps, thus acquiring limited information. Summary of the Invention

[0005] This application provides a limit sensing system and its control method that complements a fixed sensing system. Based on the fixed sensing system, a serpentine multi-joint bending mechanism is introduced to achieve precise sensing of the upper, lower, left, right, front, and rear limits, providing an all-round accurate limit gap field of view and limit gap distance. The sensing is flexible and has a high degree of freedom, solving the technical problems of blind spots and insufficient flexibility in the prior art.

[0006] In a first aspect, embodiments of this application provide an extreme perception system that complements a fixed perception system, comprising: a plurality of numbered perception modules, at least one at each of the front, rear, roof, underside, left side and right side of the vehicle; each perception module includes a serpentine multi-joint bending mechanism and a perception component, each perception component being fixed to the telescopic end of the serpentine multi-joint bending mechanism; the perception component includes a camera and a lidar. Several driving mechanisms, each driving mechanism is used to drive a sensing module; The limit controller is used to acquire vehicle status information, navigation information and external environment information from the fixed perception system, including but not limited to the fixed perception system, determine the driving scenario, generate an initial activation scheme, and control the corresponding numbered perception module to extend to the target posture through the drive mechanism based on the initial activation scheme; the limit controller is also used to fuse the data information acquired by the camera and lidar into the data information of the fixed perception system and display it on the central control screen.

[0007] In conjunction with the first aspect, in one implementation, the limit controller includes: The vehicle status monitoring module is used to acquire vehicle status information, navigation information, and external environment information in real time from the fixed sensing system, including but not limited to the fixed sensing system. The scene recognition module is used to determine driving scenarios that require image and size assistance in terms of up, down, left, right, front, and rear, based on information obtained from the vehicle status monitoring module. The decision scheduling module is used to generate an initial activation scheme for the perception module based on the driving scenario; the drive mechanism controls the corresponding perception module to extend to the target posture based on the initial activation scheme; wherein, the initial activation scheme includes the module number, extension displacement, joint bending angle and target position of the perception component. The multi-sensor fusion module is used to fuse data information acquired by cameras and lidar into data information from a fixed sensing system and display it on the central control screen.

[0008] In conjunction with the first aspect, in one embodiment, the limit controller is further configured to dynamically change in accordance with the actual gap after the sensing module extends and bends to the initial target posture.

[0009] In conjunction with the first aspect, in one embodiment, the central control screen includes a manual operation interface that is connected to the limit controller via signal communication; the manual operation interface includes an expand button, a retract button, and an operation button for three-dimensional directional adjustment of the target position of the sensing component.

[0010] In conjunction with the first aspect, in one implementation, the driving scenarios include automatic parking assistance scenarios, height-restricted driving scenarios, narrow road meeting scenarios, off-road obstacle scenarios, narrow road U-turn scenarios, and sentry monitoring scenarios.

[0011] In conjunction with the first aspect, in one embodiment, each of the sensing modules further includes a telescopic arm driven by the drive mechanism, one end of the telescopic arm being fixed to the vehicle body structure and the other end being fixed to the fixed end of each serpentine multi-joint bending mechanism. The sensing component also includes an illumination unit for providing light to the camera at night.

[0012] Secondly, embodiments of this application provide a sensing method based on the above-described extreme sensing system, comprising the following steps: In the initial state, each sensing module is in a stowed-away state; The limit controller acquires vehicle status information, navigation information, and external environment information in real time, determines the driving scenario, and generates an initial activation scheme. Based on the initial activation scheme, the limit controller controls the snake-shaped multi-joint bending mechanism of the corresponding numbered sensing module to extend to the target posture through the drive mechanism; The camera and lidar of the sensing component collect images and distance data, which are then fused with data from the fixed sensing system and displayed on the central control screen.

[0013] In conjunction with the second aspect, in one embodiment, the limit controller includes a vehicle state monitoring module, a scene recognition module, a decision scheduling module, and a multi-sensor fusion module, and the perception method includes the following steps: The vehicle status monitoring module acquires vehicle status information, navigation information, and external environment information in real time from the fixed sensing system, including but not limited to the fixed sensing system. The scene recognition module determines the driving scenes that require image and size assistance in terms of up, down, left, right, front, and rear based on the information obtained by the vehicle status monitoring module; the decision scheduling module generates an initial activation scheme for the perception module based on the driving scenes. The drive mechanism controls the serpentine multi-joint bending mechanism of the corresponding sensing module to extend to the target posture based on the initial activation scheme; wherein, the initial activation scheme includes the module number, the amount of extension and retraction displacement, the joint bending angle and the target position of the sensing component; The multi-sensor fusion module integrates data from cameras and lidar with data from a fixed sensing system, which is then displayed on the central control screen.

[0014] In conjunction with the second aspect, in one embodiment, the limit controller further includes a scene following module, and the display on the central control screen further includes: The scene following module uses a drive mechanism to make the sensing components dynamically follow the actual gap changes; The driving scenario includes an automatic parking cooperative scenario, which is further divided into a side extreme parking sub-scenario, a perpendicular extreme parking sub-scenario, and a narrow extreme parking space parking sub-scenario. If the scene recognition module identifies the vehicle as parallel parking, the decision scheduling module activates the side perception module and the rear perception module closest to the parking space through the drive mechanism. If the scene recognition module identifies the vehicle as perpendicular parking, the decision scheduling module activates the front perception module through the drive mechanism. If the scene recognition module identifies a narrow parking space, the decision scheduling module activates all side modules through the drive mechanism.

[0015] In conjunction with the second aspect, in one implementation, the driving scenarios include height-restricted driving scenarios, narrow road meeting scenarios, and off-road obstacle scenarios; If the scene recognition module determines that it is about to enter a height-restricted driving scene based on the navigation information obtained by the vehicle status monitoring module, the initial activation scheme generated by the decision scheduling module is to activate the roof perception module when the vehicle speed drops below the first set speed, so as to collect images and spacing information of the space above the roof. If the scene recognition module determines that a narrow road encountering an oncoming vehicle is a narrow road meeting scenario based on the external environment information obtained by the vehicle status monitoring module, the initial activation scheme generated by the decision scheduling module is to reduce the vehicle speed to the second set speed and activate the perception modules on the left and right sides to obtain the extreme distance between the two sides. If the scene recognition module determines that the road is full of rocks and ditches based on the external environment information obtained by the vehicle status monitoring module, it is considered an off-road obstacle scene. The initial activation scheme generated by the decision scheduling module is to activate the perception modules of the front, bottom, left side and right side of the vehicle to obtain obstacle images and spacing below the front of the vehicle, below the bottom of the vehicle, and to the left and right of the tires on both sides.

[0016] The beneficial effects of the technical solutions provided in this application include: 1. The limit perception system of this application, based on the fixed perception system, introduces a serpentine multi-joint bending mechanism that can bend with multiple degrees of freedom. This mechanism can deliver the perception components (including cameras and lidar) to positions that traditional cameras cannot reach, completely eliminating all blind spots in the three-dimensional space around the vehicle. Simultaneously, several numbered perception modules are installed at least at the front of the vehicle, the roof, the left side panel, the right side panel, and the bottom of the vehicle. The limit controller can automatically determine various driving scenarios that require image and size assistance in the up, down, left, right, front, and rear directions. Based on preset activation rules, it generates an initial activation scheme, and the drive mechanism extends the corresponding perception modules involved to the target position. Compared with the traditional fixed perception system, it solves the problem of poor perception accuracy in the front and rear extreme positions and adds precise perception in the up, down, left, and right directions. It can obtain the required limit gap field of view images and limit gap distances in the up, down, left, right, front, and rear directions of the corresponding driving scenario, thereby achieving omnidirectional, blind-spot-free precise perception. At the same time, it is integrated and displayed on the central control screen, which can assist the driver in precise driving in various limit gap scenarios, resulting in a good driving experience, strong practicality, and high economic value.

[0017] The perception method of this application typically involves each perception module being in a retracted state. When a certain driving scenario is reached, a corresponding initial activation scheme is generated. Based on the initial activation scheme, the limit controller controls the serpentine multi-joint bending mechanism of the corresponding numbered perception module to extend to the target posture through the drive mechanism. More accurate limit data is collected and fused with the data information of the fixed perception system, and displayed on the central control screen to improve perception accuracy and assist in completing more refined vehicle control, which has high economic value.

[0018] The perception method of this application can provide distance information with millimeter-level accuracy for a variety of different parking scenarios, significantly improving the parking success rate and enhancing the driving experience. The perception method of this application can provide powerful and accurate perception assistance for height-restricted driving scenarios, narrow road meeting scenarios, off-road obstacle scenarios, and narrow road U-turn scenarios, improving the vehicle's ability to cope with various extreme scenarios, greatly enhancing the driving experience, and having high economic value. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A block diagram of the limit sensing system provided in the embodiments of this application; Figure 2A flowchart of the sensing method provided in the embodiments of this application. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0022] First, a brief introduction to traditional fixed sensing systems: In the field of automotive engineering, currently mass-produced vehicles are equipped with fixed perception systems as standard. At the hardware sensor level, fixed perception systems include onboard cameras (visual sensors) and ultrasonic radar (ultrasonic sensors).

[0023] Among them, the vehicle-mounted cameras include front-view cameras, rear-view cameras and surround-view cameras. The front-view cameras are mainly used for lane keeping, the rear-view cameras are mainly used for reversing images, and the surround-view cameras usually have 4 surround-view cameras distributed in front, rear, left and right for 360-degree panoramic imaging.

[0024] Ultrasonic radar is mainly used for parking, and is usually distributed on the front and rear bumpers (4 to 12 units) for obstacle distance measurement. However, ultrasonic radar has obvious limitations. Its detection range is short, usually less than 2.5 meters, and it has a blind zone, which is usually between 0 cm and 20 cm. In other words, when the obstacle is less than 20 cm away from the probe, the radar often cannot detect it or the data jumps sharply. This is the distance range in which the most scratches are most likely to occur.

[0025] The fixed perception system can also provide a 360-degree panoramic view by stitching together images from four surround-view cameras to provide a top-down view. The fixed perception system also supports automatic parking assistance, which combines ultrasonic radar and cameras to control the vehicle to enter the parking space. In practice, the screen will display specific distance values ​​(such as 0.3m, 0.5m) or colored warning bars. However, it is not applicable in some extreme front and rear scenarios (within 20cm), and there is no distance perception in the left and right directions, only image display. The driver has to visually estimate the distance by looking at the fixed auxiliary lines on the screen. There is also no perception of up and down.

[0026] Based on the shortcomings of traditional fixed sensing systems, this application provides a complementary limit sensing system. Building upon the fixed sensing system, it introduces a serpentine multi-joint bending mechanism for precise forward and backward limit sensing, solving the problems of poor or severely abrupt sensing accuracy at extreme positions (where the gap between the sensor and the obstacle is less than 20cm). It also enables precise upward and downward limit sensing, providing higher accuracy compared to existing technologies that only offer incompletely accurate gap views. Furthermore, it enables precise left and right limit sensing, offering higher accuracy than fixed sensing systems that rely solely on image display and driver estimation of distance using fixed auxiliary lines on the screen. Simultaneously, the serpentine multi-joint bending mechanism's flexible swing allows for deep exploration into narrow spaces, providing high flexibility and freedom of perception. This application provides comprehensive and accurate limit gap views and distances, solving the technical problems of blind spots and insufficient flexibility in existing technologies.

[0027] like Figure 1 As shown, this application discloses an extreme sensing system that complements a fixed sensing system. The extreme sensing system includes an extreme controller, several sensing modules, and several driving mechanisms.

[0028] Several sensing modules are numbered for easy differentiation and control later. These sensing modules are installed at the front, rear, roof, undercarriage, left side panel, and right side panel of the vehicle, with at least one sensing module at each of these locations. Each sensing module includes a serpentine multi-joint bending mechanism and sensing components. Each sensing component is fixed to the telescopic end of the serpentine multi-joint bending mechanism; the sensing components include a camera and a lidar. Specifically, the serpentine multi-joint bending mechanism can bend flexibly at multiple angles, providing flexible sensing and a high degree of freedom, enabling the camera and lidar to be positioned in a location with a good field of view.

[0029] Specifically, the serpentine multi-joint bending mechanism has a fixed end opposite to the telescopic end, and the serpentine multi-joint bending mechanism is directly or indirectly fixed to the vehicle body structure.

[0030] Each drive mechanism is used to drive one sensing module, and several drive mechanisms are used to independently drive each module to extend from the storage state to the working position, or to retract from the working position.

[0031] The limit controller is used to acquire vehicle status information, navigation information and external environment information, determine the driving scenario, generate an initial activation scheme, and control the corresponding numbered perception module to extend to the target posture through the drive mechanism based on the initial activation scheme; the limit controller is also used to fuse the data information acquired by the camera and lidar into the data information of the fixed perception system and display it on the central control screen.

[0032] The driving scenarios include various driving situations requiring image and size assistance in terms of up, down, left, right, front, and rear directions. The limit controller acquires vehicle status information, navigation information, and external environment information through methods including, but not limited to, fixed perception systems.

[0033] The initial activation scheme includes module number, telescopic displacement, joint bending angle and target position of sensing component. The initial activation scheme also includes which numbered modules to telescopicate and how to telescopicate them, including telescopic length and telescopic angle.

[0034] Specifically, the limit controller communicates with the fixed sensing system, and can acquire all information sensed by the fixed sensing system. The lidar of the sensing component is used to accurately measure the distance to obstacles, providing the limit gap dimensions, while the camera of the sensing component provides images of the limit gap.

[0035] Preferably, a number of sensing modules are respectively installed at the front of the vehicle, the front of the roof, the rear of the roof, the front of the left side panel, the rear of the left side panel, the front of the right side panel, the rear of the right side panel, the front of the vehicle bottom, and the rear of the vehicle bottom.

[0036] The limit perception system of this application, based on the fixed perception system, introduces a serpentine multi-joint bending mechanism that can bend with multiple degrees of freedom. This mechanism can deliver the perception components (including cameras and lidar) to positions that traditional cameras cannot reach, completely eliminating all blind spots in the three-dimensional space around the vehicle. Simultaneously, several numbered perception modules are installed at least at the front of the vehicle, the roof, the left side panel, the right side panel, and the underside. The limit controller can automatically determine various driving scenarios requiring image and size assistance in the up, down, left, right, front, and rear directions. Based on preset activation rules, it generates an initial activation scheme, and the drive mechanism extends the corresponding perception modules to the target positions. Compared to traditional fixed perception systems, this solves the problem of poor perception accuracy in the front and rear extreme positions and adds precise perception in the up, down, left, and right directions. It can obtain the required limit gap field of view images and limit gap distances for the corresponding driving scenarios in the up, down, left, right, front, and rear directions, thereby achieving omnidirectional, blind-spot-free precise perception. Simultaneously, it integrates the perception with the central control screen, assisting the driver in precise driving in various limit gap scenarios, providing a good driving experience, strong practicality, and high economic value.

[0037] In this application's extreme perception system, all perception modules are completely stored inside the vehicle body when not in use. The serpentine mechanism folds down to take up little space, making it easy to hide and providing both protection and dirt prevention. The serpentine multi-joint bending mechanism can achieve precise angle adjustment and automatically adjust the camera to face the target based on the location of obstacles detected by radar, enabling intelligent tracking and observation. In confined spaces, the serpentine mechanism can bend around obstructions to detect the situation in obstructed areas, such as detecting the curb obstructed by a neighboring vehicle in a narrow parking space.

[0038] Preferably, the two side-mounted sensing modules are located below the two A-pillars.

[0039] Furthermore, in one embodiment, the limit controller includes a vehicle status monitoring module, a scene recognition module, a decision scheduling module, and a multi-sensor fusion module.

[0040] The vehicle status monitoring module is used to acquire vehicle status information, navigation information, and external environment information in real time from the fixed sensing system, including but not limited to the fixed sensing system.

[0041] Specifically, vehicle status information mainly includes motion parameters (such as vehicle speed, acceleration, and yaw rate), control intention parameters (such as steering wheel angle, turn signal status, gear information, accelerator / brake pedal opening), and vehicle geometry parameters (such as vehicle length, width, height, wheelbase, track width, minimum ground clearance, and rearview mirror unfolding width). Navigation information mainly includes location information (such as current GPS / BeiDou coordinates and altitude), route planning (expected driving route, distance to the next intersection, and destination type), and map attribute data (such as road width, number of lanes, height restrictions for roads, tunnels, and parking garages, slope information, and road surface material). External environment information mainly includes obstacle information (distance values ​​measured by existing fixed sensing systems, obstacle types identified by cameras (vehicles, people, walls), and relative speeds of obstacles), meteorological environment (light intensity, weather conditions, light intensity, and visibility), and road features (lane line type, curb height, ground markings, and traffic light status).

[0042] Navigation information mainly refers to the location you are about to reach, while external environment information mainly refers to information about obstacles in the surrounding environment.

[0043] The scene recognition module is used to determine driving scenes requiring image and size assistance in terms of top, bottom, left, right, front, and rear, based on information obtained from the vehicle status monitoring module. Specifically, the driving scenes have been pre-defined with clear conditions.

[0044] The decision scheduling module is used to generate an initial activation scheme for the perception module based on the driving scenario.

[0045] The drive mechanism controls the corresponding sensing module to extend and bend to the target posture based on an initial activation scheme. The initial activation scheme includes the module number, extension / retraction displacement, joint bending angle, and target position of the sensing component.

[0046] Specifically, in some driving scenarios, such as parking scenarios, information can be obtained directly from the fixed perception system to determine that it is a parking scenario.

[0047] The multi-sensor fusion module is used to integrate data from cameras and lidar with data from a fixed sensing system and display it on the central control screen.

[0048] Preferably, the central control screen directly displays a fused view or displays images and distance information in a multi-screen format.

[0049] The limit perception system of this application enables the vehicle to automatically determine various driving scenarios requiring image and size assistance related to the upper, lower, left, right, front, and rear directions with the help of the vehicle status monitoring module and the scene recognition module. The decision scheduling module generates an initial activation scheme for the perception module based on the driving scenario according to preset activation rules. The drive mechanism extends the relevant perception modules to the target position to obtain the required limit gap field of view images and limit gap distances for the upper, lower, left, right, front, and rear directions of the corresponding driving scenario, thereby achieving omnidirectional and blind-spot-free accurate perception and solving the technical problems of blind spots and insufficient flexibility in the prior art. The multi-sensor fusion module integrates and displays the data on the central control screen, which can assist the driver in accurate driving in various limit gap scenarios.

[0050] Furthermore, in one embodiment, the limit controller is also used to dynamically change according to the actual gap change after the sensing module extends and bends to the initial target posture, thereby dynamically adjusting the state of the sensing module.

[0051] Specifically, when measuring the field of view image and the distance of the limit gap, the sensing component is moved to the middle of the gap as much as possible, and then the camera obtains a more accurate field of view image of the limit gap. At the same time, the lidar emits an oblique laser towards one of the walls or the vehicle body on both sides of the gap to obtain an accurate distance of the limit gap.

[0052] One example is ensuring that the sensing module is always located in the gap slit or the extension of the gap slit.

[0053] The limit perception system of this application, after extending the perception module to the target posture based on the initial activation scheme, can also dynamically change with the actual gap changes, which can further improve the perception accuracy.

[0054] Furthermore, in one embodiment, the central control screen includes a manual operation interface connected to the limit controller via signal communication. The manual operation interface includes an expand button, a retract button, and operation buttons for three-dimensional adjustment of the target position of the sensing component. Specifically, the three-dimensional adjustment operation buttons include buttons for adjusting up, down, forward, backward, left, and right.

[0055] Preferably, the manual operation interface includes a main interface button, that is, a main interface button is usually displayed on the central control screen. After the main interface button is opened, the main interface button expands into an expand button, a collapse button, and operation buttons for up, down, forward, backward, left, and right.

[0056] Furthermore, in one embodiment, the driving scenarios include, but are not limited to, automatic parking assistance scenarios, height-restricted driving scenarios, narrow road meeting scenarios, off-road obstacle scenarios, narrow road U-turn scenarios, and sentry monitoring scenarios.

[0057] Among them, the automatic parking cooperative scenario mainly corresponds to the perception of the extreme distances in front, behind, left and right; the height-restricted driving scenario mainly corresponds to the perception of the distance above the vehicle; the narrow road meeting scenario mainly corresponds to the perception of the distances in the left and right; the off-road obstacle scenario mainly corresponds to the perception of the distances in front and below; the narrow road U-turn scenario mainly corresponds to the perception of the extreme distances in the left and right; and the sentry monitoring scenario mainly corresponds to the perception of the distances in front, behind, left and right.

[0058] Furthermore, in one embodiment, each sensing module also includes a telescopic arm driven by a drive mechanism, one end of which is fixed to the vehicle body structure, and the other end of which is fixed to the fixed end of each serpentine multi-joint bending mechanism. Specifically, the fixed end of the serpentine multi-joint bending mechanism is opposite to the telescopic end of the serpentine multi-joint bending mechanism.

[0059] The sensing component also includes an illumination unit for providing light at night, which illuminates the user in the absence of light, making it easier for the camera to capture images.

[0060] Specifically, each telescopic arm can extend from a fixed angle from the front of the vehicle, the roof, the left side panel, the right side panel, and the bottom of the vehicle, and then bend using a serpentine multi-joint bending mechanism; the telescopic arm can also swing at a certain angle when it extends to reduce the bending amount of the serpentine multi-joint bending mechanism.

[0061] The limit sensing system of this application can use a serpentine multi-joint bending mechanism as the support structure for the sensing component, or it can use a telescopic arm plus a serpentine multi-joint bending mechanism. The ultimate goal is to allow the sensing component to move freely and flexibly.

[0062] Specifically, the serpentine multi-joint bending mechanism is used to achieve multi-degree-of-freedom, highly flexible bending of the sensing component. The mechanism employs a biomimetic snakeskin joint design, comprising multiple sequentially connected biomimetic snakeskin joints. Connecting ears are installed on both sides of each joint for connecting adjacent joints or the power compartment. An elastic rod inside each joint acts as a biomimetic spine, providing elastic support for the entire mechanism (a structure with separate control of the skeleton wire and drive wire can be used; the skeleton wire ensures tight and reliable joint connections and uniform deformation, while multiple drive wires correspond to bending movements in different directions). The serpentine multi-joint bending mechanism can achieve at least two rotational degrees of freedom, including independent or combined bending in the pitch and horizontal directions, with a maximum bending angle exceeding ±120°.

[0063] Secondly, this application discloses a perception method based on the above-mentioned extreme perception system, comprising the following steps: In the initial state, each sensing module is in a stowed state, meaning that in ordinary scenarios, the vehicle mainly uses a fixed sensing system for perception. The limit controller acquires vehicle status information, navigation information, and external environment information in real time, and determines the driving scenario based on this information. If the conditions for determining the driving scenario are not met, the fixed perception system continues to be used for perception. If the conditions for determining the driving scenario are met, the driving scenario category is determined, and a corresponding initial activation scheme is generated based on the driving scenario. The initial activation scheme already includes detailed information on how the perception components should be extended.

[0064] Based on the initial activation scheme, the limit controller controls the snake-shaped multi-joint bending mechanism of the corresponding numbered sensing module to extend to the target posture through the drive mechanism, reaching a position that is more convenient for acquiring video images and distance results.

[0065] The camera and lidar of the sensing component collect images and distance data, which are then fused with data from the fixed sensing system and displayed on the central control screen.

[0066] The perception method of this application typically involves each perception module being in a retracted state. When a certain driving scenario is reached, a corresponding initial activation scheme is generated. Based on the initial activation scheme, the limit controller controls the serpentine multi-joint bending mechanism of the corresponding numbered perception module to extend to the target posture through the drive mechanism. More accurate limit data is collected and fused with the data information of the fixed perception system, and displayed on the central control screen to improve perception accuracy and assist in completing more refined vehicle control, which has high economic value.

[0067] Furthermore, regarding the perception method, in one embodiment, the limit controller includes a vehicle state monitoring module, a scene recognition module, a decision scheduling module, and a multi-sensor fusion module.

[0068] The perception method includes the following steps: The vehicle status monitoring module acquires vehicle status information, navigation information, and external environment information in real time from the fixed sensing system, including but not limited to the fixed sensing system. The scene recognition module determines the driving scenes that require image and size assistance in terms of up, down, left, right, front, and rear based on the information obtained by the vehicle status monitoring module; the decision scheduling module generates an initial activation scheme for the perception module based on the driving scenes. The drive mechanism controls the snake-shaped multi-joint bending mechanism of the corresponding sensing module to extend to the target posture based on the initial activation scheme; wherein, the initial activation scheme includes the module number, the amount of extension and retraction displacement, the joint bending angle and the target position of the sensing component; The multi-sensor fusion module integrates data from cameras and lidar with data from a fixed sensing system, which is then displayed on the central control screen.

[0069] The perception method of this application involves a vehicle status monitoring module and a scene recognition module determining various driving scenarios that require image and size assistance related to the top, bottom, left, right, front, and rear. The decision scheduling module generates an initial activation scheme for the perception modules based on the driving scenario according to preset activation rules. The drive mechanism extends the relevant perception modules to the target position to obtain the required limit gap field of view images and limit gap distances for the top, bottom, left, right, front, and rear of the corresponding driving scenario, thereby achieving omnidirectional and blind-spot-free accurate perception. The multi-sensor fusion module integrates and displays the data on the central control screen, which can assist the driver in accurate perception and assisted driving in various limit gap scenarios.

[0070] Furthermore, regarding the perception method, in one embodiment, the limit controller further includes a scene following module, which activates each perception component according to the initial activation scheme and displays the image and distance on the central control screen, and also includes: The scene following module uses a drive mechanism to make the sensing components dynamically follow the actual gap changes; for example, when parking at the limit gap, as the gap narrows, the position of the sensing components is dynamically adjusted to always obtain the best image and gap data. The driving scenario includes the automatic parking cooperation scenario, which is further divided into the parallel extreme parking scenario, the perpendicular extreme parking scenario, and the narrow extreme parking space parking scenario.

[0071] Specifically, if the scene recognition module identifies a side-parking scenario, the decision scheduling module activates the side perception module and the rear perception module near the parking space through the drive mechanism to accurately detect the distance between the curb or wall and the wheels or vehicle body. If the scene recognition module identifies a vertical extreme parking scenario, the decision scheduling module activates the front perception module through the drive mechanism to detect whether there are low obstacles (such as wheel stops) ahead. If the scene recognition module identifies a parking scenario in a narrow, extreme parking space, the decision scheduling module activates all side modules through the drive mechanism to monitor the distance to nearby obstacles (i.e., vehicles, walls, or curbs) in real time, which assists the parking system in accurately controlling steering and braking.

[0072] Specifically, the judgment rule for the automatic parking cooperative scenario is: the vehicle speed is within a set range close to zero, and the fixed perception system detects the parking space. The judgment rules for the parallel parking extreme limit scenario, the perpendicular parking extreme limit scenario, and the narrow parking space extreme limit scenario are also pre-set.

[0073] The perception method described in this application can provide distance information with millimeter-level accuracy for a variety of different parking scenarios, significantly improving parking success rate and enhancing the driving experience.

[0074] Furthermore, regarding the perception method, in one embodiment, the driving scenario includes a height-restricted driving scenario, a narrow road meeting scenario, and an off-road obstacle scenario.

[0075] Regarding height-restricted driving scenarios, if the scenario recognition module determines that it is about to encounter a height restriction frame based on the navigation information obtained by the vehicle status monitoring module, the initial activation scheme generated by the decision scheduling module is to activate the roof perception module when the vehicle speed drops below the first set speed, in order to scan and collect images and spacing of the space above.

[0076] Specifically, for example: During normal vehicle operation, the navigation information obtained by the vehicle status monitoring module includes a warning that the vehicle will enter an underground parking garage with a height restriction of 2.2 meters 200 meters ahead, and the current actual height of the vehicle (including the roof rack) is 1.85 meters. Based on the navigation information, roof height parameters, and current vehicle speed (30 km / h), the scene recognition module predicts that the vehicle will soon enter a height-restricted driving scenario. The decision scheduling module generates an instruction: when the vehicle speed drops below 20 km / h (corresponding to the first set speed), the roof sensing module is activated. The drive mechanism makes the sensing components (camera and lidar) of the roof sensing module face upward and forward. The serpentine multi-joint bending mechanism gradually bends according to the preset height restriction detection posture through the traction drive wire, so that the sensing components point to the upward and forward direction at about 45°. The lidar begins to scan the space above, and the camera simultaneously acquires images.

[0077] The multi-sensor fusion module maps the distance data detected by the LiDAR onto the image, generating a real-time view with a distance scale. This view is displayed on the central control screen, along with text prompts such as "Remaining roof clearance: 35cm". If the system determines that the remaining clearance is less than a safe threshold (e.g., 8cm), it will issue a continuous buzzer alarm.

[0078] After the vehicle passes through the height restriction area, its speed returns to above 30km / h. The limit controller determines that the height restriction scenario has ended and controls the serpentine bending mechanism to slowly reset.

[0079] In particular, regarding the narrow road meeting scenario, if the scene recognition module determines that a narrow road meeting scenario is encountered when an oncoming vehicle is encountered based on the external environment information obtained by the vehicle status monitoring module, the initial activation scheme generated by the decision scheduling module is to reduce the vehicle speed to the second set speed and activate the perception modules of the left and right sides to obtain the extreme distance between the two sides. Specifically, for example: a vehicle is traveling on a road that is only wide enough for one vehicle to pass at a time, and an oncoming vehicle is approaching. The limit controller learns from the fixed sensing system that the distances on both sides are narrow (e.g., 0.5m from the wall on the left and 1.2m from the oncoming vehicle on the right), and combined with the vehicle speed being less than 15km / h (corresponding to the second set speed), the scene recognition module determines that it has entered a narrow road meeting scenario.

[0080] The decision-making and scheduling module activates the side perception modules below the A-pillars on both sides. The serpentine bending mechanism of the side perception modules bends upward at about 30°, so that the camera's field of view covers the side of the vehicle and the rearview mirror area. At the same time, the lidar continuously measures the distance.

[0081] The multi-sensor fusion module integrates side images with radar data and fixed sensing systems. On the display unit, centered on a 3D vehicle model, it displays side images in real-time on both sides, dynamically updating digital distances: "e.g., left side distance to the wall: 12cm, right side distance to oncoming vehicle: 25cm." When the right distance is less than 10cm, the right side of the screen border turns yellow and flashes; when it's less than 8cm, it turns red and an alarm sounds. After passing the oncoming vehicle, the limit controller automatically retracts the side sensing modules.

[0082] Regarding off-road obstacle scenarios, if the scene recognition module determines that the road is full of rocks and ditches based on the external environment information obtained by the vehicle status monitoring module, the initial activation scheme generated by the decision scheduling module is to activate the front of the vehicle, the bottom of the vehicle, the left side and the right side, in order to obtain the obstacle images and spacing below the front of the vehicle, below the bottom of the vehicle, and to the left and right of the tires on both sides.

[0083] Specifically, when the vehicle is driving on off-road terrain, the fixed sensing system detects that the road surface is covered with rocks and ditches, and the scene recognition module identifies it as an off-road obstacle scene, activating the following modules: Front-end sensing module: Bends forward and downward to detect potholes or protruding rocks on the road ahead; Left and right side sensing modules: curved downwards and slightly outwards to detect obstacles in the tire's path; Vehicle undercarriage sensing module: Vertically downward, it displays the gap between the chassis and the ground in real time.

[0084] All sensor data is fused and processed to generate a 3D terrain model of the vehicle's surroundings, which is then displayed on the central control screen, using different colors to indicate terrain at different elevations (green for safety, yellow for warning, and red for danger). The driver can intuitively select the best route. When the limit controller detects that one wheel is about to sink into a deep ditch, it automatically issues steering suggestions or provides a vibration warning through the steering wheel.

[0085] Regarding U-turn scenarios on narrow roads, when the fixed sensing system detects a U-turn on a very narrow road, or when traditional fixed radar indicates that the rear of the vehicle has already touched roadside bushes on a narrow road, the wheels may still be some distance from the roadside. The scene recognition module identifies the narrow road U-turn scenario, and the decision-making and scheduling module automatically activates the front / rear / side sensing modules. Alternatively, the user can manually activate the front / rear / side sensing modules to provide images and distance information of the wheels from the roadside. When the user makes a U-turn at an extreme position, the display unit can also display the distance. If the distance between the wheels and the roadside edge is less than the safe distance (e.g., 5cm), an image or sound alarm will be given. After the user completes the U-turn, the sensing modules are automatically or manually retracted.

[0086] In the sentry monitoring scenario, when a vehicle is parked in an unfamiliar environment, the driver activates "Sentry Mode." The system periodically (e.g., every 5 minutes) or when the vehicle's vibration sensors are triggered, it activates certain sensing modules (such as the roof and side sensing modules) to scan and monitor the area around the vehicle. The serpentine bending mechanism can rotate 360° to cover a wider area. Once it detects someone lingering for an extended period or approaching abnormally, it automatically records video and sends an alert to the vehicle owner's mobile phone. The perception method described in this application can provide powerful and accurate perception assistance for scenarios with height restrictions, narrow road encounters, off-road obstacles, and narrow road U-turns, improving the vehicle's ability to cope with some extreme scenarios, greatly enhancing the driving experience, and has high economic value.

[0087] In the description of this application, it should be noted that the terms "upper," "lower," etc., indicating the orientation or positional relationship are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Unless otherwise expressly specified and limited, the terms "installed," "connected," and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication between two elements. For those skilled in the art, the specific meaning of the above terms in this application can be understood according to the specific circumstances.

[0088] It should be noted that in this application, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0089] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A limit sensing system complementary to a fixed sensing system, characterized in that, Include: Several numbered sensing modules, at least one at each of the front, rear, roof, under, left side and right side of the vehicle; each sensing module includes a serpentine multi-joint bending mechanism and a sensing component, each sensing component being fixed to the telescopic end of the serpentine multi-joint bending mechanism; the sensing component includes a camera and a lidar. Several driving mechanisms, each driving mechanism is used to drive a sensing module; The limit controller is used to acquire vehicle status information, navigation information and external environment information from the fixed sensing system, including but not limited to the fixed sensing system, determine the driving scenario, generate an initial activation scheme, and control the corresponding numbered sensing module to extend to the target posture through the drive mechanism based on the initial activation scheme. The limit controller is also used to fuse the data information acquired by the camera and lidar into the data information of the fixed sensing system and display it on the central control screen.

2. The limit sensing system as described in claim 1, which complements a fixed sensing system, is characterized in that, The limit controller includes: The vehicle status monitoring module is used to acquire vehicle status information, navigation information, and external environment information in real time from the fixed sensing system, including but not limited to the fixed sensing system. The scene recognition module is used to determine driving scenarios that require image and size assistance in terms of up, down, left, right, front, and rear, based on information obtained from the vehicle status monitoring module. The decision scheduling module is used to generate an initial activation scheme for the perception module based on the driving scenario; the drive mechanism controls the corresponding perception module to extend to the target posture based on the initial activation scheme; wherein, the initial activation scheme includes the module number, extension displacement, joint bending angle and target position of the perception component. The multi-sensor fusion module is used to fuse data information acquired by cameras and lidar into data information from a fixed sensing system and display it on the central control screen.

3. The limit sensing system as described in claim 2, which complements a fixed sensing system, is characterized in that: The limit controller is also used to dynamically change according to the actual gap after the sensing module extends and bends to the initial target posture.

4. The limit sensing system as described in claim 1, which complements a fixed sensing system, is characterized in that: The central control screen includes a manual operation interface connected to the limit controller via signal communication; the manual operation interface includes an expand button, a retract button, and operation buttons for three-dimensional directional adjustment of the target position of the sensing component.

5. The limit sensing system as described in claim 1, which complements a fixed sensing system, is characterized in that: The driving scenarios include automatic parking assistance, height-restricted driving, narrow road meeting, off-road obstacle course, narrow road U-turn, and sentry monitoring.

6. The limit sensing system as described in claim 1, which complements a fixed sensing system, is characterized in that: Each of the sensing modules also includes a telescopic arm driven by the drive mechanism, one end of which is fixed to the vehicle body structure and the other end of which is fixed to the fixed end of each serpentine multi-joint bending mechanism. The sensing component also includes an illumination unit for providing light to the camera at night.

7. A sensing method based on the extreme sensing system as described in claim 1, characterized in that, Includes the following steps: In the initial state, each sensing module is in a stowed-away state; The limit controller acquires vehicle status information, navigation information, and external environment information in real time, determines the driving scenario, and generates an initial activation scheme. Based on the initial activation scheme, the limit controller controls the snake-shaped multi-joint bending mechanism of the corresponding numbered sensing module to extend to the target posture through the drive mechanism; The camera and lidar of the sensing component collect images and distance data, which are then fused with data from the fixed sensing system and displayed on the central control screen.

8. The sensing method of the extreme sensing system as described in claim 7, characterized in that, The limit controller includes a vehicle status monitoring module, a scene recognition module, a decision scheduling module, and a multi-sensor fusion module. The perception method includes the following steps: The vehicle status monitoring module acquires vehicle status information, navigation information, and external environment information in real time from the fixed sensing system, including but not limited to the fixed sensing system. The scene recognition module determines the driving scenes that require image and size assistance in terms of up, down, left, right, front, and rear based on the information obtained by the vehicle status monitoring module; the decision scheduling module generates an initial activation scheme for the perception module based on the driving scenes. The drive mechanism controls the serpentine multi-joint bending mechanism of the corresponding sensing module to extend to the target posture based on the initial activation scheme; wherein, the initial activation scheme includes the module number, the amount of extension and retraction displacement, the joint bending angle and the target position of the sensing component; The multi-sensor fusion module integrates data from cameras and lidar with data from a fixed sensing system, which is then displayed on the central control screen.

9. The sensing method of the limit sensing system as described in claim 8, characterized in that: The limit controller also includes a scene following module, and the display on the central control screen further includes: The scene following module uses a drive mechanism to make the sensing components dynamically follow the actual gap changes; The driving scenario includes an automatic parking cooperative scenario, which is further divided into a side extreme parking sub-scenario, a perpendicular extreme parking sub-scenario, and a narrow extreme parking space parking sub-scenario. If the scene recognition module identifies the vehicle as parallel parking, the decision scheduling module activates the side perception module and the rear perception module closest to the parking space through the drive mechanism. If the scene recognition module identifies the vehicle as perpendicular parking, the decision scheduling module activates the front perception module through the drive mechanism. If the scene recognition module identifies a narrow parking space, the decision scheduling module activates all side modules through the drive mechanism.

10. The sensing method of the limit sensing system as described in claim 8, characterized in that: The driving scenarios include height-restricted driving scenarios, narrow road passing scenarios, and off-road obstacle scenarios; If the scene recognition module determines that it is about to enter a height-restricted driving scene based on the navigation information obtained by the vehicle status monitoring module, the initial activation scheme generated by the decision scheduling module is to activate the roof perception module when the vehicle speed drops below the first set speed, so as to collect images and spacing information of the space above the roof. If the scene recognition module determines that a narrow road encountering an oncoming vehicle is a narrow road meeting scenario based on the external environment information obtained by the vehicle status monitoring module, the initial activation scheme generated by the decision scheduling module is to reduce the vehicle speed to the second set speed and activate the perception modules on the left and right sides to obtain the extreme distance between the two sides. If the scene recognition module determines that the road is full of rocks and ditches based on the external environment information obtained by the vehicle status monitoring module, it is considered an off-road obstacle scene. The initial activation scheme generated by the decision scheduling module is to activate the perception modules of the front, bottom, left side and right side of the vehicle to obtain obstacle images and spacing below the front of the vehicle, below the bottom of the vehicle, and to the left and right of the tires on both sides.

Images