Control method and device, electronic device, movable device, vehicle or robot

Abstract

The application discloses a control method and device, electronic equipment, movable equipment, vehicle or robot, relates to the technical field of artificial intelligence, and mainly comprises the following technical schemes: acquiring sensing information of a target driving area of a mobile device based on a multi-source environment sensing device; identifying a target obstacle in the target driving area based on the sensing information; wherein the target obstacle comprises at least one of a suspended obstacle and a special-shaped obstacle; and controlling the mobile device to perform a braking operation in the case where it is determined that there is a collision risk between the mobile device and the target obstacle. Through the multi-source environment sensing device, the target obstacle can be effectively detected, and the braking operation of the mobile device is performed when it is determined that there is a collision risk with the target obstacle, so that the anti-collision safety performance is improved.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more particularly to a control method and apparatus, electronic devices, mobile devices, vehicles or robots. Background Technology

[0002] As a core technology of intelligent vehicles, assisted driving safety is widely used in low-speed parking and active collision avoidance in complex scenarios. However, existing collision avoidance methods, which directly use a single sensor or a simple fusion strategy, may lead to missed detections or misjudgments of irregularly shaped or visually unrecognizable targets, thus affecting safety performance in low-speed scenarios. Summary of the Invention

[0003] This disclosure provides a control method and apparatus, electronic device, mobile device, vehicle, or robot. Its main purpose is to solve the problem of missed detection or misjudgment of irregularly shaped or visually unrecognizable targets, thereby affecting safety performance in low-speed scenarios.

[0004] According to a first aspect of this disclosure, a control method is provided, comprising: Based on a multi-source environmental perception device, obtain perception information of the target driving area of ​​the mobile device; Based on the perceived information, target obstacles within the target driving area are identified; wherein, the target obstacles include at least one of: suspended obstacles and irregularly shaped obstacles; If a collision risk is determined between the mobile device and the target obstacle, the mobile device is controlled to perform a braking operation.

[0005] In some embodiments, identifying target obstacles within the target driving area based on the perceived information includes: The target obstacle is identified based on the outline and / or shape information of the obstacle in the perceived information.

[0006] In some embodiments, identifying target obstacles within the target driving area based on the perceived information includes: Determine the height of the obstacle above the ground in the perceived information; If the ground clearance is within the target height range, it is determined that the target obstacle exists within the target driving area.

[0007] In some embodiments, identifying target obstacles within the target driving area based on the perceived information includes: Determine the ground clearance information of the obstacle and the horizontal extension information of the obstacle from the perceived information; If the ground clearance information is within the target height range and the horizontal extension information is within the target extension condition, it is determined that the target obstacle exists within the target driving area.

[0008] In some embodiments, determining that the target obstacle exists within the target driving area includes: Based on the perceived information, obstacles with tubular outlines are identified. If the diameter of the tubular obstacle is greater than the target diameter and the ground clearance is within the target height range, then the suspended obstacle is determined to exist within the target driving area.

[0009] In some embodiments, determining that the target obstacle exists within the target driving area includes: Based on the perceived information, obstacles with block-like shapes are identified. If the size of the block-shaped obstacle does not exceed the size of the mobile device and its ground clearance is within the target height range, then the suspended obstacle is determined to exist within the target driving area.

[0010] In some embodiments, identifying target obstacles within the target driving area based on the perceived information includes: Obtain the three-dimensional contour shape of the obstacle in the perceived information; wherein, the contour shape includes the three-dimensional contour shape; and / or, Obtain the size information of the obstacles from the perceived information; If the three-dimensional contour shape matches the target contour shape, and / or the size information of the obstacle does not exceed the target size information, it is determined that the irregular obstacle exists in the target driving area.

[0011] In some embodiments, determining that there is a collision risk between the mobile device and the target obstacle includes: Determine the degree of overlap and spacing information between the target obstacle and the mobile device; If the degree of overlap is greater than or equal to a first safety threshold and the spacing information is less than or equal to a second safety threshold, then a collision risk is determined to exist between the mobile device and the target obstacle.

[0012] In some embodiments, the multi-source environmental sensing device is deployed in at least one of the head sensing region, side sensing region, and tail sensing region of the mobile device; The method of acquiring perception information of the target driving area of ​​the mobile device based on the multi-source environmental perception device includes: The original sensing information within the target driving area is collected from multiple angles using the multi-source environmental sensing devices corresponding to the head sensing area, the side sensing area, and / or the tail sensing area. The original perception information from multiple regions is fused to obtain the perception information of the target driving area.

[0013] In some embodiments, the multi-source environmental sensing device includes at least one of an optical detection sensing module, an image acquisition module, a microwave detection module, and a wireless positioning module.

[0014] In some embodiments, the method further includes: A prompt message is displayed on the interactive interface of the mobile device; wherein the prompt message is used to inform the user that the mobile device is at risk of collision and that a braking operation has been performed.

[0015] According to a second aspect of this disclosure, a control device is provided, comprising: The acquisition unit is used to acquire perception information of the target driving area of ​​the mobile device based on the multi-source environmental perception device. A determining unit is configured to identify target obstacles within the target driving area based on the perceived information; wherein the target obstacles include at least one of: suspended obstacles and irregularly shaped obstacles; A control unit is configured to control the mobile device to perform a braking operation when it is determined that there is a risk of collision between the mobile device and the target obstacle.

[0016] In some embodiments, the determining unit is further configured to: The target obstacle is identified based on the outline and / or shape information of the obstacle in the perceived information.

[0017] In some embodiments, the determining unit is further configured to: Determine the height of the obstacle above the ground in the perceived information; If the ground clearance is within the target height range, it is determined that the target obstacle exists within the target driving area.

[0018] In some embodiments, the determining unit is further configured to: Determine the ground clearance information of the obstacle and the horizontal extension information of the obstacle from the perceived information; If the ground clearance information is within the target height range and the horizontal extension information is within the target extension condition, it is determined that the target obstacle exists within the target driving area.

[0019] In some embodiments, the determining unit is further configured to: Based on the perceived information, obstacles with tubular outlines are identified. If the diameter of the tubular obstacle is greater than the target diameter and the ground clearance is within the target height range, then the suspended obstacle is determined to exist within the target driving area.

[0020] In some embodiments, the determining unit is further configured to: Based on the perceived information, obstacles with block-like shapes are identified. If the size of the block-shaped obstacle does not exceed the size of the mobile device and its ground clearance is within the target height range, then the suspended obstacle is determined to exist within the target driving area.

[0021] In some embodiments, the determining unit is further configured to: Obtain the three-dimensional contour shape of the obstacle in the perceived information; wherein, the contour shape includes the three-dimensional contour shape; and / or, Obtain the size information of the obstacles from the perceived information; If the three-dimensional contour shape matches the target contour shape, and / or the size information of the obstacle does not exceed the target size information, it is determined that the irregular obstacle exists in the target driving area.

[0022] In some embodiments, determining that there is a collision risk between the mobile device and the target obstacle includes: Determine the degree of overlap and spacing information between the target obstacle and the mobile device; If the degree of overlap is greater than or equal to a first safety threshold and the spacing information is less than or equal to a second safety threshold, then a collision risk is determined to exist between the mobile device and the target obstacle.

[0023] In some embodiments, the multi-source environmental sensing device is deployed in at least one of the head sensing region, side sensing region, and tail sensing region of the mobile device; The method of acquiring perception information of the target driving area of ​​the mobile device based on the multi-source environmental perception device includes: The original sensing information within the target driving area is collected from multiple angles using the multi-source environmental sensing devices corresponding to the head sensing area, the side sensing area, and / or the tail sensing area. The original perception information from multiple regions is fused to obtain the perception information of the target driving area.

[0024] In some embodiments, the multi-source environmental sensing device includes at least one of an optical detection sensing module, an image acquisition module, a microwave detection module, and a wireless positioning module.

[0025] In some embodiments, the apparatus further includes: The display unit is used to display a prompt message on the interactive interface of the mobile device; wherein the prompt message is used to inform the user that the mobile device has the risk of collision and that a braking operation has been performed.

[0026] According to a third aspect of this disclosure, an electronic device is provided, comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect above.

[0027] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are configured to cause the computer to perform the method described in the first aspect above.

[0028] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method described in the first aspect above.

[0029] According to a sixth aspect of this disclosure, a mobile device is provided, the mobile device including a control device as described in the second aspect or an electronic device as described in the third aspect.

[0030] According to a seventh aspect of this disclosure, a vehicle or robot is provided, the vehicle or robot including a movable device as described in the sixth aspect; or, a control device as described in the second aspect.

[0031] The control method, apparatus, electronic device, mobile device, vehicle, or robot provided in this disclosure acquires perception information of the target driving area of ​​the mobile device based on a multi-source environmental perception device; identifies target obstacles within the target driving area based on the perception information; wherein the target obstacles include at least one of: suspended obstacles and irregularly shaped obstacles; and controls the mobile device to perform a braking operation when it is determined that there is a collision risk between the mobile device and the target obstacle. Through the multi-source environmental perception device, target obstacles can be effectively detected, and when a collision risk with a target obstacle is determined, the braking operation of the mobile device can be performed, thereby improving collision avoidance safety performance.

[0032] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0033] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein: Figure 1 A flowchart illustrating a control method provided in an embodiment of this disclosure; Figure 2 This is a schematic diagram of a suspended tubular obstacle or a suspended block obstacle provided in an embodiment of the present disclosure; Figure 3 A schematic diagram of a stone pier obstacle provided in an embodiment of this disclosure; Figure 4 This is a schematic diagram of a columnar obstacle provided in an embodiment of the present disclosure; Figure 5 This is a schematic diagram of the sensing area of ​​a multi-source environmental sensing device provided in an embodiment of the present disclosure; Figure 6 This is a flowchart illustrating another control method provided in an embodiment of the present disclosure; Figure 7 This is a flowchart illustrating another control method provided in an embodiment of the present disclosure; Figure 8 This is a schematic diagram illustrating the overlap of a target obstacle and a mobile device, provided as an embodiment of this disclosure. Figure 9 This is a flowchart illustrating another control method provided in an embodiment of the present disclosure; Figure 10 A schematic diagram of an interactive interface provided in an embodiment of this disclosure; Figure 11 This is a schematic diagram of the structure of a control device provided in an embodiment of the present disclosure; Figure 12 A schematic diagram of another control device provided in an embodiment of this disclosure; Figure 13 A schematic block diagram of an example electronic device provided for embodiments of this disclosure. Detailed Implementation

[0034] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0035] The control method, apparatus, electronic device, and storage medium of the present disclosure are described below with reference to the accompanying drawings.

[0036] Figure 1 This is a flowchart illustrating a control method provided in an embodiment of this disclosure. The method is applied to a mobile device, which may be a robot, a vehicle, or other mobile devices with control functions, etc. Specific embodiments of this disclosure do not limit the demonstration scenario. Figure 1 As shown, the method includes the following steps: Step 101: Based on the multi-source environmental perception device, obtain the perception information of the target driving area of ​​the mobile device.

[0037] Multi-source environmental sensing devices, configured at different locations on a mobile device and with different field of view angles, are used to actively scan and detect the three-dimensional space surrounding the mobile device. In some embodiments, the multi-source environmental sensing device includes at least one of an optical detection sensing module, at least one image acquisition module, at least one microwave detection module, and at least one wireless positioning module. For example, the optical detection sensing module includes a lidar, the image acquisition module includes a vision sensor, the microwave detection module includes a millimeter-wave radar, and the wireless positioning module includes an ultra-wideband (UWB) positioning antenna, etc. Specifically, the specific type and number of multi-source environmental sensing devices are not limited in the embodiments of this disclosure.

[0038] The multi-source environmental sensing device comprises multiple modules that work together to effectively expand the sensing range, covering the forward, backward, lateral, and near-field blind zones of the mobile device, forming a comprehensive and multi-layered sensing information of the environment surrounding the mobile device. This sensing information includes not only the three-dimensional coordinates of each reflection point but also attribute information such as reflection intensity. These data collectively constitute a raw description of the spatial position and contours of static and dynamic objects around the mobile device.

[0039] In one implementation, the mobile device is equipped with four lidars, including one forward-facing lidar and three blind-spot lidars respectively arranged on the sides and rear. This combination of multiple lidars enables the collection of blind-spot perception information of the surrounding environment of the mobile device, providing sufficient and redundant environmental information for subsequent processing.

[0040] Step 102: Identify target obstacles within the target driving area based on the perceived information; wherein the target obstacles include at least one of: suspended obstacles and irregularly shaped obstacles.

[0041] By analyzing the geometric shape, size parameters, and relative positional relationships in the perceived information, target obstacles are identified and classified into different types. Specifically, the identified target obstacles include at least one of the following: suspended obstacles and irregularly shaped obstacles. Suspended obstacles refer to objects with a certain three-dimensional dimension and a gap between their lower end and the ground; their identification relies on a comprehensive judgment of their height above the ground, horizontal extension range, and / or vertical thickness. For example, Figure 2 The obstacles shown include suspended tubular obstacles and suspended block-shaped obstacles (obstacles shown in red), etc.; irregular obstacles include columnar obstacles, wall-shaped obstacles, stone pier obstacles, etc. Columnar obstacles refer to upright objects with an approximately circular or elliptical cross-section and a height greater than the target diameter. Their identification is based on the clustering shape on the horizontal plane and continuity in the vertical direction. For example, circular columnar utility poles, streetlights, trees, etc. Wall-shaped obstacles refer to upright or inclined objects with a relatively large planar area and a certain thickness. For example, cement walls, load-bearing columns, etc. Stone pier obstacles refer to obstacles that are fixedly placed on the road surface and have a regular or irregular shape. Figure 3 The stone bollard obstacle shown; a columnar obstacle refers to an object that is upright and cylindrical or nearly cylindrical, with its base fixed to the road surface and its upper part capable of integrating functional modules, such as... Figure 4 The columnar obstacle shown. It should be noted that... Figure 2 , Figure 3 and Figure 4 The illustrative description of the target obstacle is provided for ease of understanding and is not intended to limit the target obstacle to only including [specific types of obstacles]. Figure 2 , Figure 3 and Figure 4 The three obstacles shown.

[0042] As one implementation method, the geometric parameter thresholds of various obstacles can be preset, such as the range of the lower end of the suspended obstacle from the ground, the minimum diameter and height of the columnar obstacle, the minimum area and thickness of the wall-like obstacle, the shape and size of the stone block obstacle, etc., and the clustering or segmentation results in the perception information that meet the corresponding threshold conditions are determined as the target obstacles of the corresponding type.

[0043] Step 103: If it is determined that there is a risk of collision between the mobile device and the target obstacle, control the mobile device to perform a braking operation.

[0044] When a target obstacle is identified within the target driving area, and a collision risk is identified between the mobile device and the target obstacle, the mobile device's automatic emergency braking system is triggered. This system causes the mobile device to autonomously decelerate until it comes to a complete stop, thereby avoiding or mitigating a collision with the target obstacle. This braking operation can encompass various braking strategies, including but not limited to dynamically adjusting the braking intensity and timing based on factors such as the collision risk level, the mobile device's current speed, the type of target obstacle, and its relative position, thereby ensuring both safety and driving smoothness.

[0045] In one implementation, the braking operation can be triggered in scenarios where the mobile device (such as a vehicle) is traveling at low speed or reversing. For example, when there is a risk of collision between the vehicle and a suspended obstacle, pole obstacle, or wall obstacle behind or to the side, the vehicle is controlled to brake, so that the vehicle comes to a complete stop or reduces the collision speed before the collision occurs.

[0046] By introducing a braking operation mechanism, the system achieves proactive response and autonomous intervention to the risk of collision with target obstacles, enhancing the safety protection capability of mobile devices in complex environments and effectively reducing the incidence of collision accidents caused by driver misjudgment or blind spots.

[0047] The control method provided in this disclosure acquires perception information of the target driving area of ​​a mobile device based on a multi-source environmental sensing device; identifies target obstacles within the target driving area based on the perception information; wherein the target obstacles include at least one of: suspended obstacles and irregularly shaped obstacles; and controls the mobile device to perform a braking operation when it is determined that there is a collision risk between the mobile device and the target obstacle. By using a multi-source environmental sensing device, target obstacles can be effectively detected, and when a collision risk with a target obstacle is determined, the braking operation of the mobile device can be performed, thereby improving collision avoidance safety performance.

[0048] In some embodiments, the multi-source environmental sensing device is deployed in at least one of the head sensing region, side sensing region, and tail sensing region of the mobile device. Figure 5 This is a schematic diagram of the sensing area of ​​a multi-source environmental sensing device provided in an embodiment of this disclosure, as shown below. Figure 5As shown, the mobile device is equipped with a head sensing area, a side sensing area, and a tail sensing area. The head sensing area is located at the front of the mobile device, covering the sensing range of the area in front of the mobile device; the side sensing area is located at the side of the mobile device, covering the sensing range of the area to the side of the mobile device; and the tail sensing area is located at the rear of the mobile device, covering the sensing range of the area behind the mobile device. Active detection devices deployed in each sensing area detect in their respective directions, achieving omnidirectional acquisition of the surrounding environment of the mobile device. The active detection devices in the tail sensing area have corresponding sensing ranges and blind spots. The sensing range covers the road surface area behind the mobile device, and the blind spot is the detection dead zone that cannot be covered by the tail sensing area. When the mobile device is moving backward, the multi-source environmental sensing device corresponding to the tail sensing area detects the area behind the mobile device to obtain sensing information of the rear area. When there is a target obstacle behind the mobile device, the multi-source environmental sensing device corresponding to the tail sensing area can collect sensing information reflecting that area.

[0049] Optionally, based on any of the above embodiments or optional methods, the present disclosure embodiments may also employ the following methods, but are not limited thereto: when identifying target obstacles within the target driving area based on the perception information, the following methods may be employed, but are not limited to: identifying the target obstacle based on the outline shape and / or shape information of the obstacle in the perception information.

[0050] In the process of identifying target obstacles, the embodiments of this disclosure are based on point cloud cluster data corresponding to the perception information to identify and determine the outline shape and / or morphological information of the obstacle. In some implementations, the outline shape and / or morphological information includes, but is not limited to, tubular structure, block structure, three-dimensional outline shape, box shape, cylindrical shape, irregular outline, etc. The embodiments of this disclosure do not specifically limit the outline shape and morphological information of the obstacle.

[0051] In other embodiments, when identifying target obstacles within the target driving area based on the perceived information, the following methods may be used, but are not limited to: Figure 6 As shown, it includes: Step 201: Determine the height of the obstacle above the ground in the perceived information.

[0052] The lower edge height and horizontal projection size of the obstacle point cloud cluster are extracted based on perception information. In some embodiments, the coordinates of the lowest point of the point cloud cluster in the vertical direction are determined to obtain the ground clearance of the obstacle.

[0053] Step 202: If the ground clearance is within the target height range, determine that the target obstacle exists within the target driving area.

[0054] In this embodiment, the target obstacle can be a suspended obstacle, which refers to an obstacle that has a gap between itself and the ground.

[0055] To facilitate obstacle identification, the gap was quantified. If the ground clearance of the obstacle is determined to be within the target height range, it is determined that the target obstacle exists within the target driving area, i.e., there is a suspended obstacle.

[0056] This disclosure provides an adaptive and configurable optimization for the target height range during obstacle recognition, adapting to the collision avoidance requirements of different vehicle models and driving scenarios, and avoiding misidentification and mis-braking issues caused by fixed parameter thresholds. In some embodiments, the target height range is a configurable, dynamically variable parameter range that can be adapted and adjusted according to device parameters such as the chassis height and height of the mobile device. For example, the target height range is [20 cm, 220 cm], [15 cm, 200 cm], [10 cm, 250 cm], etc., and the above ranges can match the collision protection boundaries of conventional mobile devices, used to filter suspended obstacles that pose a real collision risk to the mobile device.

[0057] In some embodiments, if the lower limit of the target height range is configured too small, for example, set to [2 cm, 300 cm], the obstacle's ground clearance is 2 cm. This height is lower than the normal ground clearance of the mobile device's chassis, so the obstacle cannot collide with the device's body or chassis, posing no actual safety risk. In this case, if the obstacle is included in the target obstacle recognition range and braking is triggered, it will result in invalid braking. If the upper limit of the target height range is configured too large, i.e., the suspended height is higher than the maximum vehicle height, for example, the maximum suspended distance is configured to 300 cm, such high-mounted suspended obstacles pose no risk of collision with the mobile device. Braking them would greatly reduce the smoothness of the mobile device's operation and the user's driving experience.

[0058] Optionally, based on any of the above embodiments or optional methods, the present disclosure embodiments may also employ the following methods, but not limited thereto, including: determining the ground clearance information of the obstacle and the ground in the perception information, and the horizontal extension information of the obstacle; and determining that the target obstacle exists in the target driving area when the ground clearance information is within the target height range and the horizontal extension information is within the target extension condition.

[0059] In this embodiment, the ground clearance information represents the vertical distance between the obstacle and the road surface on which the mobile device travels; the horizontal extension information represents the maximum physical dimension of the obstacle extending outward relative to its own attachment surface (e.g., wall surface, vertical wall surface, etc.) in the horizontal direction; the target extension condition is a physical quantification threshold value set for the horizontal extension information; if the ground clearance information of the obstacle is within the target height range, but the horizontal extension information is extremely small, for example, protruding outward by less than 1 cm, then such a small protrusion is merely a structural feature of the wall surface, and even if the mobile device comes into contact with it during actual movement, it will not cause substantial physical interference or collision damage; therefore, ground clearance information alone cannot accurately locate obstacles with real collision threats, and horizontal extension information must be added as a supplementary constraint for the physical space dimension.

[0060] As one possible approach, the target height range can be set to a vertical distance between the lower end of the obstacle and the ground that is between 20 cm and 220 cm; the target extension condition can be set to a maximum extension distance of the obstacle in the horizontal plane away from the attached base surface that is greater than or equal to 10 cm, or a total projected area of ​​the obstacle in the horizontal plane that is greater than or equal to 50 square centimeters; in the actual detection process, based on the three-dimensional point cloud data obtained by the multi-source environmental sensing device, the ground clearance information can be obtained by extracting the coordinates of the lowest point of the point cloud cluster in the vertical direction, and the horizontal extension information can be obtained by calculating the outer contour boundary value of the point cloud cluster on the horizontal projection plane; when the ground clearance information is within the target height range and the horizontal extension information simultaneously meets the target extension condition, it can be determined that there is a target obstacle in the target driving area.

[0061] By introducing horizontal extension information as an auxiliary dimension for judging target obstacles based on the above-mentioned height information, it is possible to effectively filter out small protrusions on the wall surface, such as tiny bolt protrusions and extremely short decorative lines, which do not pose a physical collision threat. This avoids ineffective braking caused by tiny structures. While improving the accuracy of identifying suspended obstacles, it also clearly limits the judgment boundary of target obstacles to the size range with real physical interference risks.

[0062] Optionally, based on any of the above embodiments or optional methods, the present disclosure embodiments may also employ the following methods, but not limited thereto, including: identifying obstacles with a tubular outline based on the perception information; and determining that the suspended obstacle exists within the target driving area when it is determined that the diameter of the tubular obstacle is larger than the target diameter and the ground clearance information is within the target height range.

[0063] In this embodiment, an obstacle with a tubular outline refers to a physical entity whose perceived data (e.g., 3D point cloud clustering) spatially exhibits cylindrical or near-cylindrical geometric features. The pipe diameter is the physical diameter of the circular cross-section of the tubular structure. The target pipe diameter is a pre-set physical threshold value used to distinguish between robust pipes with rigid collision strength and thin, slender ropes or cables that pose no collision threat. When the pipe diameter of a tubular obstacle is less than or equal to the target pipe diameter, such obstacles are typically too thin. When a mobile device comes into contact with such an obstacle, it is easily deformed or directly crushed, without causing substantial physical damage or obstruction. Therefore, there is no need to trigger emergency braking from a safety perspective. Ground clearance information refers to the vertical distance between the lowest point of the tubular obstacle and the ground. Only when the pipe diameter is greater than the target pipe diameter and the ground clearance information is within the target height range will the tubular structure be identified as a suspended obstacle within the target driving area.

[0064] As one feasible approach, the target pipe diameter can be set to be greater than or equal to 10 centimeters; the target height range can be set to 20 centimeters to 220 centimeters above the ground. During the actual detection process, the fitted diameter of the tubular obstacle can be calculated using 3D point cloud data obtained from LiDAR as the pipe diameter. For example, when a mobile device is reversing, the perception system detects a suspended pipe on the wall. Calculations show that the pipe's outline is tubular, with a diameter of 15 centimeters, which is 10 centimeters larger than the target pipe diameter, and a height of 80 centimeters above the ground, falling within the target height range of 20 to 220 centimeters. This confirms the presence of a suspended obstacle within the target driving area. Conversely, if a thin rope with a diameter of less than 1 centimeter hangs from the wall, even if its height above the ground also meets the target height range, it will not be considered a suspended obstacle because its diameter is less than 10 centimeters larger than the target pipe diameter. This effectively eliminates many thin cables that pose no physical threat.

[0065] By incorporating the above method into the identification logic of suspended obstacles, a comparison condition between the pipe diameter and the target pipe diameter is introduced. Combined with the ground clearance information, a comprehensive judgment is made, which can accurately locate robust and corresponding rigid suspended tubular structures. This effectively avoids meaningless braking behavior caused by misjudging flexible and slender objects such as thin ropes and antennas as suspended obstacles. While ensuring safety protection performance, it also ensures the driving continuity of mobile devices in complex reversing scenarios.

[0066] Optionally, based on any of the above embodiments or optional methods, the present disclosure embodiments may also employ the following methods, but not limited thereto, including: identifying obstacles with block-shaped structures based on the perception information; and determining that the suspended obstacle exists within the target driving area when the size information of the block-shaped obstacle does not exceed the size information of the mobile device and the ground clearance information is within the target height range.

[0067] In this embodiment, obstacles with blocky shape information refer to physical entities with obvious blocky features, such as cuboids, cubes, boxes, or irregular polyhedra, which are spatially clustered by perceived information (such as 3D point cloud data). Size information refers to the physical length values ​​of the obstacle in the three dimensions of length, width, and height in the spatial coordinate system. The size information of the mobile device refers to the physical values ​​of the maximum length, width, and height of the mobile device itself in three-dimensional space (i.e., the actual length, width, and height of the entire mobile device). The size information of the blocky obstacle does not exceed the judgment condition of the size information of the mobile device, which aims to exclude large structural suspended obstacles (such as giant bridge beams or elevated corridors spanning the road) that are much larger than the mobile device in the environment from the scope of suspended obstacle identification. This is because once the size of the blocky obstacle (especially the length and width in the horizontal direction) exceeds the size of the mobile device itself, such obstacles are usually three-dimensional spatial structures that the mobile device can pass through normally. Even if its ground clearance meets the conditions, it will not physically interfere with the vehicle body of the mobile device, and therefore does not have a real collision risk that triggers braking protection.

[0068] As one feasible approach, during identification, point cloud data from the perceived information is first used to extract obstacles with block-like structures, and the actual length, width, and height of these obstacles are calculated. Then, these dimensions are compared item by item with the pre-configured dimensions of the mobile device. If the length of the obstacle is less than or equal to the length of the mobile device, the width is less than or equal to the width of the mobile device, and the height is less than or equal to the height of the mobile device, then the dimensions of the obstacle are determined to be within the dimensions of the mobile device. If this condition and the ground clearance are both within the target height range (e.g., between 20 cm and 220 cm), then a suspended obstacle is determined to exist within the target driving area. For example, the mobile device has dimensions of 4.8 meters in length, 1.9 meters in width, and 1.7 meters in height; if a fire extinguisher box (e.g., 0.5 meters long, 0.4 meters wide, and 0.6 meters high) is detected hanging on the wall... Figure 2 As shown in the red area, since all its dimensions are smaller than the external dimensions of the mobile device and its height above the ground is 80 centimeters, which meets the target height range, the system determines it to be a suspended obstacle and prepares to perform a braking operation.

[0069] By using the above method, the actual length, width, and height of the mobile device itself are used as the comparison benchmark to clarify the physical size judgment boundary of block obstacles, accurately filter out macroscopic structural interference objects that exceed the size of the mobile device, avoid frequent triggering of invalid emergency braking due to misdetection of large suspended structures in the environment, and ensure effective protection against real collision risks while taking into account the normal passage efficiency of mobile devices in complex scenarios.

[0070] Optionally, based on any of the above embodiments or optional methods, the present disclosure embodiments may also employ the following methods, but not limited thereto, including: acquiring the three-dimensional contour shape of the obstacle in the perception information; wherein the contour shape includes the three-dimensional contour shape; and / or acquiring the size information of the obstacle in the perception information; and determining that the irregular obstacle exists in the target driving area when it is determined that the three-dimensional contour shape matches the target contour shape, and / or the size information of the obstacle does not exceed the target size information.

[0071] In this embodiment of the disclosure, the three-dimensional contour morphology refers to the geometric shape characteristics of the obstacle in three-dimensional space in the perceived information, such as cylindrical, cuboid, conical, wall-shaped, etc.; the target contour morphology is a pre-constructed set of standard three-dimensional feature templates for irregular obstacles, which includes reference contours such as columnar obstacles (approximately cylindrical), wall-shaped obstacles, stone block obstacles, etc.; the size information refers to the physical measurement values ​​of the obstacle in the spatial coordinate system, such as length, width, and height; the target size information is the preset physical dimension range of various irregular obstacles.

[0072] As one feasible approach, after acquiring point cloud clustering data of obstacles from the perceived information, the matching degree between the actual obstacle's 3D contour shape and the target contour shape is calculated. When the matching degree is greater than or equal to a preset matching degree threshold, the 3D contour shape is determined to match the target contour shape. Simultaneously, the obstacle's size information can be obtained through a 3D bounding box extraction algorithm, and the obstacle's length, width, and height values ​​are compared with the target size information range. If the obstacle's length, width, and height values ​​all fall within the target size information range, the obstacle's size information is determined to not exceed the target size information. In practical application scenarios… Figure 4 The cylindrical obstacle shown can be identified as an irregular obstacle if its cylindrical three-dimensional outline can match the preset cylindrical target outline. Figure 3 Although the three-dimensional outline of the stone pier obstacle shown may not match the preset outline well, its size information falls within the target size information range, and it can still be identified as an irregular obstacle in the target driving area.

[0073] Through the above methods, a method for identifying irregular obstacles based on three-dimensional contour matching and size information constraints was constructed. Three-dimensional contour matching can identify obstacles with significant shape features, while size information constraints can identify obstacles with irregular shapes but whose physical dimensions meet the expected characteristics. The combination of the two judgment conditions can ensure accurate identification of various irregular obstacles such as utility poles, streetlights, trees, cement walls, load-bearing columns, and stone piers in complex perception environments.

[0074] In this embodiment of the disclosure, the existence of a collision risk is assessed based on the relative position between the mobile device and the target obstacle and / or the motion state of the mobile device.

[0075] When assessing collision risk based on the relative position and / or motion state between a target obstacle and a mobile device, it is first necessary to obtain the real-time spatial position information of the target obstacle relative to the mobile device, including but not limited to geometric parameters such as the straight-line distance, azimuth angle, and height difference between the target obstacle and the mobile device. Simultaneously, the relative motion state between the target obstacle and the mobile device should be obtained, such as dynamic parameters like relative velocity, relative acceleration, and direction of motion. Based on this multi-dimensional spatial and motion information, a quantitative index characterizing the probability of collision is determined. This quantitative index can comprehensively consider the size characteristics and motion trend of the target obstacle, as well as the motion state of the mobile device itself.

[0076] In some embodiments, collision risk assessment can be based on the prediction of the relative motion trajectory between the target obstacle and the mobile device, combined with the spatial overlap relationship between the geometric contour of the target obstacle and the driving path of the mobile device, to determine the degree of spatial overlap in which a collision occurs.

[0077] As one implementation method, for different types of target obstacles such as suspended pipes, boxes (e.g., fire boxes), cement walls, cement blocks, and circular columnar targets, the risk assessment results can be dynamically adjusted according to parameters such as their size range, height above the ground, and degree of overlap with the mobile device, thereby achieving accurate quantification of collision risks for various obstacles.

[0078] By using the relative position between the mobile device and the target obstacle and / or the motion state of the mobile device to conduct collision risk assessment, the accuracy of identifying collision risks of various target obstacles and the timeliness of response can be effectively improved, avoiding misjudgment or omission due to insufficient information in a single dimension, thereby providing a reliable basis for subsequent emergency braking decisions.

[0079] In this embodiment of the disclosure, when the collision risk is determined to exist, the mobile device is controlled to perform the braking operation.

[0080] Optionally, based on any of the above embodiments or optional methods, the embodiments of this disclosure may also employ the following methods, but are not limited thereto: when assessing the existence of a collision risk based on the relative position between the mobile device and the target obstacle and / or the motion state of the mobile device, the following methods may be used, but are not limited to: Figure 7 As shown, it includes: Step 301: Determine the degree of overlap and spacing information between the target obstacle and the mobile device.

[0081] Based on the geometric dimensions and spatial coordinates of the target obstacle identified by perception information, as well as the real-time pose data of the mobile device itself (including the width of the mobile device, the current driving direction and speed), the degree of overlap between the target obstacle and the moving device (also known as the lateral overlap rate) is calculated.

[0082] In some embodiments, the lateral overlap rate is: the percentage of the vehicle width relative to the overlap length between the projection of the target obstacle onto the width of the mobile device and the width of the mobile device on a plane perpendicular to the direction of travel of the mobile device, such as... Figure 8 As shown.

[0083] For example, when the target obstacle is a suspended pipe or a circular cylindrical target, the overlap length is calculated based on its diameter or cross-sectional dimensions and its lateral offset relative to the centerline of the mobile device, thus obtaining the lateral overlap rate. Simultaneously, the distance information between the target obstacle and the mobile device (also known as the longitudinal distance) is calculated. This longitudinal distance refers to the straight-line distance between the nearest point of the target obstacle and the front or rear end of the mobile device (determined according to the travel direction) along the direction of travel of the mobile device. In one possible implementation, the longitudinal distance is calculated based on the nearest point distance between the three-dimensional bounding box of the target obstacle and the vehicle bounding box.

[0084] In some embodiments, when determining the degree of overlap and spacing information between the target obstacle and the mobile device, the information can be obtained by fusion processing based on information collected by at least one optical detection sensing module, at least one image acquisition module, at least one microwave detection module, and at least one wireless positioning module.

[0085] Step 302: If the degree of overlap is greater than or equal to a first safety threshold and the spacing information is less than or equal to a second safety threshold, then the collision risk between the mobile device and the target obstacle is determined.

[0086] The calculated lateral overlap rate is compared with a first safety threshold for overlap (e.g., 20%), and the longitudinal distance is compared with a second safety threshold for spacing information. The second safety threshold for spacing information can be dynamically determined based on the mobile device's current speed, braking performance, and safety margin. Specifically, the first safety threshold for overlap can be 15%, 30%, etc., in addition to 20%, and the second safety threshold for spacing information includes, but is not limited to, 0.5 meters, 0.8 meters, 1 meter, etc.

[0087] When the lateral overlap rate is greater than or equal to the first safety threshold and the longitudinal distance is less than or equal to the second safety threshold, a collision risk is determined, triggering subsequent automatic emergency braking. This process accurately quantifies the collision risk between the target obstacle and the mobile device, avoiding misjudgments or omissions caused by a single-dimensional assessment.

[0088] By simultaneously introducing quantitative indicators of two dimensions—overlap degree and / or spacing information (i.e., lateral overlap rate and longitudinal distance)—and combining them with the first and second safety thresholds for joint judgment, the accuracy of collision risk assessment is improved. This is especially suitable for collision avoidance scenarios involving non-standard shaped targets such as suspended obstacles and columnar obstacles, effectively reducing the false trigger rate while ensuring timely response in dangerous conditions.

[0089] Based on any of the above embodiments or optional methods, the embodiments of this disclosure may also employ the following methods, but are not limited thereto. When performing the acquisition of perception information of the target driving area of ​​the mobile device based on a multi-source environmental perception device, the following methods may be employed, but are not limited to: Figure 9 As shown, it includes: Step 401: Collect raw perception information from multiple angles within the target driving area using the multi-source environmental perception devices corresponding to the head perception area, the side perception area, and / or the tail perception area.

[0090] In this embodiment of the present disclosure, the head sensing area, side sensing area and tail sensing area of ​​the mobile device are all equipped with a multi-source environmental sensing device consisting of an optical detection sensing module, an image acquisition module, a microwave detection module and a wireless positioning module. The sensing devices in each sensing area are arranged differently according to the coverage scene, so as to realize the full circumference and no blind spots of the mobile device for environmental data collection.

[0091] During the data acquisition process, the multi-source environmental sensing devices corresponding to each area operate synchronously. They collect three-dimensional point cloud contour data of each area through lidar, collect real-time two-dimensional image semantic data of each area through visual sensors, collect target distance, relative speed and azimuth dynamic data through millimeter-wave radar around the clock, and calibrate near-range relative position and precise spacing data through ultra-wideband wireless positioning modules. This allows for the batch acquisition of raw sensing information covering the head, sides and tail of the equipment in all directions, ensuring the dimensional diversity and spatial integrity of the sensing data.

[0092] Specifically, the multi-source environmental sensing device includes at least an optical detection sensing module; as an achievable method, the identification of suspended obstacles can be realized using only the optical detection sensing module; the optical detection sensing module is a lidar, which generates three-dimensional point cloud data of the target area by emitting a laser beam towards the target area and receiving the reflected laser signal; the three-dimensional point cloud data can reflect the measured distance values ​​between each position in the target area and the lidar; when the lidar continuously scans the target area, if the three-dimensional point cloud shows a clustering feature that presents a specific contour shape in space and has a clear gap with the ground, the existence of a suspended obstacle can be determined based on this clustering feature and spatial gap information.

[0093] Based on this, mobile devices can extract the ground clearance and horizontal extension information of suspended obstacles. Taking the identification of suspended obstacles using only LiDAR as an example, when the ground clearance information of the suspended obstacle is within the target height range and the horizontal extension information is within the target extension condition, it can be determined that there is a target obstacle in the target driving area, and subsequent collision risk assessment and braking operation can be triggered.

[0094] For example, if there is a suspended pipe with a diameter of 150mm and a height of 800mm above the ground in the target direction (e.g., behind) of a mobile device (such as a vehicle), when only an optical detection sensing module (LiDAR) is used for detection, when the LiDAR scans the clustered outline of the tubular structure in the target direction of the mobile device, the diameter of the tubular structure can be further calculated to be 150mm and its height above the ground to be 800mm using three-dimensional point cloud data; the control unit of the mobile device obtains the pre-configured target pipe diameter (e.g., 10cm) and target height range (e.g., 20cm to 220cm); after physical size comparison, it is determined that the pipe diameter (150mm) of the tubular structure is larger than the target pipe diameter (100mm), and the height above the ground information (800mm) is within the target height range (200mm to 2200mm). Therefore, the tubular structure is determined to be a suspended obstacle, and subsequent collision risk assessment and braking operation are triggered.

[0095] As another possible approach, the multi-source environmental sensing device can also include one or more of the following: an optical detection sensing module, an image acquisition module, a microwave detection module, and a wireless positioning module, to achieve a more robust detection effect on suspended obstacles and irregularly shaped obstacles.

[0096] Specifically, the optical detection sensing module is a lidar, which can collect three-dimensional point cloud data corresponding to the detection points in the target area in real time, accurately acquiring the contour shape, three-dimensional spatial coordinates, and radial distance information relative to the mobile device of suspended or irregularly shaped obstacles; the image acquisition module is a vision sensor, which can continuously acquire two-dimensional image frames of the environment around the mobile device, and complete the classification and identification, contour edge detection, and pixel region positioning of suspended or irregularly shaped obstacles through image recognition, feature extraction, and other methods, clarifying the three-dimensional contour shape, relative orientation, and visual occlusion and overlapping areas of the obstacles, providing two-dimensional image verification for the position of the obstacles in the three-dimensional point cloud data; the microwave detection module is a millimeter-wave radar, which collects the relative distance, relative speed, and orientation angle information of suspended or irregularly shaped obstacles, is not affected by adverse environmental factors such as light, rain, snow, and sandstorms, and can output dynamic distance change data between the suspended obstacle and the mobile device in real time, while also assisting in sensing the spatial distribution of obstacles in the area around the mobile device, compensating for the deficiency of the optical detection sensing module in sensing failure under adverse conditions; the wireless positioning module is an ultra-wideband (Ultra) radar. Wideband (UWB) positioning modules can achieve high-precision short-range ranging and relative position positioning through ultra-wideband signals. They accurately output the absolute distance and relative position deviation data between suspended or irregularly shaped obstacles and mobile devices, providing core data support for short-range parameter calibration when the distance between the mobile device and the obstacle reaches the warning distance or safe distance, effectively improving the detection accuracy in short-range working conditions.

[0097] Step 402: The original perception information from multiple regions is fused to obtain the perception information of the target driving area.

[0098] After the data acquisition of multiple modules is completed, the original perception information of multiple regions is first processed for time synchronization and spatial registration to unify the timestamps and spatial coordinate systems of each sensor data and eliminate data deviations caused by differences in the acquisition frequency and installation location of different sensors. Then, the three-dimensional point cloud data, two-dimensional image data, microwave detection data, and ultra-wideband positioning data are fused and analyzed in multiple dimensions. The advantageous data features of each module are combined to extract the perception information that can characterize the target driving area.

[0099] For example, consider a suspended pipe with a diameter of 150mm, a height of 800mm above the ground, and a horizontal extension of 150mm in the target direction of the mobile device: A multi-source environmental sensing device (such as lidar) first detects clustered targets with a tubular outline, generating a 3D point cloud outline of the suspended pipe; simultaneously, a visual sensor acquires an image of the suspended pipe, confirming it as a tubular suspended obstacle through image recognition; millimeter-wave radar verifies the relative distance and relative speed between the mobile device and the suspended pipe; an ultra-wideband (UWB) positioning module calibrates the near-field distance accuracy between the mobile device and the suspended pipe; and the control unit, based on... The fused information on the size of the suspended pipe (e.g., pipe diameter of 150mm), the height above ground (e.g., 800mm), and the horizontal extension (e.g., 150mm) is compared with the preset target pipe diameter (e.g., 100mm), target height range (e.g., 200mm to 2200mm), and target extension conditions (e.g., horizontal extension greater than or equal to 100mm). If it is determined that the pipe diameter is greater than the target pipe diameter, the height above ground is within the target height range, and the horizontal extension is within the target extension conditions, it is determined that there is a suspended obstacle in the target driving area, and subsequent collision risk assessment and braking operation are triggered.

[0100] By cross-verifying and complementaryly correcting multi-source data, the perceived information of the target driving area can be accurately obtained, effectively solving the problems of single sensor perception dimension, susceptibility to environmental interference, and limited detection accuracy. This significantly improves the comprehensiveness, accuracy, and environmental adaptability of obstacle spatial state detection, providing reliable data support for subsequent mobile device braking operations.

[0101] In other embodiments, taking a vehicle as an example, the mobile device is equipped with one forward-facing main LiDAR and three blind-spot LiDARs. The three blind-spot LiDARs are respectively positioned on the left front, right front, and rear sides of the vehicle to cover a wide area in front of, to the sides, and behind the vehicle. During the data acquisition process, the four LiDARs operate simultaneously in a synchronous triggering mode, each independently acquiring raw point cloud data within its field of view. Each raw point cloud data contains the three-dimensional spatial coordinates and reflection intensity information of each reflection point. The four raw point cloud data are then spatiotemporally aligned and fused. The raw point cloud data are merged into a complete point cloud set, and finally, perception information is generated based on the fused global point cloud data. This perception information contains three-dimensional information of all obstacles within a 360-degree range around the vehicle, providing a complete environmental perception foundation for target obstacle identification and collision risk assessment.

[0102] The embodiments disclosed herein can utilize the complementary fields of view of four lidars to eliminate the blind spots of a single lidar, improve the detection coverage and data integrity of target obstacles such as suspended obstacles, columnar obstacles and wall-like obstacles, thereby providing more reliable environmental perception input for subsequent collision risk assessment and braking control, and effectively enhancing the performance of active safety.

[0103] In some embodiments, linked prompt information is output on the interactive interface of the mobile device; wherein the linked prompt information is synchronously associated with the collision risk status and braking execution status. The linked prompt information includes at least one of text prompts, audio prompts, haptic feedback prompts, and visual highlighting prompts.

[0104] The linked prompt information is presented visually in the interactive interface to inform the user of the current driving status of the mobile device; the linked prompt information is used to inform the user that the mobile device is at risk of collision and that braking operation has been performed; the content of the linked prompt information may include at least one of the following: the location information of the collision risk, the type information of the braking operation, and the execution status information of the braking operation.

[0105] In some embodiments, text prompts refer to prompts presented in text form; text prompts can be displayed on the interactive interface of a mobile device to show users relevant information about collision risks; audio prompts refer to prompts presented in sound form; audio prompts can be output through the speaker of a mobile device, such as outputting voice broadcast content or prompt sound effects; haptic feedback prompts refer to prompts presented in tactile form; haptic feedback prompts can be provided through vibration devices on a mobile device, such as prompting the user through seat vibration or steering wheel vibration; visual highlight prompts refer to prompts presented in visual highlight form; visual highlight prompts can be presented in a highlighted manner on the interactive interface of a mobile device, such as highlighting important content in the prompt information with a highlight color or flashing icon.

[0106] As an optional approach, the text prompts, audio prompts, haptic feedback prompts, and visual highlighting prompts corresponding to the linked prompts can be used individually or in combination. When the interactive interface of the mobile device is unavailable, only audio prompts or haptic feedback prompts can be output. When the user is in a scenario where it is inconvenient to look at the screen, audio prompts or haptic feedback prompts can be used first. Different types of prompts can be conveyed to the user from different sensory channels, so that the user can receive effective reminders in a variety of usage scenarios.

[0107] Figure 10 This is a schematic diagram of an interactive interface provided in an embodiment of the present disclosure, such as... Figure 10As shown, the mobile device's interactive interface can be used to present multi-dimensional images and prompts. The interface is divided into multiple display areas, including a rear image area, a ground image area, and a side image area. The rear image area is located in the main part of the interface and is used to present real-time environmental images behind the mobile device. The ground image area is located in the lower left of the rear image area and is used to present road images behind the mobile device. The side image area is located in the lower right of the rear image area and is used to present side environmental images behind the mobile device. The interface displays real-time images in the target area (e.g., the rear image area of ​​the interface) and displays braking status indicators and protection strategy icons in a floating window at the edge or overlay position of the target area, with the floating window covering part of the real-time image. The braking status indicator graphically indicates whether the mobile device is currently performing a braking action. The protection strategy icon graphically indicates the specific type of protection operation currently being performed by the mobile device. The interface also displays wheel position information at the bottom, including icons for the front wheels and rear wheels. The icons for the front and rear wheels are referenced to the positions of the ground image area and the side image area, allowing the user to... The system senses the relative position of the mobile device to its surrounding environment. In the real-time image displayed in the rear image area, the position of target obstacles (suspended or irregularly shaped obstacles) is highlighted visually to alert the user to the physical location of the collision risk. For example, the boundary position and range of the target obstacle in the real-time image are marked using at least one of the following: a red border, a highlighted line, or a semi-transparent color overlay layer. When a collision risk is determined between the mobile device and the target obstacle, and the mobile device is controlled to perform a braking operation, a prompt message is displayed on the interactive interface, such as "Emergency Braking in Progress" and "Collision Risk." This prompt message informs the user that the mobile device is currently braking, and the reason for the braking is the risk of a collision with a suspended or irregularly shaped obstacle behind it. The interactive interface simultaneously displays prompt messages, multi-view environmental images, braking status indicators, illustrations of protection strategies, visually highlighted marks, and wheel position information, enabling the user to simultaneously perceive the current braking status of the mobile device, the physical location of the target obstacle, the relative distance between the mobile device and the target obstacle, and the real-time situation of the surrounding environment during the braking operation of the mobile device.

[0108] As an optional approach, a floating window is displayed at the edge of the target area in a transparent or semi-transparent style to reduce the obstruction of the real-time rear image. The braking status indicator can distinguish the protection execution status of the mobile device through color changes or pattern switching. For example, when a collision risk is identified but braking has not yet been performed, the braking status indicator displays a first color or pattern; when braking has been performed and the device is currently braking, the braking status indicator switches to a second color or pattern. The graphic indicator of the protection strategy can be dynamically displayed according to changes in physical parameters during the braking operation. For example, during the braking action of the mobile device, the graphic indicator of the protection strategy displays the current braking force or progress using a progress bar or dynamic animation. The distance information between the rear wheel graphic indicator and the edge of the target obstacle can be displayed as a text value superimposed on the rear image area, or, when the distance between the rear wheel graphic indicator and the edge of the target obstacle changes, the color, transparency, coverage, and display position of the visually highlighted marker are dynamically adjusted synchronously. The graphic indicators of the front and rear wheels... The display position of the markers can also be physically linked to the real-time position of the actual mobile device's wheels. When the distance between the mobile device and the edge of the target obstacle reaches the warning distance, the rear wheel icon or the visually highlighted mark of the target obstacle will show a first visual state change. When the distance between the mobile device and the edge of the target obstacle reaches the target distance (i.e., the physical distance to trigger emergency braking), the rear wheel icon or the visually highlighted mark of the target obstacle will show a second visual state change, so as to more intuitively reflect the different levels of collision risk to the user. By setting different display areas in the interactive interface to present real-time images, braking status markers, protection strategy icon markers, visually highlighted marks, and wheel position information in multiple dimensions, users can understand the physical position and range of the target obstacle, the relative distance between the mobile device and the edge of the target obstacle, and the current braking status and braking progress of the mobile device through the rear image and ground image of the interactive interface during the braking operation of the mobile device. This improves the perceptibility of the prompt information, the scene understanding, and the intuitiveness of human-computer interaction.

[0109] In some embodiments, controlling the mobile device to perform a braking operation includes: When the mobile device is traveling in reverse, its speed is within the target speed range, and there is a risk of collision in the area behind it, the mobile device is controlled to perform a rearward braking operation; and / or, When the mobile device is traveling laterally, the travel speed is within the target vehicle speed range, and there is a risk of collision in the lateral area, the mobile device is controlled to perform lateral braking.

[0110] In this embodiment of the disclosure, when a collision risk with a target obstacle is determined, the braking control unit first acquires the current driving status information of the mobile device, including the mobile device's driving direction and current speed. The driving direction is determined by combining the mobile device's gear position signal and steering angle sensor data, while the vehicle speed is acquired through wheel speed sensors. Based on the acquired driving direction and speed data, the braking control unit determines the specific operating condition of the mobile device.

[0111] In some embodiments, when the determination result indicates that the mobile device is in a rearward driving state and the current vehicle speed is within a target speed range (the target speed range is a low speed range, for example: 1 km / h to 10 km / h), the braking control unit generates and outputs a rearward low-speed braking command. This command is sent to the electronic stability control system or braking actuator of the mobile device to control the mobile device to perform a rearward braking operation, thereby effectively avoiding collisions with identified target obstacles during reversing.

[0112] In other embodiments, when the determination indicates that the mobile device is in a lateral turning state, such as when the mobile device is performing a reverse turn or a forward turn, and the current vehicle speed is also within the low speed range of 1 km / h to 10 km / h, the braking control unit generates and outputs a lateral braking command. This command is also sent to the braking actuator to control the mobile device to perform an automatic lateral emergency braking operation, thereby effectively avoiding collisions with lateral or rearward-facing target obstacles during the turn. These two braking operations ensure timely and precise braking intervention for different types of target obstacles in typical high-risk scenarios involving low-speed driving.

[0113] By distinguishing between two low-speed operating conditions—reverse driving and lateral turning—and setting corresponding braking strategies for each, precise control of collision risks for mobile devices in low-speed reversing and turning scenarios is achieved. This improves the active safety performance of mobile devices in complex low-speed environments and effectively reduces the probability of collisions with targets and obstacles such as concrete walls, concrete blocks, circular cylindrical targets, and suspended obstacles.

[0114] In some embodiments, the target obstacles identified in this disclosure include, but are not limited to, suspended obstacles and irregularly shaped obstacles. Irregularly shaped obstacles include, but are not limited to, columnar obstacles, wall-shaped obstacles, and stone pier obstacles. For the identification of suspended obstacles, the height of the lower edge and the horizontal projection size of the obstacle are extracted to obtain the coordinates of the lowest point in the vertical direction, thereby determining the height of the lower edge of the target obstacle from the ground. Simultaneously, the obstacle is projected onto a horizontal plane, and its minimum circumscribed circle diameter is calculated. When the lower edge height is between 20 cm and 220 cm, and the minimum circumscribed circle diameter of the horizontal projection is greater than or equal to 10 cm, the obstacle is determined to be a suspended obstacle. Suspended obstacles include suspended fire extinguisher boxes, suspended pipes, air conditioner outdoor units, etc. For example, the dimensions of a suspended fire extinguisher box are more than 650 mm in length, more than 550 mm in width, and more than 240 mm in height, and the typical diameter of a suspended pipe is more than 10 cm.

[0115] For identifying columnar obstacles, the vertical projection diameter and height of the obstacle are extracted. The obstacle is projected onto a horizontal plane, and its smallest circumscribed circle diameter is calculated as the vertical projection diameter. Simultaneously, the difference between the highest and lowest points of the obstacle in the vertical direction is calculated as the height. When the vertical projection diameter is greater than or equal to 100 mm and the height is greater than or equal to 500 mm, the obstacle is determined to be a columnar obstacle. Columnar obstacles include utility poles, streetlights, trees, etc., with typical diameters of 100 mm or more and heights of 500 mm or more.

[0116] For identifying wall-like obstacles, the side length and height of the obstacle are extracted. The minimum bounding rectangle of the obstacle on the horizontal plane is calculated, and its minimum side length is obtained. Simultaneously, the height of the obstacle in the vertical direction is calculated. If the minimum side length is greater than or equal to 100 mm and the height is greater than or equal to 500 mm, the obstacle is determined to be a wall-like obstacle. Wall-like obstacles include concrete walls, square columns, and concrete blocks, among which a typical concrete block has dimensions of 1000 mm in length, 300 mm in width, and 500 mm in height.

[0117] It should be noted that the above size information regarding target obstacles—suspended obstacles, columnar obstacles, and wall-like obstacles—is merely an exemplary example and not a specific limitation. The size information of suspended obstacles, columnar obstacles, and wall-like obstacles in this embodiment of the disclosure is not limited.

[0118] Corresponding to the control method described above, the present invention also proposes a control device. Since the device embodiments of the present invention correspond to the method embodiments described above, details not disclosed in the device embodiments can be referred to in the method embodiments described above, and will not be repeated here.

[0119] Figure 11 This is a schematic diagram of the structure of a control device provided in an embodiment of the present disclosure, such as... Figure 11 As shown, it includes: The acquisition unit 21 is used to acquire perception information of the target driving area of ​​the mobile device based on the multi-source environmental perception device; The determining unit 22 is used to identify target obstacles within the target driving area based on the perceived information; wherein the target obstacles include at least one of: suspended obstacles and irregularly shaped obstacles; The control unit 23 is configured to control the mobile device to perform a braking operation when it is determined that there is a risk of collision between the mobile device and the target obstacle.

[0120] Furthermore, in one possible implementation of the embodiments of this disclosure, such as Figure 12 As shown, the determining unit 22 is further configured to: The target obstacle is identified based on the outline and / or shape information of the obstacle in the perceived information.

[0121] Furthermore, in one possible implementation of the embodiments of this disclosure, such as Figure 12 As shown, the determining unit 22 is further configured to: Determine the height of the obstacle above the ground in the perceived information; If the ground clearance is within the target height range, it is determined that the target obstacle exists within the target driving area.

[0122] Furthermore, in one possible implementation of the embodiments of this disclosure, such as Figure 12 As shown, the determining unit 22 is further configured to: Determine the ground clearance information of the obstacle and the horizontal extension information of the obstacle from the perceived information; If the ground clearance information is within the target height range and the horizontal extension information is within the target extension condition, it is determined that the target obstacle exists within the target driving area.

[0123] Furthermore, in one possible implementation of the embodiments of this disclosure, such as Figure 12 As shown, the determining unit 22 is further configured to: Based on the perceived information, obstacles with tubular outlines are identified. If the diameter of the tubular obstacle is greater than the target diameter and the ground clearance is within the target height range, then the suspended obstacle is determined to exist within the target driving area.

[0124] Furthermore, in one possible implementation of the embodiments of this disclosure, such as Figure 12 As shown, the determining unit 22 is further configured to: Based on the perceived information, obstacles with block-like shapes are identified. If the size of the block-shaped obstacle does not exceed the size of the mobile device and its ground clearance is within the target height range, then the suspended obstacle is determined to exist within the target driving area.

[0125] Furthermore, in one possible implementation of the embodiments of this disclosure, such as Figure 12 As shown, the determining unit 22 is further configured to: Obtain the three-dimensional contour shape of the obstacle in the perceived information; wherein, the contour shape includes the three-dimensional contour shape; and / or, Obtain the size information of the obstacles from the perceived information; If the three-dimensional contour shape matches the target contour shape, and / or the size information of the obstacle does not exceed the target size information, it is determined that the irregular obstacle exists in the target driving area.

[0126] Furthermore, in one possible implementation of the embodiments of this disclosure, such as Figure 12 As shown, the control unit 23 is further configured to: Determine the degree of overlap and spacing information between the target obstacle and the mobile device; If the degree of overlap is greater than or equal to a first safety threshold and the spacing information is less than or equal to a second safety threshold, then a collision risk is determined to exist between the mobile device and the target obstacle.

[0127] Furthermore, in one possible implementation of the embodiments of this disclosure, such as Figure 12 As shown, the multi-source environmental sensing device is deployed in at least one of the head sensing area, side sensing area, and tail sensing area of ​​the mobile device. The acquisition unit 21 is further configured to: The original sensing information within the target driving area is collected from multiple angles using the multi-source environmental sensing devices corresponding to the head sensing area, the side sensing area, and / or the tail sensing area. The original perception information from multiple regions is fused to obtain the perception information of the target driving area.

[0128] Furthermore, in one possible implementation of the embodiments of this disclosure, such as Figure 12 As shown, the multi-source environmental sensing device includes at least one of the following: an optical detection sensing module, an image acquisition module, a microwave detection module, and a wireless positioning module.

[0129] Furthermore, in one possible implementation of the embodiments of this disclosure, such as Figure 12As shown, the device further includes: The display unit 24 is used to display a prompt message on the interactive interface of the mobile device; wherein the prompt message is used to inform the user that the mobile device has the risk of collision and that a braking operation has been performed.

[0130] It should be noted that the foregoing explanation of the method embodiments also applies to the apparatus of the embodiments of this disclosure, and the principle is the same. Therefore, the embodiments of this disclosure are not limited thereto.

[0131] According to embodiments of this disclosure, this disclosure also provides a mobile device, which includes the control device or electronic device described in the above embodiments.

[0132] In some embodiments, the mobile device includes a vehicle, a mobile smart terminal, a robot, an aircraft, an embodied smart agent, etc. For example, in this embodiment of the disclosure, the mobile device may be a mobile smart terminal, a robot, or an embodied smart agent. The mobile smart terminal may be an intelligent transportation tool. For example, when the intelligent transportation tool is an intelligent vehicle, it includes acquiring perception information of the target driving area of ​​the mobile device based on a multi-source environmental perception device; and controlling the mobile device to perform a braking operation when it is determined based on the perception information that a target obstacle exists and there is a risk of collision with the target obstacle. Through the multi-source environmental perception device, target obstacles can be effectively detected, and when a collision risk with the target obstacle is determined, the braking operation of the mobile device is performed, improving collision avoidance safety performance.

[0133] A robot is an automated device capable of carrying people and / or transporting goods, and able to perform physical tasks such as movement and / or grasping and / or manipulation. An embodied intelligent agent is a physical carrier capable of carrying people and / or transporting goods, and can achieve environmental perception and behavioral interaction, as well as possess an intelligent control system capable of scene recognition, autonomous decision-making, and dynamic task execution. Both robots and embodied intelligent agents can be equipped with the aforementioned control devices.

[0134] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0135] Figure 13 A schematic block diagram of an example electronic device 300 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0136] like Figure 13 As shown, the electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory 302 or a computer program loaded from a storage unit 308 into a random access memory 303. The random access memory 303 may also store various programs and data required for the operation of the electronic device 300. The computing unit 301, the read-only memory 302, and the random access memory 303 are interconnected via a bus 304. An input / output interface 305 is also connected to the bus 304.

[0137] Multiple components in electronic device 300 are connected to input / output interface 305, including: input unit 306, such as keyboard, mouse, etc.; output unit 307, such as various types of monitors, speakers, etc.; storage unit 308, such as disk, optical disk, etc.; and communication unit 309, such as network card, modem, wireless transceiver, etc. Communication unit 309 allows electronic device 300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0138] The computing unit 301 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, CPUs (Central Processing Units), GPUs (Graphics Processing Units), various special-purpose AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSPs (Digital Signal Processors), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above, such as control methods. For example, in some embodiments, the control method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 300 via read-only memory 302 and / or communication unit 309. When the computer program is loaded into random access memory 303 and executed by the computing unit 301, one or more steps of the methods described above may be performed. Alternatively, in other embodiments, the computing unit 301 may be configured to perform the aforementioned control method by any other suitable means (e.g., by means of firmware).

[0139] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application-Specific Standard Products), SOCs (System-on-Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0140] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0141] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0142] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0143] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include LANs (Local Area Networks), WANs (Wide Area Networks), the Internet, and blockchain networks.

[0144] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system that addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0145] It's important to note that artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0146] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0147] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A control method, characterized in that, include: Based on a multi-source environmental perception device, obtain perception information of the target driving area of ​​the mobile device; Based on the perceived information, target obstacles within the target driving area are identified; wherein, the target obstacles include at least one of: suspended obstacles and irregularly shaped obstacles; If a collision risk is determined between the mobile device and the target obstacle, the mobile device is controlled to perform a braking operation.

2. The method according to claim 1, characterized in that, The step of identifying target obstacles within the target driving area based on the perceived information includes: The target obstacle is identified based on the outline and / or shape information of the obstacle in the perceived information.

3. The method according to claim 1 or 2, characterized in that, The step of identifying target obstacles within the target driving area based on the perceived information includes: Determine the height of the obstacle above the ground in the perceived information; If the ground clearance is within the target height range, it is determined that the target obstacle exists within the target driving area.

4. The method according to any one of claims 1-3, characterized in that, The step of identifying target obstacles within the target driving area based on the perceived information includes: Determine the ground clearance information of the obstacle and the horizontal extension information of the obstacle from the perceived information; If the ground clearance information is within the target height range and the horizontal extension information is within the target extension condition, it is determined that the target obstacle exists within the target driving area.

5. The method according to any one of claims 1-4, characterized in that, The determination that the target obstacle exists within the target driving area includes: Based on the perceived information, obstacles with tubular outlines are identified. If the diameter of the tubular obstacle is greater than the target diameter and the ground clearance is within the target height range, then the suspended obstacle is determined to exist within the target driving area.

6. The method according to any one of claims 1-5, characterized in that, The determination that the target obstacle exists within the target driving area includes: Based on the perceived information, obstacles with block-like shapes are identified. If the size of the block-shaped obstacle does not exceed the size of the mobile device and its ground clearance is within the target height range, then the suspended obstacle is determined to exist within the target driving area.

7. The method according to any one of claims 1-6, characterized in that, The step of identifying target obstacles within the target driving area based on the perceived information includes: Obtain the three-dimensional contour shape of the obstacle in the perceived information; wherein, the contour shape includes the three-dimensional contour shape; and / or, Obtain the size information of the obstacles from the perceived information; If the three-dimensional contour shape matches the target contour shape, and / or the size information of the obstacle does not exceed the target size information, it is determined that the irregular obstacle exists in the target driving area.

8. The method according to any one of claims 1-7, characterized in that, Determining that there is a collision risk between the mobile device and the target obstacle includes: Determine the degree of overlap and spacing information between the target obstacle and the mobile device; If the degree of overlap is greater than or equal to a first safety threshold and the spacing information is less than or equal to a second safety threshold, then a collision risk is determined to exist between the mobile device and the target obstacle.

9. The method according to any one of claims 1-8, characterized in that, The multi-source environmental sensing device is deployed in at least one of the head sensing area, side sensing area, and tail sensing area of ​​the mobile device. The method of acquiring perception information of the target driving area of ​​the mobile device based on the multi-source environmental perception device includes: The original sensing information within the target driving area is collected from multiple angles using the multi-source environmental sensing devices corresponding to the head sensing area, the side sensing area, and / or the tail sensing area. The original perception information from multiple regions is fused to obtain the perception information of the target driving area.

10. The method according to any one of claims 1-9, characterized in that, The multi-source environmental sensing device includes at least one of the following: an optical detection sensing module, an image acquisition module, a microwave detection module, and a wireless positioning module.

11. The method according to any one of claims 1-10, characterized in that, The method further includes: A prompt message is displayed on the interactive interface of the mobile device; wherein the prompt message is used to inform the user that the mobile device is at risk of collision and that a braking operation has been performed.

12. A control device, characterized in that, include: The acquisition unit is used to acquire perception information of the target driving area of ​​the mobile device based on the multi-source environmental perception device. A determining unit is configured to identify target obstacles within the target driving area based on the perceived information; wherein the target obstacles include at least one of: suspended obstacles and irregularly shaped obstacles; A control unit is configured to control the mobile device to perform a braking operation when it is determined that there is a risk of collision between the mobile device and the target obstacle.

13. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-11.

14. A mobile device, characterized in that, The mobile device includes the control device as claimed in claim 12; or the electronic device as claimed in claim 13.

15. A vehicle or robot, characterized in that, The vehicle or the robot includes a mobile device as described in claim 14; or a control device as described in claim 12.

Images