A vehicle-mounted panoramic image control method and device and vehicle

Abstract

This application relates to the field of vehicle control, specifically to a control method, device, and vehicle for an in-vehicle panoramic imaging system. By detecting negative obstacles in the vehicle's surrounding environment, this application can promptly identify potential hazards. When a negative obstacle is detected, the in-vehicle panoramic imaging system is activated and marked, allowing the driver to clearly understand the obstacle's location and preventing vehicle damage or accidents due to neglect, thus improving driving safety. The system continuously monitors the display of negative obstacles in the panoramic imaging system and deactivates the image based on whether the obstacle no longer exists or has been confirmed eliminated. This ensures that the panoramic imaging system functions effectively when a hazard exists and can be deactivated promptly after safety, avoiding resource waste.

Description

Technical Field

[0001] This invention relates to the field of vehicle control, and more specifically to a control method, device, and vehicle for in-vehicle panoramic imaging. Background Technology

[0002] A 360-degree panoramic imaging system is a common driver assistance technology that displays a seamless 360-degree image of the vehicle's surroundings on an in-vehicle display screen. This helps drivers observe blind spots and improves driving safety. Current methods for controlling the activation and deactivation of panoramic imaging systems mainly include: automatic activation / deactivation based on turn signals and gear position, manual control via voice or physical buttons, and automatic activation / deactivation based on radar-detected obstacle distance thresholds. All of these methods provide basic assistance functions for typical driving scenarios and forward-facing obstacles.

[0003] However, existing panoramic imaging control logic is not adapted to scenarios with negative road obstacles, and cannot identify, respond to, or provide intelligent warnings for dangerous areas such as potholes, collapses, ditches, and missing manhole covers. In scenarios such as blind spots, nighttime, and insufficient light, drivers find it difficult to intuitively judge the condition of road surface depressions, which can easily lead to safety hazards such as tire damage and chassis scraping. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a control method, device and vehicle for vehicle-mounted panoramic imaging, in order to solve the problem that existing vehicle-mounted 360-degree panoramic imaging can only control conventional scenes and positive obstacles, and cannot identify and adapt to negative obstacles on the road surface.

[0005] In a first aspect, embodiments of the present invention provide a control method for an in-vehicle panoramic image, the method comprising: Detect whether there are negative obstacles in the environment surrounding the vehicle; If there are negative obstacles in the environment around the vehicle, the vehicle's panoramic image is activated, and the negative obstacles are marked in the panoramic image according to their relative positions to the vehicle. The display status of the negative obstacle in the vehicle panoramic image is continuously monitored until the display status includes: the negative obstacle no longer exists in the vehicle panoramic image, and / or the negative obstacle has been confirmed to be eliminated, at which point the vehicle panoramic image is turned off.

[0006] Furthermore, the continuous detection of the negative obstacle's display status in the vehicle-mounted panoramic image includes: Acquire the real-time movement trajectory of the vehicle, and / or detect the shape changes of the negative obstacle; Based on the real-time motion trajectory, the relative position of the negative obstacle in the vehicle panoramic image is continuously updated, and / or, based on the morphological changes, the real-time status of the negative obstacle in the vehicle panoramic image is continuously updated.

[0007] Furthermore, the negative obstacle no longer existing in the vehicle panoramic image includes: the relative position is outside the effective display area of ​​the vehicle panoramic image, or the relative position is in an area blocked by the vehicle itself, so that the negative obstacle cannot be displayed in the vehicle panoramic image. The elimination of the negative obstacle is confirmed to include: the real-time status is that the physical form of the negative obstacle has been repaired and no longer poses a driving threat to the vehicle, or the real-time status is that the negative obstacle has been removed.

[0008] Furthermore, marking the negative obstacle in the vehicle panoramic image according to the relative position of the negative obstacle and the vehicle includes: Detect the obstacle type of the negative obstacle; Based on the preset mapping relationship between obstacle types and marking rules, the target marking rule corresponding to the obstacle type is retrieved. The marking parameters corresponding to different relative positions of the marking rule include at least one of color, line style, and display icon. The target marker parameters that match the relative position are extracted from the target marker rules, and the negative obstacle is differentiatedly marked in the vehicle panoramic image using the target marker parameters.

[0009] Furthermore, after activating the vehicle's in-vehicle panoramic imaging system, the method further includes: Detect the real-time distance between the vehicle and the negative obstacle; The view mode of the in-vehicle panoramic image is adjusted according to the real-time distance. Based on the view mode, the negative obstacles are displayed differently in the corresponding in-vehicle panoramic image.

[0010] Furthermore, adjusting the view mode of the in-vehicle panoramic image based on the real-time distance includes: If the real-time distance is less than a preset distance threshold, the view mode of the vehicle panoramic image will be switched from a front / rear single view to a front / rear vehicle magnified view. If the real-time distance is greater than or equal to a preset distance threshold, the view mode of the vehicle panoramic image will be switched from a magnified view of the front / rear vehicle body to a single view of the front / rear vehicle body.

[0011] Furthermore, before detecting the presence of negative obstacles in the environment surrounding the vehicle, the method further includes: Detect whether the sensing device in the vehicle used to detect negative obstacles is abnormally obstructed; If the sensing device is abnormally obstructed, a first prompt message is output to the user, wherein the first prompt message is used to prompt the user to clear the abnormal obstruction of the sensing device. After outputting the first prompt message, it is detected in real time whether the abnormal occlusion of the sensing device has been cleared; If the abnormal occlusion has been cleared, a recalibration process for the sensing device is triggered, and a second prompt message is output to the user after calibration, wherein the second prompt message is used to indicate that the sensing device is functioning normally.

[0012] Furthermore, the method also includes: When multiple negative obstacles exist in the vehicle panoramic image, the vehicle's driving requirements are obtained; Based on the driving requirements and the relative positions of each of the negative obstacles to the vehicle, a target driving route is planned to avoid all negative obstacles. The target driving route is overlaid on the vehicle panoramic image and displayed separately from the markings of the negative obstacles; Based on the deviation between the vehicle's real-time location and the target driving route, the driving guidance information is updated in real time in the vehicle-mounted panoramic image.

[0013] Secondly, embodiments of the present invention provide a control device for vehicle-mounted panoramic imaging, the device comprising: The detection module is used to detect whether there are negative obstacles in the environment around the vehicle; The activation module is used to activate the vehicle's panoramic image if there is a negative obstacle in the environment around the vehicle, and mark the negative obstacle in the panoramic image according to the relative position of the negative obstacle and the vehicle. The processing module is used to continuously detect the relative position of the negative obstacle in the vehicle panoramic image until the negative obstacle is no longer in the field of view of the vehicle panoramic image, or the negative obstacle has been confirmed to be eliminated, and then turn off the vehicle panoramic image.

[0014] Thirdly, embodiments of the present invention provide a vehicle, including: a sensing device and a controller, wherein the sensing device and the controller are communicatively connected to each other, the sensing device is used to detect negative obstacles in the environment surrounding the vehicle, and the controller stores computer instructions, and the controller executes the computer instructions to perform the method of the first aspect or any corresponding embodiment described above.

[0015] Fourthly, embodiments of the present invention provide a computer device, including: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the method described in the first aspect or any corresponding embodiment thereof.

[0016] Fifthly, embodiments of the present invention provide a computer-readable storage medium storing computer instructions for causing a computer to perform the method described in the first aspect or any corresponding embodiment thereof.

[0017] This application detects negative obstacles in the vehicle's surrounding environment, enabling timely identification of potential hazards. When a negative obstacle is detected, the in-vehicle panoramic imaging system is activated and marked, allowing the driver to clearly understand the location of the obstacle and preventing vehicle damage or accidents due to neglect, thus improving driving safety. The system continuously monitors the display of negative obstacles in the panoramic imaging system and deactivates the imaging system based on the condition that the obstacle no longer exists or has been confirmed to have been eliminated. This ensures that the panoramic imaging system functions effectively when a hazard exists and can be deactivated promptly after safety is achieved, avoiding resource waste. Attached Figure Description

[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the control method for in-vehicle panoramic imaging according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating another method for controlling a vehicle-mounted panoramic image according to an embodiment of the present invention; Figure 3 This is a flowchart illustrating another method for controlling a vehicle-mounted panoramic image according to an embodiment of the present invention; Figure 4 This is a flowchart illustrating another method for controlling a vehicle-mounted panoramic image according to an embodiment of the present invention; Figure 5 This is a structural block diagram of a vehicle-mounted panoramic imaging control device according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] According to embodiments of the present invention, a control method, device, and vehicle for in-vehicle panoramic imaging are provided. It should be noted that the steps shown in the flowcharts in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0022] This embodiment provides a control method for vehicle-mounted panoramic imaging. Figure 1 This is a flowchart of a control method for in-vehicle panoramic imaging according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps: Step S101: Detect whether there are negative obstacles in the environment around the vehicle.

[0023] In this embodiment of the application, the surrounding environment is detected and analyzed in real time by the sensing devices mounted on the vehicle. The sensing devices include a vehicle-mounted panoramic camera, ultrasonic radar, lidar, visual recognition sensor, and image acquisition units arranged around the vehicle body. The above sensing devices work together to obtain image information, distance information, and depth information of the road surface around the vehicle.

[0024] Negative obstacles refer to downward depressions, missing parts, collapses, or trench-like structures on the road surface that pose safety hazards to normal vehicle operation. Examples include potholes, manholes formed by missing manhole covers, collapsed areas of the road surface, trenches formed by construction excavation, ground ditches, and pits caused by road damage.

[0025] During the identification process, the depth, contour, and size of the target area are comprehensively judged by combining the preset negative obstacle feature library, distinguishing between normal road surface areas and dangerous road surface areas, and finally outputting the judgment result of whether there are negative obstacles in the environment around the vehicle.

[0026] Step S102: If there are negative obstacles in the environment around the vehicle, the vehicle's panoramic image is activated, and the negative obstacles are marked in the panoramic image according to their relative positions to the vehicle.

[0027] In this embodiment of the application, when it is determined that there is a negative obstacle in the environment around the vehicle, the command to activate the vehicle panoramic image is triggered, and the real-time image around the vehicle is stitched together and displayed on the vehicle central control screen to provide the driver with a complete view of the surroundings.

[0028] Specifically, negative obstacles are marked in the vehicular panoramic image based on their relative position to the vehicle, including: Step A1: Detect the obstacle type of negative obstacles.

[0029] Obstacle types are the result of classifying these negative obstacles according to their shape, cause, or structural characteristics, such as distinguishing them into types like "pothole type," "collapse type," "well type," or "ditch type," so that differentiated marking and treatment strategies can be adopted for different types in the future. The process is as follows: After detecting a negative obstacle in the vehicle's surrounding environment, the system acquires the obstacle's depth image, point cloud data, or 3D contour information through sensing devices (such as visual sensors, lidar, ultrasonic radar, etc.). Combined with a pre-defined negative obstacle feature library, the system analyzes and matches features such as the obstacle's depression depth, edge shape, opening size, and slope changes to determine the specific type of the obstacle, such as whether it is a shallow pothole or a deep collapse, a regular manhole or a narrow ditch. During the determination process, the system can also correct or refine the type identification results based on the vehicle's current driving scenario (such as low-speed parking, off-road driving, or urban road driving). For example, in off-road scenarios, irregular depressions are classified as "off-road potholes," while in urban roads, regular depressions are classified as "manholes," thus providing a basis for subsequent differentiated labeling and warnings.

[0030] For example, when a vehicle is driving through a construction section at night, the front-mounted lidar detects a depression about 20 centimeters deep with irregular edges on the road surface ahead. Through feature comparison, the depression is identified as matching the characteristics of a "collapsed" obstacle, thus determining that the obstacle type is "road collapse". This provides a basis for marking the obstacle with a thick red solid line and a collapse icon in the vehicle's panoramic image.

[0031] Step A2: Based on the preset mapping relationship between obstacle types and marking rules, retrieve the target marking rule corresponding to the obstacle type. The marking parameters corresponding to different relative positions of the marking rule include at least one of color, line style, and display icon.

[0032] The mapping relationship between preset obstacle types and marking rules refers to the pre-established and stored correspondence. It associates different categories of negative obstacles (such as potholes, collapses, pits, ditches, etc.) with their respective applicable marking rules, so as to quickly locate the corresponding rule according to the specific type identified. The target marking rule is the specific rule retrieved from this mapping relationship that matches the currently detected obstacle type. It defines the specifications for how to mark obstacles of this type. The marking rule itself is a set of data that specifies the marking parameters to be used for a certain type of negative obstacle when it is in different relative positions (such as distance and azimuth) with the vehicle. The marking parameters are the specific visual elements used to render the markings on the vehicle panoramic image. They include at least one of the following: color (e.g., red for danger, yellow for warning), line style (e.g., dashed lines, solid lines, dotted lines for outlining contours or boundaries), and display icon (e.g., pothole graphics, triangular warning symbols, etc.). These parameters together determine the presentation effect of the markings to convey differentiated warning information to the driver.

[0033] Based on the identification of specific types of negative obstacles, a pre-stored mapping table (generated through experimental calibration or machine learning training) is accessed. Using the current obstacle type as an index, the corresponding target labeling rule is retrieved and extracted. This rule may exist in the form of structured data, such as an array containing multiple distance intervals and corresponding labeling parameter lists, or a parameter configuration set dynamically generated by the algorithm. After extraction, the rule is temporarily stored to combine with the real-time measured relative position of the obstacle and the vehicle (e.g., precise distance and orientation measured by radar or vision) to select the most matching labeling parameters from the rules. During this process, the mapping relationship can be dynamically adjusted according to vehicle model, driving mode (e.g., off-road, urban), or user-personalized settings. For example, high-contrast colors and flashing icons can be automatically enabled at night or in inclement weather to improve the warning effect; however, the core logic always remains based on type-based rule retrieval to ensure consistency and relevance of the labeling.

[0034] Step A3: Extract target marker parameters that match the relative position from the target marking rules, and use the target marker parameters to differentiate the negative obstacle in the vehicle panoramic image.

[0035] A 360-degree panoramic view of a vehicle is generated by stitching together images captured by multiple cameras positioned around the vehicle. It is typically displayed on the vehicle's central control screen to assist the driver in observing the surrounding environment. Relative position refers to the orientation and distance of negative obstacles relative to the vehicle, including the straight-line distance, relative angle (e.g., front, back, left, right), and possible three-dimensional coordinates. This information is acquired and updated in real-time by sensing devices. Differentiated marking refers to highlighting negative obstacles on the 360-degree panoramic view using specific visual parameters based on obstacle type, relative position, and other factors, clearly distinguishing them from ordinary road surfaces or other obstacles to convey different warning levels or characteristic information to the driver.

[0036] Based on the retrieved target marking rules corresponding to the current negative obstacle type, the precise relative position between the obstacle and the vehicle is first obtained from real-time measurements by sensing devices (e.g., distance and azimuth calculated from visual ranging or radar data). Then, using this relative position as an index, the target marking parameters that match are searched in the mapping table contained in the target marking rules. This mapping table is usually divided into distance or azimuth intervals, with each interval corresponding to a set of preset parameters such as color, line style, and display icon. If the relative position happens to be on the boundary between two intervals, the parameters can be determined using interpolation algorithms or the nearest principle, or based on the driving mode (e.g., Sport). The system dynamically adjusts parameter selection strategies (either in a standard mode or comfort mode); after extracting the target marker parameters, these parameters are applied to the rendering engine of the in-vehicle panoramic image. A specified color and style outline is drawn at the actual location of the negative obstacle in the image, and a specific icon is filled in. Dynamic effects (such as slow flashing or brightness changes) can be superimposed as needed to achieve differentiated marking of negative obstacles. During the marking process, the system continuously tracks changes in relative position and updates the marker's position and parameters in real time (e.g., the closer the distance, the darker the marker color and the thicker the lines), ensuring that the marker always accurately corresponds to the real obstacle, providing the driver with intuitive and timely risk warnings.

[0037] As an example, if a pothole is detected 1.5m in front of the vehicle, a pre-defined differential marking rule is retrieved based on this type of obstacle. This rule includes different types of negative obstacles such as potholes, collapses, and manholes, with corresponding colors, line styles, and display icons for different relative positions. Based on the retrieved differential marking rule, the marking parameters corresponding to the pothole type and its 1.5m relative position to the vehicle are matched (red color, thick solid line, and concave icon). Based on the matched red, thick solid line, and concave icon marking parameters, the pothole is differentially marked in the vehicle's panoramic image for easy identification by the user.

[0038] By detecting the type of negative obstacles, their specific properties can be identified, providing a classification basis for subsequent targeted marking. Based on the mapping relationship between preset obstacle types and marking rules, the target marking rules corresponding to the obstacle types can be retrieved. These marking rules include marking parameters such as color, line style, and display icon corresponding to different relative positions, enabling precise matching between marking methods and obstacle characteristics and improving the intuitiveness of information transmission. By extracting target marking parameters that match the relative positions from the target marking rules and using these parameters for differentiated marking in the vehicle panoramic image, obstacles can present corresponding visual characteristics at different distances or angles, thereby more clearly reflecting their degree of danger and spatial relationship, and assisting the driver in making accurate judgments.

[0039] Step S103: Continuously monitor the display status of negative obstacles in the vehicle panoramic image until the display status includes: the negative obstacle no longer exists in the vehicle panoramic image, and / or the negative obstacle has been confirmed to have been eliminated, then turn off the vehicle panoramic image.

[0040] In this embodiment of the application, the continuous detection of negative obstacles in the display of the vehicle panoramic image includes: Step B1: Obtain the real-time movement trajectory of the vehicle, and / or detect the shape changes of negative obstacles.

[0041] Specifically, real-time motion trajectory refers to the trajectory information formed by the vehicle's position, speed, acceleration, driving direction, and attitude changes, which are collected and calculated in real time by on-board sensors (such as wheel speed sensors, inertial measurement units, gyroscopes, and global positioning systems) during the vehicle's operation. This information can accurately describe the vehicle's motion state and path relative to the ground. Morphological changes refer to the changes in the physical form of the negative obstacle itself. Specifically, this includes two states: the physical form of the negative obstacle is repaired (such as potholes being filled, manhole covers being reinstalled, and collapses being repaired and leveled) and the negative obstacle is removed (such as temporary pothole structures being cleared or removed). These changes mean that the obstacle no longer poses a threat to the vehicle's driving.

[0042] During the continuous display of the vehicle's panoramic image, multiple sensors work together to continuously collect data such as wheel speed pulses, steering angle, yaw rate, and acceleration. Combined with the vehicle's kinematic model, this data is used to calculate the vehicle's current displacement, heading angle, and attitude changes in real time, thereby generating a continuous real-time motion trajectory. Simultaneously, sensing devices (such as panoramic cameras, LiDAR, and ultrasonic radar) periodically or continuously detect the areas containing identified negative obstacles. The currently collected depth images, point cloud data, or 3D contours of the obstacle areas are compared and analyzed with previously stored feature data to determine the obstacle's edge shape, concavity, and other characteristics. Whether there are significant changes in the depth and outline size of the depression, if the originally depressed area is filled to be basically flush with the surrounding road surface, or the original pothole structure is completely disappeared, the shape change is determined to be "repaired" or "removed". In the process, the detection frequency and comparison accuracy can be dynamically adjusted according to the current scene of the vehicle (such as urban roads or construction sections). For example, the detection sensitivity of shape changes can be increased near construction areas. At the same time, when acquiring real-time motion trajectory, the trajectory can be corrected and predicted by combining high-precision map data, providing an accurate basis for subsequent judgment on whether the obstacle is still within the display range or whether the panoramic image needs to be turned off.

[0043] Step B2: Continuously update the relative position of the negative obstacle in the vehicle panoramic image based on the real-time motion trajectory, and / or continuously update the real-time status of the negative obstacle in the vehicle panoramic image based on the shape change.

[0044] Specifically, after acquiring the vehicle's real-time trajectory and / or the morphological changes of negative obstacles, based on the real-time trajectory data (such as vehicle displacement and heading angle changes), coordinate transformation and projection algorithms are used to continuously calculate and update the latest orientation and distance of the negative obstacles relative to the vehicle, i.e., their relative positions. This updated relative position is then reflected in real-time on the display interface of the in-vehicle panoramic image, ensuring that the marker is always accurately attached to the actual position of the obstacle. Simultaneously, based on detected morphological changes (e.g., determining whether a pothole has been filled or disappeared through image comparison or depth data), the real-time status of the negative obstacles is judged and updated. If the determination... If the status is "repaired" or "removed (in the simulation scene)," the corresponding status information is recorded, and the display method in the vehicle panoramic image is adjusted accordingly, such as removing the marker or changing the marker to the "eliminated" state, to provide a basis for determining whether to turn off the panoramic image. In the implementation process, the update strategy can be optimized for different scenarios. For example, when the vehicle is traveling at high speed, the relative position update frequency can be increased to ensure the real-time performance of the marker, or a multiple confirmation mechanism can be used to avoid misjudgment when the confidence of obstacle shape change detection is low. At the same time, the vehicle attitude information can be combined to compensate for the deviation in relative position calculation caused by vehicle body bumps, so as to ensure the accuracy and stability of the display.

[0045] As an example, when a vehicle slowly drives over a pothole of a certain depth, the real-time motion trajectory calculation shows that the vehicle has moved forward 3 meters, and the position of the pothole relative to the vehicle changes from 1.5 meters in front to 1.5 meters behind. The in-vehicle panoramic image then smoothly moves the pothole marker from the front area of ​​the screen to the rear area. At the same time, the rear-view camera continuously captures images of the pothole area and detects that the pothole has been filled and smoothed with asphalt by construction workers, and the shape change status is "repaired". Therefore, the real-time status is updated to "repaired", and the pothole marker in the panoramic image changes from a flashing red state to a gray static outline, indicating to the driver that the obstacle has been eliminated. After confirming that there are no other obstacles, the command to turn off the panoramic image is triggered.

[0046] By acquiring the vehicle's real-time movement trajectory and / or detecting changes in the shape of negative obstacles, the vehicle's movement trend and obstacle evolution information can be dynamically grasped, providing a data foundation for accurate updates. By continuously updating the relative position of negative obstacles in the vehicle's panoramic image based on the real-time movement trajectory and / or continuously updating the real-time status of negative obstacles in the vehicle's panoramic image based on shape changes, it is ensured that obstacle markers always match their actual positions and reflect the shape evolution of obstacles in real time, thereby improving the accuracy and real-time performance of driver assistance.

[0047] It should be noted that the absence of a negative obstacle in the vehicle panoramic image includes: its relative position being outside the effective display area of ​​the vehicle panoramic image, or its relative position being in an area obscured by the vehicle itself, making the negative obstacle unable to be displayed in the vehicle panoramic image; the elimination of a negative obstacle includes: the physical form of the negative obstacle being repaired and no longer posing a driving threat to the vehicle, or the negative obstacle being removed.

[0048] Specifically, the effective display area refers to the physical space range that the vehicle panoramic image can clearly present. It is usually determined by the camera's field of view, installation position, image stitching algorithm, and preset display boundaries. Objects outside this range will not be displayed in the image. The area obscured by the vehicle itself refers to the negative obstacle that is still located around the vehicle, but is obscured by the vehicle structure, such as the bottom of the vehicle body, the inside of the wheels, or under the bumper, so that the vehicle panoramic image camera cannot directly capture it. The physical form has been repaired. After the negative obstacle has been filled, compacted, or reinstalled, its dent or missing structure has been eliminated, the road surface has been restored to a smooth surface, and it no longer poses a threat to vehicle passage. The negative obstacle has been removed. It refers to the negative obstacle (such as a temporarily placed warning pothole or a shallow pit excavated for construction) that has been cleaned up, moved, or backfilled and has completely disappeared.

[0049] Based on acquiring real-time motion trajectories and shape changes, and continuously updating relative positions and real-time status, a dual determination is made regarding whether negative obstacles still exist in the vehicle-mounted panoramic image and whether they have been eliminated. Firstly, based on the latest relative position calculated from the real-time motion trajectory, combined with preset display boundary parameters (such as a range of several meters in front, behind, to the left, and to the right of the vehicle, and the horizontal viewing angle boundary of the camera), it is determined whether the position exceeds the effective display area. Simultaneously, using the vehicle's 3D model and the obstacle's coordinates, it is determined whether the obstacle is located in an area obscured by the vehicle itself (e.g., by confirming the obstacle is within the chassis projection range through collision detection algorithms). If either of these conditions is met, the negative obstacle is considered to no longer exist in the vehicle-mounted panoramic image. On the other hand, based on real-time status information, if the physical form of a negative obstacle is detected to have been repaired (e.g., image feature comparison confirms the depression depth is below a safety threshold, point cloud data indicates the road surface is smooth) or removed (e.g., a previously existing temporary pothole structure is manually cleared or removed; a temporary pothole structure refers to a non-permanent depression, ditch, or pit formed by construction, excavation, or temporary stacking, such as shallow pits excavated during road construction, temporary warning pit templates, openings left after the removal of temporary ditch covers, or trenches temporarily excavated for drainage), then it is confirmed that the negative obstacle has been eliminated. To prevent false alarms, a confirmation count can be set or a comprehensive verification can be performed in conjunction with vehicle movement status (such as speed and steering). When any condition is consistently met, a command to turn off the vehicle panoramic image is triggered.

[0050] By setting the specific circumstances under which a negative obstacle no longer exists in the vehicle panoramic image system—that is, its relative position is outside the effective display area or is located in an area obscured by the vehicle itself—it can accurately identify that the obstacle has left the field of view, thereby promptly turning off the image to save power and avoid interference. By setting the specific circumstances under which a negative obstacle has been confirmed to have been eliminated—that is, its real-time status is that its physical form has been repaired and does not pose a threat, or that it has been removed—it can reliably confirm that the obstacle has actually been eliminated, ensuring that the auxiliary function is turned off under safe conditions and avoiding the risks caused by misjudgment.

[0051] The method provided in this application utilizes multiple types of vehicle-mounted sensing devices to collaboratively collect road surface depth, contour, and distance information. Combined with a pre-set obstacle feature database, it can accurately identify various negative obstacles such as potholes, collapses, and manholes. Specifically, when a negative obstacle is detected, a panoramic image is automatically activated, and marking rules are matched according to the obstacle type and relative position. Obstacles are intuitively marked with differentiated colors, lines, and icons, clearly distinguishing the level of danger and improving the driver's visual recognition efficiency. Simultaneously, the obstacle marking position is dynamically updated based on the vehicle's real-time movement trajectory, and changes in obstacle shape are monitored to determine whether they have been eliminated. The panoramic image is automatically deactivated after the obstacle moves out of the image range or is physically repaired. This solves the problem of negative obstacles being difficult to detect in blind spots, at night, and in low-light conditions, reducing driving risks such as vehicle chassis scrapes and tire collisions. Furthermore, the intelligent and reasonable image start-stop logic reduces device power consumption and effectively improves vehicle driving safety and the intelligent assistance capabilities of the panoramic image system.

[0052] In this embodiment of the application, after activating the vehicle's in-vehicle panoramic imaging, as follows: Figure 2 As shown, the method also includes: Step S201: Detect the real-time distance between the vehicle and the negative obstacle.

[0053] Specifically, after the in-vehicle panoramic imaging is activated, the vehicle's perception devices (including ultrasonic radar, lidar, and visual sensors) are continuously used to track and measure marked negative obstacles. A multi-sensor fusion algorithm is used to comprehensively acquire the precise location information of the obstacles. Combined with the vehicle's own positioning data (such as GPS and inertial measurement unit) and the vehicle's size model, the shortest straight-line distance or the distance in a specified direction between the vehicle and the obstacle (e.g., the distance from the front bumper to the edge of a pothole) is calculated in real time. During the calculation process, the nearest point can be selected as the ranging benchmark based on the shape and size of the obstacle. The refresh rate and sensor priority of the ranging can be dynamically adjusted for different driving scenarios (such as low-speed parking or normal driving). For example, high-precision ultrasonic radar or lidar data is prioritized at close range, while visual ranging is used for verification at long range to ensure the accuracy and stability of the real-time distance. Simultaneously, the calculated real-time distance is compared with a preset distance threshold (such as D), providing a trigger condition for view mode switching. For example, when a vehicle approaches a pit formed by a missing manhole cover, the front-mounted lidar and ultrasonic radar work together to measure and calculate in real time the straight-line distance between the vehicle's front bumper and the edge of the pit as 1.8 meters. This value is dynamically updated to 1.5 meters, 1.2 meters, and until it is less than a preset threshold D (e.g., 2 meters), triggering a view switch.

[0054] Step S202: Adjust the view mode of the vehicle panoramic image according to the real-time distance.

[0055] In this embodiment of the application, adjusting the view mode of the vehicle-mounted panoramic image according to the real-time distance includes: If the real-time distance is less than the preset distance threshold, the view mode of the vehicle panoramic image will be switched from the front / rear single view to the front / rear vehicle magnified view. If the real-time distance is greater than or equal to the preset distance threshold, the view mode of the vehicle panoramic image will be switched from the front / rear vehicle magnified view to the front / rear single view.

[0056] Specifically, view modes refer to the different display methods provided by the in-vehicle panoramic imaging system to adapt to different driving scenarios. Common ones include front / rear single view and front / rear magnified view. Front / rear single view refers to displaying an image from only a single perspective in front of or behind the vehicle, usually used to observe road conditions at a distance. Front / rear magnified view refers to magnifying the area near the vehicle body based on the single view, so as to observe the relative position and details of the vehicle and obstacles more clearly. The preset distance threshold is a pre-set distance value (such as D) used to judge the proximity of the vehicle to the negative obstacle, as a critical point for switching view modes. This threshold can be calibrated according to the vehicle model, driving mode, or user preference.

[0057] After calculating the distance between the vehicle and the negative obstacle in real time, the system continuously compares this real-time distance with a preset distance threshold. If the real-time distance is less than the threshold, it indicates that the vehicle has entered an area requiring more detailed observation. The current view mode (such as front single view) is switched to a front / rear vehicle magnified view to magnify the details near the obstacle and help the driver more accurately judge the passage conditions. If the real-time distance is greater than or equal to the threshold, it indicates that the vehicle has left the danger zone or has not yet approached it. The view mode is switched back from the vehicle magnified view to the regular front / rear single view to provide a wider field of view. During the switching process, a smooth transition animation (such as zoom gradient) can be added to avoid abrupt changes in the screen. At the same time, auxiliary information such as vehicle speed and steering signals can be combined to optimize the switching timing. For example, when the vehicle is stationary, the magnified view can be maintained to avoid frequent switching, or the switching of the corresponding view mode can be independently controlled for obstacles in different directions (such as front or rear). In addition, the preset distance threshold can be dynamically adjusted according to the obstacle type. For example, a larger threshold can be used for high-risk obstacles such as deep pits to provide early warning, but this adjustment still falls within the scope of distance-based view mode control.

[0058] When the real-time distance is less than a preset distance threshold, the view mode of the vehicle panoramic image is switched from a front / rear single view to a front / rear magnified view of the vehicle body. This provides magnified details when approaching obstacles, assisting the driver in accurately controlling the distance. When the real-time distance is greater than or equal to the preset distance threshold, the view mode is switched from a front / rear magnified view to a front / rear single view. This restores a wide-angle field of view when the vehicle is far from obstacles, ensuring that the driver regains global environmental awareness and achieving intelligent view switching that considers both near and far distances.

[0059] Step S203: Based on the view mode, differentiate the display of negative obstacles in the corresponding vehicle panoramic image.

[0060] Specifically, after switching view modes, the system obtains the currently active view mode (such as the front single view or the front vehicle zoomed-in view) and, combined with the extracted target marker parameters (such as thick red solid lines and recessed icons) and the real-time position of the negative obstacle relative to the vehicle, renders it in the corresponding view. In single view mode, the marker is superimposed on the corresponding location in the image at a normal size (such as the lower area of ​​the front view), and the marker style is kept stable. It can also be used in conjunction with the distance indicator bar at the edge of the view. In vehicle zoomed-in view mode, the outline thickness of the marker is automatically enlarged, the flashing frequency or brightness of the icon is increased, and the real-time distance value (such as "1.2m") is dynamically displayed near the marker to enhance the warning effect. In addition, the display layer of the marker can be dynamically adjusted according to the zoom ratio of the view to avoid the marker from obscuring key road surface details. For example, in the zoomed-in view, the marker is placed on a semi-transparent upper layer while retaining the visibility of the original image of the obstacle. If there are multiple negative obstacles at the same time, the most urgent obstacle marker will be displayed first in the zoomed-in view according to the danger level or distance order, or it will be distinguished by different colors in the single view.

[0061] By detecting the real-time distance between the vehicle and negative obstacles, the degree of proximity can be dynamically obtained, providing a basis for view adjustment; by adjusting the view mode of the in-vehicle panoramic image according to the real-time distance, the most suitable observation angle (such as wide-angle at long distance and magnification at close distance) can be provided at different distances, optimizing the driver's observation experience; by differentiating the display of negative obstacles in the corresponding in-vehicle panoramic image based on the view mode, the obstacles are made more prominent in specific views, ensuring that the driver can clearly identify them regardless of distance, thereby improving driving safety.

[0062] In this embodiment of the application, before detecting whether there are negative obstacles in the environment surrounding the vehicle, such as Figure 3 As shown, the method also includes: Step S301: Detect whether there is abnormal occlusion in the sensing device used to detect negative obstacles in the vehicle.

[0063] Specifically, sensing devices refer to various sensors on a vehicle used to detect the surrounding environment, especially to identify negative obstacles. These include vehicle-mounted panoramic cameras, ultrasonic radar, lidar, visual recognition sensors, and image acquisition units arranged around the vehicle body. These devices work together to acquire road images, distance, and depth information. Abnormal occlusion refers to the state in which the lens or detection surface of a sensing device is blocked by external objects (such as mud, snow, leaves, etc.) or the signal is attenuated or data is missing due to dirt, thus affecting its normal detection function. This state can be identified by analyzing sensor data characteristics (such as the appearance of regularly shaped black hole areas in point cloud images, or blurring or missing specific areas in images).

[0064] Before detecting negative obstacles, each sensing device undergoes a health self-check. This involves real-time acquisition of raw sensor data and algorithmic analysis to identify any abnormal occlusion. For example, for LiDAR, the system checks for continuous, regular blank areas (i.e., black holes) in the point cloud data. If the shape of the black hole matches the characteristics of an occlusion (e.g., circular or strip-shaped) and its position is fixed, it is determined to be occluded or dirty. For cameras, the system analyzes the texture, brightness, and sharpness of image frames. If a large area of ​​blur, abnormal grayscale values, or missing edge information is detected in the image, and this area does not change over time or with vehicle movement, it is determined that the lens is covered by dirt. Furthermore, cross-validation is performed using multi-sensor data fusion. For instance, if the ultrasonic radar echo intensity suddenly weakens and dirt appears in the images of adjacent cameras, abnormal occlusion is jointly confirmed. The detection process can be configured with dynamic thresholds to consider environmental factors (such as rain and snow) to avoid false alarms. The system also records the time, location, and device number of the occlusion to provide a basis for subsequent alerts and calibration.

[0065] Step S302: If the sensing device is abnormally obstructed, a first prompt message is output to the user, wherein the first prompt message is used to prompt the user to clear the abnormal obstruction of the sensing device.

[0066] Specifically, after determining that the sensing device is abnormally obstructed (e.g., by the appearance of regular black holes in the point cloud map or blurry identification of specific areas in the image), the priority and specific content of the first prompt message are determined according to the type of obstructed device (such as camera, LiDAR, ultrasonic radar) and its installation position on the vehicle (such as front view, rear view, side view). For example, if the front view camera is obstructed, a banner prompting "Front view camera is dirty, please clean it" will pop up at the top of the central control screen, and the corresponding icon will be displayed on the instrument panel; if it is LiDAR, the system will combine this with a voice broadcast saying "LiDAR is obstructed, ranging may be inaccurate" and simultaneously highlight a radar position diagram on the central control screen.

[0067] Secondly, the alert strategy can be adjusted based on the confidence level, duration, and environmental conditions (such as rain and snow). For example, during brief false alarms, only the data can be recorded without alerting, and the data can be output only after confirmation of multiple consecutive frames. During the alert period, the perception function of the affected device can be temporarily disabled or its data weight can be reduced, and the occluded area can be marked with a gray mask or a flashing border in the panoramic image interface to inform the user that the current field of view is unavailable.

[0068] In addition, the initial notification may include instructions on how to clear the obstruction, such as playing a cleaning animation of the camera location or displaying the text "Please wipe the camera." If multiple devices are obstructed simultaneously, they will be displayed together to avoid frequently disturbing the user. The initial notification will continue to be displayed until the abnormal obstruction is detected and cleared.

[0069] Step S303: After outputting the first prompt message, detect in real time whether the abnormal occlusion of the sensing device has been cleared.

[0070] Specifically, after outputting the initial prompt to the user, monitoring of the sensing devices continues uninterrupted. Data from each sensor is collected at a preset sampling frequency (e.g., 10 times per second), and the same or more stringent algorithms are used to analyze the data to verify the fading of occlusion features. For example, for LiDAR, the system continuously checks whether valid point cloud data reappears in areas where regular black holes previously appeared in the point cloud image. If black holes disappear for several consecutive frames (e.g., 5 frames) and the point cloud density returns to normal, it is preliminarily determined that the occlusion has been cleared. For cameras, the system analyzes the texture complexity, brightness gradient, and edge information of areas identified as dirty in the image. If these... As the area gradually becomes clearer, more textured, and consistent with the surrounding environment, it is confirmed that the dirt is no longer present. During the detection process, dynamic calibration is performed by combining the vehicle's motion status (such as speed and steering) and the external environment (such as lighting and weather) to avoid misjudging temporary interference (such as flying insects) as removal. Different confidence thresholds can also be set, such as requiring a higher number of consecutive confirmation frames when the vehicle is stationary to eliminate misjudgments caused by shaking. In addition, multi-sensor cross-validation can be used. For example, when the LiDAR point cloud is restored and the corresponding camera image is clear, the abnormal occlusion is jointly confirmed to have been removed. If removal is determined, a recalibration process is triggered.

[0071] In step S304, if the abnormal occlusion has been cleared, a recalibration process for the sensing device is triggered, and a second prompt message is output to the user after calibration is completed. The second prompt message is used to indicate that the sensing device is functioning normally.

[0072] Specifically, once it is confirmed that the abnormal occlusion has been cleared, a recalibration process for the affected sensing devices is triggered. This process automatically starts the corresponding calibration procedure according to the device type. For example, for cameras, it performs autofocus, exposure adjustment, white balance reset, and image distortion correction, and verifies the image clarity and color reproduction by comparing with historical calibration data or fixed reference objects in the real-time scene (such as lane lines and building edges). For lidar, it performs point cloud noise filtering, reflectivity calibration, and angle deviation correction, and checks the integrity and accuracy of point cloud data by emitting standard test pulses or using known reference objects. For ultrasonic radar, it sends self-test pulses and receives echoes to check sensor sensitivity, ranging consistency, and collaborative working ability with adjacent radars.

[0073] During the calibration process, the output data of the device is continuously monitored and compared with the built-in reference model in real time until all indicators stabilize within the normal threshold, at which point the calibration is considered complete. After calibration, the device outputs a second prompt message to the user through pop-up windows on the central control screen, voice broadcasts, or dashboard icons, such as "Perception device function is normal" or "XX radar / camera has been restored". At the same time, all functions of the device are restored, and it re-participates in the detection of negative obstacles and the construction of panoramic images.

[0074] By detecting abnormal occlusion of the sensing devices used to detect negative obstacles in vehicles, problems with device obstruction can be identified in a timely manner, preventing subsequent detection failures. If abnormal occlusion is detected, an initial prompt message is output to the user, reminding them to clear the obstruction promptly and ensuring normal device operation. Real-time monitoring of the clearing progress after the initial prompt message is output allows for dynamic monitoring of the clearing progress, providing a basis for subsequent operations. If the abnormal occlusion has been cleared, a recalibration process is triggered, and a second prompt message indicating that the device is functioning normally is output after calibration, ensuring the device returns to optimal performance and providing user confirmation, thus improving the user experience.

[0075] In the embodiments of this application, such as Figure 4 As shown, the method also includes: Step S401: When there are multiple negative obstacles in the vehicle panoramic image, obtain the vehicle's driving requirements.

[0076] Specifically, driving needs refer to the type of operation that the user intends to perform in the current driving scenario. This includes two main modes: parking (such as reversing into a parking space, parallel parking, parallel parking, etc.) and driving out (such as driving out of a parking space, starting through a narrow road section, etc.). This need is accurately judged by comprehensively analyzing the vehicle's gear status, steering wheel operation signals, central control commands, environmental information perceived by sensors, and vehicle motion status (such as vehicle speed and acceleration), aiming to make subsequent route planning fit the user's actual driving intentions.

[0077] When multiple negative obstacles (such as potholes, manholes, and collapses) are detected in the vehicle's panoramic imaging system, the driving demand acquisition process is initiated. First, it collects information from the vehicle's gear position (e.g., reverse gear triggering a parking demand, drive gear triggering an exit demand), steering wheel angle changes, central control screen touch commands (e.g., the user actively clicking the parking assist button or selecting the automatic parking function), and surrounding environment information perceived by onboard sensors (e.g., ultrasonic radar detecting parking spaces, cameras recognizing parking lines). This information is combined with vehicle speed sensor data (e.g., speeds below 5 km / h indicating low-speed operation) and the vehicle's stationary state to comprehensively determine the user's current driving intention. For example, if the vehicle is in reverse gear and at zero speed, and the central control screen displays a parking interface or the user has activated the parking function, the driving demand is determined to be parking; if the vehicle is in drive gear, stationary and waiting to start, and there are no parking signs or parking lines nearby, the demand is determined to be exiting. While acquiring driving requirements, the system simultaneously obtains basic information about multiple negative obstacles from the perception module, including the type of each obstacle (such as potholes and pits), geometric dimensions (depth and diameter), and real-time relative position (distance and azimuth) with the vehicle. Combined with established differentiated marking rules, the system confirms the current marking status (such as color and icon) of each obstacle. This data is packaged and stored to provide complete and accurate data support for planning a target driving route that avoids all obstacles, ensuring the safety and rationality of subsequent route planning.

[0078] Step S402: Based on driving requirements and the relative positions of each negative obstacle and the vehicle, plan a target driving route that avoids all negative obstacles.

[0079] Specifically, the target driving route refers to a driving path generated by a route planning algorithm based on the vehicle's current driving needs (such as parking or exiting) and the distribution of multiple negative obstacles. This path can safely avoid all negative obstacles and guide the vehicle smoothly to the target area (such as a parking space or open road). The path is stored in the form of a series of continuous path points or curves and will be visualized and superimposed on the in-vehicle panoramic image in subsequent steps to provide the driver with accurate driving guidance.

[0080] First, the determined driving demand (parking or exiting) and basic information on multiple negative obstacles are obtained, including the type, size, and real-time relative position of each obstacle to the vehicle (e.g., distance, azimuth). This data is then confirmed by marking according to differentiated marking rules. Next, a route planning algorithm is initiated. This algorithm comprehensively considers the vehicle's kinematic characteristics (e.g., minimum turning radius, wheelbase, vehicle width), preset safe distance thresholds (e.g., maintaining a safe clearance of at least 0.3 meters from obstacles), and surrounding environmental boundaries (e.g., the defined parking area, lane lines, curbs, etc.). If the driving requirement is parking, the algorithm combines the target parking location (such as an empty parking space identified by a camera or a parking spot specified by the user) with the distribution of obstacles to generate multiple candidate paths. It prioritizes paths with small turning angles, short path lengths, and safe distances to all obstacles. Simultaneously, it checks for path interference with obstacles; if a candidate path is too close to an obstacle, it automatically adjusts the curvature or avoidance radius until a safe parking route is generated. If the driving requirement is exiting, the algorithm focuses on the passage space in front of and to the sides of the vehicle, prioritizing areas with sparse obstacles and sufficient width. It plans a straight or slightly turning exit route, ensuring that the distance between the route and obstacles on both sides is greater than the safety threshold. During the planning process, multiple rounds of iterative optimization can be performed, dynamically adjusting route parameters (such as fine-tuning avoidance distances based on obstacle size) using real-time perception data, ultimately outputting a safe and efficient target driving route that meets the driving requirements.

[0081] Step S403: The target driving route is overlaid on the vehicle panoramic image and displayed separately from the markings of negative obstacles.

[0082] Specifically, after the target driving route is planned, the route visualization process is initiated.

[0083] First, based on preset exclusive display rules, a set of visual parameters that do not conflict with all negative obstacle markers are assigned to the currently generated target driving route. For example, a thin blue solid line is used as the main body of the route, and white auxiliary lines are drawn on both sides of the route to clarify the driving boundary. At the same time, simple guidance icons, such as turn arrows or avoidance prompts, are superimposed on key nodes of the route (such as turning points and avoidance points) to enhance readability. The drawing scale and position of the route can also be dynamically adjusted according to the current view mode of the vehicle panoramic image (such as single view or vehicle zoom view) to ensure that the route is accurately aligned with the actual ground coordinates.

[0084] Secondly, when overlaying the display, the transparency of the route is automatically adjusted (e.g., set to semi-transparent) so that the obstacle markers and real-time road images below are not completely obscured, allowing users to clearly identify the location of obstacles and the planned path at the same time.

[0085] In addition, the system continuously monitors updates to obstacle markers (such as changes in their relative positions) and adjusts the route's overlay position in real time to ensure the route always maintains the correct relative relationship with the obstacles. The entire process follows a strict layer management strategy, placing the route on top of or on the same layer as obstacle markers but distinguished by color to ensure a logical visual hierarchy.

[0086] Step S404: Based on the deviation between the vehicle's real-time location and the target driving route, update the driving guidance information in the vehicle panoramic image in real time.

[0087] Specifically, while the vehicle is traveling along the target route, GPS, inertial navigation sensor, and panoramic image visual positioning data are fused at high frequency (e.g., every 100ms) to continuously calculate the vehicle's real-time position and heading angle. This data is then aligned and compared with the stored target route to accurately calculate lateral and longitudinal deviations. The calculated deviations are compared with a preset deviation threshold (e.g., 0.1 meters). If the deviation is less than the threshold, it indicates that the vehicle's trajectory matches the planned route, and the current driving guidance information remains unchanged. The target route is then displayed in the panoramic image in its original style (e.g., thin blue solid lines, white auxiliary lines, and turn arrows). If the deviation is greater than or equal to the threshold, a guidance information update process is initiated. Based on the direction (left or right) and magnitude of the deviation, the route display method in the panoramic image is dynamically adjusted. For example, the route may be thickened, its color changed, or a flashing effect added to attract attention. Simultaneously, guidance icons are updated (e.g., displaying left / right turn arrows, and text prompts such as "Please correct to the right"), and voice prompts are triggered (e.g., "Please slightly adjust the direction to the right") to remind the driver to correct the driving direction in a timely manner. Throughout the process, maintain a high frequency of updates to ensure that the guidance information is synchronized with the actual movement of the vehicle, always guiding the vehicle to travel safely along the target route and avoid all negative obstacles.

[0088] By acquiring the vehicle's driving needs when multiple negative obstacles exist in the vehicular panoramic image, the driving intent can be clearly defined, providing direction for route planning. By planning a target driving route that avoids all obstacles based on the driving needs and the relative positions of each obstacle, a safe and intentional path can be generated, avoiding interference with any obstacles. By overlaying the target driving route on the vehicular panoramic image and distinguishing it from the markings of negative obstacles, the driver can intuitively see the suggested trajectory and obstacle positions, making it easy to follow. By updating the driving guidance information in the vehicular panoramic image in real time based on the deviation between the vehicle's real-time position and the target driving route, the driver can be dynamically guided to correct the direction, ensuring that the vehicle always travels along a safe route, improving the accuracy and safety of obstacle avoidance.

[0089] This embodiment also provides a control device for an in-vehicle panoramic image system. This device is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0090] This embodiment provides a control device for in-vehicle panoramic imaging, such as... Figure 5 As shown, it includes: Detection module 51 is used to detect whether there are negative obstacles in the environment around the vehicle; The activation module 52 is used to activate the vehicle's panoramic image if there is a negative obstacle in the environment around the vehicle, and mark the negative obstacle in the panoramic image according to the relative position of the negative obstacle and the vehicle. The processing module 53 is used to continuously detect the relative position of negative obstacles in the vehicle panoramic image until the negative obstacles are no longer in the field of view of the vehicle panoramic image, or the negative obstacles have been confirmed to be eliminated, and then turn off the vehicle panoramic image.

[0091] In this embodiment of the application, the processing module 53 is specifically used to acquire the real-time motion trajectory of the vehicle, and / or detect the shape changes of the negative obstacles; based on the real-time motion trajectory, continuously update the relative position of the negative obstacles in the vehicle panoramic image, and / or, based on the shape changes, continuously update the real-time status of the negative obstacles in the vehicle panoramic image.

[0092] In this embodiment of the application, the absence of a negative obstacle in the vehicle panoramic image includes: its relative position being outside the effective display area of ​​the vehicle panoramic image, or its relative position being in an area obscured by the vehicle itself, making the negative obstacle unable to be displayed in the vehicle panoramic image; the elimination of a negative obstacle includes: the physical form of the negative obstacle being repaired and no longer posing a driving threat to the vehicle, or the negative obstacle being removed.

[0093] In this embodiment, the activation module 52 is specifically used to detect the obstacle type of the negative obstacle; based on the mapping relationship between the preset obstacle type and the marking rule, the target marking rule corresponding to the obstacle type is retrieved, wherein the marking parameters corresponding to different relative positions of the marking rule include at least one of color, line style, and display icon; the target marking parameters that match the relative position are extracted from the target marking rule, and the negative obstacle is differentiatedly marked in the vehicle panoramic image using the target marking parameters.

[0094] In this embodiment, the device further includes: a display module, used to detect the real-time distance between the vehicle and the negative obstacle; adjust the view mode of the vehicle panoramic image according to the real-time distance; and differentiate the display of the negative obstacle in the corresponding vehicle panoramic image based on the view mode.

[0095] In this embodiment of the application, the display module is specifically used to switch the view mode of the vehicle panoramic image from a front / rear single view to a front / rear magnified view if the real-time distance is less than a preset distance threshold; and to switch the view mode of the vehicle panoramic image from a front / rear magnified view to a front / rear single view if the real-time distance is greater than or equal to the preset distance threshold.

[0096] In this embodiment of the application, the device further includes: a prompting module, used to detect whether there is abnormal occlusion of the sensing device used for detecting negative obstacles in the vehicle; if there is abnormal occlusion of the sensing device, a first prompting message is output to the user, wherein the first prompting message is used to prompt the user to clear the abnormal occlusion of the sensing device; after outputting the first prompting message, the system detects in real time whether the abnormal occlusion of the sensing device has been cleared; if the abnormal occlusion has been cleared, the system triggers a recalibration process for the sensing device, and after calibration is completed, a second prompting message is output to the user, wherein the second prompting message is used to prompt that the sensing device is functioning normally.

[0097] In this embodiment of the application, the device further includes: a guidance module, used to obtain the vehicle's driving needs when there are multiple negative obstacles in the vehicle panoramic image; plan a target driving route to avoid all negative obstacles based on the driving needs and the relative positions of each negative obstacle and the vehicle; overlay the target driving route on the vehicle panoramic image and distinguish it from the markings of the negative obstacles; and update the driving guidance information in the vehicle panoramic image in real time based on the deviation between the real-time position of the vehicle and the target driving route.

[0098] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 6As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system).

[0099] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0100] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0101] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device as shown by a landing page for an app. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, which can be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0102] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0103] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0104] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0105] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A control method for vehicle-mounted panoramic imaging, characterized in that, The method includes: Detect whether there are negative obstacles in the environment surrounding the vehicle; If there are negative obstacles in the environment around the vehicle, the vehicle's panoramic image is activated, and the negative obstacles are marked in the panoramic image according to their relative positions to the vehicle. The display status of the negative obstacle in the vehicle panoramic image is continuously monitored until the display status includes: the negative obstacle no longer exists in the vehicle panoramic image, and / or the negative obstacle has been confirmed to be eliminated, at which point the vehicle panoramic image is turned off.

2. The method according to claim 1, characterized in that, The continuous detection of the display status of the negative obstacle in the vehicle panoramic image includes: Acquire the real-time movement trajectory of the vehicle, and / or detect the shape changes of the negative obstacle; Based on the real-time motion trajectory, the relative position of the negative obstacle in the vehicle panoramic image is continuously updated, and / or, based on the morphological changes, the real-time status of the negative obstacle in the vehicle panoramic image is continuously updated.

3. The method according to claim 2, characterized in that, The negative obstacle no longer exists in the vehicle panoramic image including: the relative position is outside the effective display area of ​​the vehicle panoramic image, or the relative position is in an area blocked by the vehicle itself, so that the negative obstacle cannot be displayed in the vehicle panoramic image. The elimination of the negative obstacle is confirmed to include: the real-time status is that the physical form of the negative obstacle has been repaired and no longer poses a driving threat to the vehicle, or the real-time status is that the negative obstacle has been removed.

4. The method according to claim 1, characterized in that, Marking the negative obstacle in the vehicle panoramic image according to the relative position of the negative obstacle and the vehicle includes: Detect the obstacle type of the negative obstacle; Based on the preset mapping relationship between obstacle types and marking rules, the target marking rule corresponding to the obstacle type is retrieved. The marking parameters corresponding to different relative positions of the marking rule include at least one of color, line style, and display icon. The target marker parameters that match the relative position are extracted from the target marker rules, and the negative obstacle is differentiatedly marked in the vehicle panoramic image using the target marker parameters.

5. The method according to claim 1, characterized in that, After activating the vehicle's in-vehicle panoramic imaging system, the method further includes: Detect the real-time distance between the vehicle and the negative obstacle; The view mode of the in-vehicle panoramic image is adjusted according to the real-time distance. Based on the view mode, the negative obstacles are displayed differently in the corresponding in-vehicle panoramic image.

6. The method according to claim 5, characterized in that, The step of adjusting the view mode of the in-vehicle panoramic image according to the real-time distance includes: If the real-time distance is less than a preset distance threshold, the view mode of the vehicle panoramic image will be switched from a front / rear single view to a front / rear vehicle magnified view. If the real-time distance is greater than or equal to a preset distance threshold, the view mode of the vehicle panoramic image will be switched from a magnified view of the front / rear vehicle body to a single view of the front / rear vehicle body.

7. The method according to claim 1, characterized in that, Before detecting the presence of negative obstacles in the environment surrounding the vehicle, the method further includes: Detect whether the sensing device in the vehicle used to detect negative obstacles is abnormally obstructed; If the sensing device is abnormally obstructed, a first prompt message is output to the user, wherein the first prompt message is used to prompt the user to clear the abnormal obstruction of the sensing device. After outputting the first prompt message, it is detected in real time whether the abnormal occlusion of the sensing device has been cleared; If the abnormal occlusion has been cleared, a recalibration process for the sensing device is triggered, and a second prompt message is output to the user after calibration, wherein the second prompt message is used to indicate that the sensing device is functioning normally.

8. The method according to claim 1, characterized in that, The method further includes: When multiple negative obstacles exist in the vehicle panoramic image, the vehicle's driving requirements are obtained; Based on the driving requirements and the relative positions of each of the negative obstacles to the vehicle, a target driving route is planned to avoid all negative obstacles. The target driving route is overlaid on the vehicle panoramic image and displayed separately from the markings of the negative obstacles; Based on the deviation between the vehicle's real-time location and the target driving route, the driving guidance information is updated in real time in the vehicle-mounted panoramic image.

9. A control device for a vehicle-mounted panoramic imaging system, characterized in that, The device includes: The detection module is used to detect whether there are negative obstacles in the environment around the vehicle; The activation module is used to activate the vehicle's panoramic image if there is a negative obstacle in the environment around the vehicle, and mark the negative obstacle in the panoramic image according to the relative position of the negative obstacle and the vehicle. The processing module is used to continuously detect the relative position of the negative obstacle in the vehicle panoramic image until the negative obstacle is no longer in the field of view of the vehicle panoramic image, or the negative obstacle has been confirmed to be eliminated, and then turn off the vehicle panoramic image.

10. A vehicle, characterized in that, include: A sensing device and a controller are interconnected, the sensing device is used to detect negative obstacles in the environment surrounding the vehicle, and the controller stores computer instructions, the controller executing the computer instructions to perform the method of any one of claims 1 to 8.

Images