Vehicle-mounted visual blind compensation control method and system and vehicle

Abstract

The invention relates to the technical field of vehicle control, and discloses a vehicle-mounted visual blind compensation control method and system and a vehicle, and the method comprises the steps: obtaining visual field compensation data, shared by a relative vehicle through a preset network, of a target vehicle in a visual field blind area; analyzing the view compensation data, and determining a real-time data stream and positioning information of the relative vehicle; and performing coordinate conversion according to respective positioning information of the target vehicle and the relative vehicle, determining a relative pose relationship, performing three-dimensional reconstruction based on the three-dimensional point cloud in the real-time data stream at the same time and the relative pose relationship, and obtaining a three-dimensional environment model reflecting the road condition of the target vehicle so as to complete vehicle-mounted visual blind compensation control. The sensing resources of the relative vehicle are shared in real time through the preset network, the view blind area of the target vehicle is effectively covered, the implementation cost is low, the maintenance cost is low, the damage of small obstacles on the road surface to the vehicle is effectively avoided, and the sensing comprehensiveness and reliability of the vehicle under the complex road condition are improved.

Description

Technical Field

[0001] This application relates to the field of vehicle control technology, and in particular to an in-vehicle vision blind spot control method, system and vehicle. Background Technology

[0002] Currently, in the automotive industry, products such as dashcams, in-vehicle cameras, and LiDAR, which can sense images, obstacles, and sounds while a vehicle is in motion, are becoming increasingly popular, with more and more users choosing to install these products in their vehicles. For example, in-vehicle cameras, in addition to recording images and sounds while the vehicle is in motion, can be further expanded to include other related functions, enabling them to perceive the surrounding environment and thus broaden the driver's field of vision, thereby improving driving safety.

[0003] However, in some scenarios, such as logistics, tourism, and business reception, multiple vehicles are required to form a convoy. During operation, each driver is equipped with a walkie-talkie to report information via voice, maintaining the convoy's integrity and coordination. On the one hand, frequent driver operation can distract drivers and compromise convoy safety. On the other hand, the convoy's arrangement can create blind spots, preventing target vehicles from understanding road conditions in these blind spots and leading to misjudgments, potentially causing collisions or damage to the vehicle's undercarriage. Therefore, a new in-vehicle vision-based blind spot control solution is urgently needed to improve driving safety. Summary of the Invention

[0004] This application provides a vehicle-mounted visual blind spot control method, system, and vehicle to solve the problem that vehicle-mounted visual blind spot control methods in related technologies cannot achieve vehicle information sharing, resulting in low vehicle safety during driving.

[0005] In a first aspect, embodiments of this application provide an in-vehicle visual blind spot control method applied to a target vehicle. The method includes: acquiring visual field compensation data of the target vehicle in the blind spot area shared by a relative vehicle through a preset network, wherein the visual field compensation data includes the positioning information of the relative vehicle when collecting real-time data streams; parsing the visual field compensation data to determine the real-time data stream and positioning information of the relative vehicle; performing coordinate transformation based on the positioning information of the target vehicle and the relative vehicle to determine the relative pose relationship; and performing three-dimensional reconstruction based on the three-dimensional point cloud and the relative pose relationship in the real-time data stream at the same time to obtain a three-dimensional environment model reflecting the road conditions of the target vehicle, so as to complete the in-vehicle visual blind spot control.

[0006] In some embodiments of this application, the preset network includes a local area network and at least a 5G communication network. If the distance between the target vehicle and the relative vehicle is less than the preset communication distance, the vehicles communicate using the local area network. If the distance between the target vehicle and the relative vehicle is greater than the preset communication distance, the vehicles communicate using at least a 5G communication network. The local area network is a WIFI network or a V2X direct connection network.

[0007] In some embodiments of this application, before obtaining the vision compensation data of the target vehicle in the blind spot shared by the relative vehicle through a preset network, the method further includes: if the target vehicle has a blind spot, sending a data sharing request to the relative vehicle so that the relative vehicle can parse the data sharing request, the data sharing request including the identification information of the target vehicle and the environmental perception data required for the preset orientation, the environmental perception data including image data and / or radar data; if the relative vehicle agrees to share, sending the real-time environmental perception data required for the preset orientation as vision compensation data to the target vehicle through the preset network.

[0008] In some embodiments of this application, coordinate transformation is performed based on the positioning information of the target vehicle and the relative vehicle to determine the relative pose relationship. Three-dimensional reconstruction is then performed based on the three-dimensional point cloud in the real-time data stream at the same time and the relative pose relationship to obtain a three-dimensional environment model reflecting the road conditions of the target vehicle. This includes: acquiring the local coordinate system formed by each vehicle with itself as the origin, and the positioning information of the target vehicle; performing coordinate transformation based on the positioning information of the target vehicle and the relative vehicle to determine the relative pose relationship; constructing a global coordinate system using the environment of all vehicles in a preset network, and determining the three-dimensional point cloud of each real-time data stream in the global coordinate system based on the relative pose relationship; aligning and fusing the three-dimensional point clouds at the same time, and obtaining a three-dimensional environment model reflecting the road conditions of the target vehicle through global optimization.

[0009] In some embodiments of this application, before determining the relative pose relationship by performing coordinate transformation based on the positioning information of the target vehicle and the relative vehicle, the method further includes: extracting invariant feature points from continuous image frames in the real-time data stream, matching invariant feature points at the same time to determine key environmental features; determining the motion information in the relative vehicle and the calibration parameters of the environmental acquisition device, performing depth calculation on the key environmental features based on the motion information and calibration parameters, and obtaining a three-dimensional environmental model of the relative vehicle in the local coordinate system through three-dimensional reconstruction, wherein each three-dimensional point cloud originates from the target vehicle and / or the relative vehicle.

[0010] In some embodiments of this application, after completing the vehicle-mounted visual blind spot control, the method further includes: generating a control strategy through the three-dimensional environment model to control the target vehicle.

[0011] In some embodiments of this application, a control strategy is generated from a three-dimensional environment model to control the target vehicle, including: if there is a target obstacle in the three-dimensional environment model, then determining the type of the target obstacle; If the target obstacle in the 3D environment model is a bumpy road surface, a first control strategy of deceleration is generated to control the target vehicle; if the target obstacle in the 3D environment model is a forward obstacle, a second control strategy of lane changing is generated to control the target vehicle; if the target obstacle in the 3D environment model is a rear collision, a third control strategy of acceleration or a fourth control strategy of lane changing is generated to control the target vehicle; if there is no target obstacle in the 3D environment model, a fifth control strategy of maintaining driving is generated to control the target vehicle.

[0012] In some embodiments of this application, after completing the vehicle-mounted visual blind spot control, the method further includes: if the target vehicle generates a sharing termination request according to the trigger command, then disconnecting the target vehicle from the preset network to exit information sharing.

[0013] In a second aspect, this application also provides an in-vehicle visual blind spot control system applied to a target vehicle. The system includes: a data acquisition module for acquiring visual field compensation data of the target vehicle in the blind spot area shared by a relative vehicle through a preset network, the visual field compensation data including the positioning information of the relative vehicle when acquiring real-time data streams; a data parsing module for parsing the visual field compensation data to determine the real-time data stream and positioning information of the relative vehicle; and a visual blind spot control module for performing coordinate transformation based on the positioning information of the target vehicle and the relative vehicle to determine the relative pose relationship, and performing three-dimensional reconstruction based on the three-dimensional point cloud and the relative pose relationship in the real-time data stream at the same time to obtain a three-dimensional environment model reflecting the road conditions of the target vehicle, so as to complete the in-vehicle visual blind spot control.

[0014] In a third aspect, embodiments of this application also provide a vehicle employing the method of any of the above embodiments.

[0015] In a fourth aspect, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of any of the above embodiments.

[0016] The above-mentioned vehicle-mounted visual blind spot control method, system, and vehicle implementation scheme involves acquiring the visual field compensation data of the target vehicle in the blind spot area shared by the relative vehicle through a preset network, parsing the visual field compensation data, determining the real-time data stream and positioning information of the relative vehicle, performing coordinate transformation based on the positioning information of the target vehicle and the relative vehicle to determine the relative pose relationship, and performing three-dimensional reconstruction based on the three-dimensional point cloud and the relative pose relationship in the real-time data stream at the same time to obtain a three-dimensional environment model reflecting the road conditions of the target vehicle, thereby completing the vehicle-mounted visual blind spot control. First, by sharing the perception resources of the target vehicle in real time through a pre-set network, the blind spots of the target vehicle's field of vision are effectively covered, resulting in low cost and low maintenance cost. This effectively avoids damage to the vehicle from small road obstacles and improves the comprehensiveness and reliability of the vehicle's perception in complex road conditions. Second, by achieving spatiotemporal alignment of multi-source data through coordinate transformation and combining it with 3D reconstruction technology to construct a high-precision 3D environment model of the blind spot, decision-making errors caused by road condition perception deviations are avoided. Third, based on the accurate 3D environment model and the real-time status of the vehicle, targeted control strategies are generated to achieve precise adjustment of steering and speed, effectively reducing the risk of accidents such as blind spot collisions and scrapes, and improving vehicle driving safety and ride comfort. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0018] In the attached diagram: Figure 1 A schematic diagram of an exemplary architecture for an in-vehicle vision blind spot control system provided in an embodiment of this application; Figure 2 A flowchart illustrating the vehicle-mounted visual blind spot control method provided in an embodiment of this application; Figure 3 A schematic diagram of screen sharing for the vehicle-mounted visual blind spot control method provided in the embodiments of this application; Figure 4 This is a schematic diagram illustrating video sharing between vehicle fleets as provided in an embodiment of this application. Figure 5 This is a schematic diagram of data interaction for the vehicle-mounted visual blind spot control method provided in an embodiment of this application; Figure 6 A schematic diagram of the structure of an in-vehicle vision blind spot control system provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application; Figure 8 This is another structural schematic diagram of an electronic device according to one embodiment of this application. Detailed Implementation

[0019] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0020] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. The drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0021] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the present application. However, it will be apparent to those skilled in the art that embodiments of the present application may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the present application.

[0022] Please see Figure 1 This application provides an exemplary architecture diagram of an in-vehicle visual blind spot control system. The implementation environment includes a terminal, which can be a vehicle 101. Vehicle 101 includes both new energy vehicles and fuel-powered vehicles. Vehicle 101 has a built-in in-vehicle visual blind spot control system 100. This system enables information sharing between vehicles, facilitating accurate warnings during driving and effectively preventing chain-reaction collisions caused by sudden braking, road potholes, etc., thus improving driving safety.

[0023] In existing technologies, blind spots are common in vehicles due to limitations in vehicle structure and driving perspective, preventing drivers from fully perceiving surrounding road conditions and increasing the risk of collisions and scrapes. For example, the lack of road perception caused by blind spots is particularly pronounced during turning, lane changes, and low-speed driving. Current mainstream blind spot perception solutions rely heavily on onboard environmental sensors such as cameras and radar, which suffer from limited sensing range, susceptibility to ambient light and weather conditions, and insufficient ability to identify distant or obscured targets. Furthermore, sensor data from a single vehicle cannot achieve full-scene coverage, failing to provide a complete perception of blind spot conditions. In addition, the lack of effective fusion and utilization of multi-source heterogeneous data makes it difficult to accurately construct a three-dimensional environment of the blind spot, thus failing to provide reliable decision-making basis for vehicle control and resulting in insufficient targeting and accuracy of control strategies.

[0024] Therefore, for the vehicle-mounted visual blind spot control method, system, and vehicle provided in this application, please refer to [link / reference needed]. Figure 2 This is a flowchart illustrating an in-vehicle vision blind spot control method provided in an embodiment of this application, applied to a target vehicle. The method includes: Step S210: Obtain the vision compensation data of the target vehicle in the blind spot shared by the relative vehicle through a preset network. The vision compensation data includes the positioning information of the relative vehicle when collecting real-time data stream. For example, a data interaction link between multiple vehicles is constructed using a pre-defined communication network, enabling the target vehicle to move its sensing nodes by utilizing the sensor resources of other vehicles to acquire road condition data within its blind spot. The synchronous collection and transmission of positioning information provides a benchmark for the spatiotemporal alignment of subsequent multi-source data, avoiding fusion deviations caused by differences in data collection time and location.

[0025] For example, a data acquisition request is sent to a relative vehicle within the blind spot coverage area via a preset network, such as a V2X vehicle-to-everything network, a 5G communication network, or a dedicated short-range communication network. After the relative vehicle responds to the request, it encapsulates the real-time data stream it collects and the corresponding location information at the time of collection into field-of-view compensation data and feeds it back to the target vehicle through the preset network. Optionally, the integrity of the field-of-view compensation data received by the target vehicle can be verified to remove packet loss and erroneous data, ensuring the validity of the data.

[0026] Step S220: Analyze the field of view compensation data to determine the real-time data stream and positioning information of the relative vehicle; For example, the target vehicle uses a preset parsing algorithm to decapsulate the received field-of-view compensation data, separating the real-time data stream and positioning information. For instance, the real-time data stream undergoes format conversion, such as converting the raw sensor data of the relative vehicle into a standardized format compatible with the target vehicle, and noise filtering, using algorithms such as Gaussian filtering and median filtering to remove noise data caused by environmental interference; the positioning information is parsed to output a standardized real-time data stream and positioning information.

[0027] Optionally, a deep learning-driven analytical model can be used to quickly depackage and convert data formats through a trained neural network, improving parsing efficiency. Alternatively, a multi-filtering algorithm fusion scheme can be employed to enhance noise filtering performance.

[0028] Step S230: Based on the positioning information of the target vehicle and the relative vehicle, coordinate transformation is performed to determine the relative pose relationship. Based on the 3D point cloud and the relative pose relationship in the real-time data stream at the same time, 3D reconstruction is performed to obtain a 3D environment model reflecting the road conditions of the target vehicle, so as to complete the vehicle-mounted visual blind spot control.

[0029] The relative pose relationship refers to the combined spatial position and attitude relationship between the target vehicle and the relative vehicle. By unifying the positioning information of the two vehicles from different coordinate systems (local coordinate systems) to the same reference system (global coordinate system) through coordinate transformation, the relative motion parameters are obtained. For example, the attitude relationship is used to describe the difference in orientation between the target vehicle and the relative vehicle, and commonly uses yaw angle ψ, pitch angle θ, and roll angle. This is represented. For example, spatial positional relationships are used to describe the spatial positional offset of a target vehicle relative to a vehicle reference frame.

[0030] For example, coordinate transformation based on rigid body kinematics eliminates the benchmark differences in positioning data from different vehicles by unifying the coordinate system, ensuring the accuracy of relative pose calculation. The relative pose relationship is calculated by transforming the difference in positioning data with attitude angles, reflecting the correlation between the spatial position and attitude relationships between the two vehicles. Furthermore, 3D reconstruction is based on point cloud fusion and geometric modeling principles. Point cloud registration achieves spatiotemporal alignment of multi-source point clouds, and reconstruction algorithms construct a continuous and complete 3D environment model to restore the spatial characteristics of blind spot road conditions.

[0031] For example, the real-time positioning information of the target vehicle itself is obtained, and the positioning information of the target vehicle and the relative vehicle is uniformly transformed to the same coordinate system; the relative pose relationship between the two is calculated using a rigid body transformation algorithm, including the relative position coordinate difference and the relative attitude angle; by extracting the three-dimensional point cloud data from the real-time data stream of the relative vehicle at the same timestamp, and combining the calculated relative pose relationship, the three-dimensional point cloud of the relative vehicle is fused with the point cloud data collected by the target vehicle itself using a point cloud registration algorithm; if they are found, they are fused; based on the fused point cloud data, a three-dimensional reconstruction algorithm is used to construct a three-dimensional environment model reflecting the road conditions of the target vehicle.

[0032] Optionally, based on the above embodiments, after step S230, the method further includes: generating a control strategy through a three-dimensional environment model to control the target vehicle.

[0033] For example, the control strategy is generated based on optimal control theory, using algorithms to solve for the optimal vehicle motion state under current road condition constraints. This ensures that the actuators accurately respond to control commands, while dynamically correcting control deviations through a feedback mechanism to guarantee control stability. For instance, road condition analysis is performed on the constructed 3D environment model to identify target obstacles within blind spots, such as obstacle type, size, and location, road curvature, and slope. It may also include the relative driving state of other vehicles. Based on the road condition analysis results, combined with the vehicle's current driving state, such as speed, steering angle, and load, algorithms such as model predictive control, PID control, or reinforcement learning are used to generate a control strategy. This strategy includes commands such as steering adjustment, speed regulation (acceleration / deceleration / braking), and lane keeping. The control strategy is converted into control signals recognizable by the vehicle's actuators and sent to the corresponding actuators to achieve precise control of the target vehicle. Simultaneously, the control effect is fed back in real time, and the control strategy is dynamically adjusted.

[0034] The above method involves acquiring vision compensation data of the target vehicle in the blind spot area shared by the relative vehicle through a preset network, parsing the vision compensation data to determine the real-time data stream and positioning information of the relative vehicle, performing coordinate transformation based on the positioning information of the target vehicle and the relative vehicle to determine the relative pose relationship, and performing 3D reconstruction based on the 3D point cloud and the relative pose relationship in the real-time data stream at the same time to obtain a 3D environment model reflecting the road conditions of the target vehicle. A control strategy is then generated using the 3D environment model to control the target vehicle. First, by sharing the perception resources of the target vehicle in real time through a pre-set network, the blind spots of the target vehicle's field of vision are effectively covered, resulting in low cost and low maintenance cost. This effectively avoids damage to the vehicle from small road obstacles and improves the comprehensiveness and reliability of the vehicle's perception in complex road conditions. Second, by achieving spatiotemporal alignment of multi-source data through coordinate transformation and combining it with 3D reconstruction technology to construct a high-precision 3D environment model of the blind spot, decision-making errors caused by road condition perception deviations are avoided. Third, based on the accurate 3D environment model and the real-time status of the vehicle, targeted control strategies are generated to achieve precise adjustment of steering and speed, effectively reducing the risk of accidents such as blind spot collisions and scrapes, and improving vehicle driving safety and ride comfort.

[0035] In related technologies, information sharing between vehicles is hampered by low transmission rates, making it impossible to provide timely references for vehicles.

[0036] In some embodiments, the preset network includes a local area network and at least a 5G communication network. If the distance between the target vehicle and the relative vehicle is less than the preset communication distance, the vehicles communicate using the local area network. If the distance between the target vehicle and the relative vehicle is greater than the preset communication distance, the vehicles communicate using at least a 5G communication network. The local area network is a WIFI network or a V2X direct connection network.

[0037] For example, based on the principle of satellite positioning, the precise spatial coordinates of the vehicle can be obtained through the vehicle positioning module, which can accurately calculate the distance between vehicles; for example, based on the principles of spherical geometry or plane geometry, the spatial straight-line distance between two vehicles can be solved according to the positioning coordinates, thereby realizing the quantification of the relative position between vehicles; the setting of the preset communication distance threshold is based on the communication characteristics of different networks, which have the advantages of short distance and low latency of local area networks, and the advantages of long distance and wide coverage of 5G networks.

[0038] For example, based on scenario adaptation and network characteristic matching, the scenario type is determined by the distance threshold, and the corresponding advantageous network is matched: for example, local area network (WIFI / V2X direct connection) has the advantages of low latency, low power consumption and high bandwidth in short-distance scenarios, which can meet the real-time transmission requirements of field-of-view compensation data; 5G network has wide coverage and long-distance transmission capabilities, which can ensure the effectiveness of data transmission in long-distance scenarios; ensuring that the network parameters of both parties are consistent and avoiding link establishment failure.

[0039] For example, monitoring is performed based on the data transmission link. By collecting network transmission parameters and vehicle distance data in real time, a dual feedback mechanism for network status and location status is constructed. When a status change is detected that causes the current network to no longer be suitable, a switching process is triggered. The optimal network is reselected based on preset rules to achieve closed-loop adaptation of distance, network, and transmission quality. During the network switching process, a data caching and breakpoint resume mechanism is adopted to avoid data loss during the switching process.

[0040] By employing the above methods and dynamically selecting the network based on distance, the vehicle can match the optimal communication network in different distance scenarios, solving the transmission defects of a single network in near / long-distance scenarios and improving the scenario adaptability of vision compensation data transmission. For near distances, local area network (WIFI / V2X direct connection) transmission is used, leveraging its low latency and high bandwidth advantages to ensure real-time transmission of vision compensation data. For long distances, 5G network transmission is used, relying on its wide coverage to ensure the effectiveness of data transmission. Simultaneously, real-time monitoring and dynamic switching further improve the stability of data transmission and reduce packet loss and latency issues. By dynamically adjusting network parameters and switching strategies, the power consumption of the onboard communication module can be reduced, improving vehicle range and balancing transmission performance and economy.

[0041] In related technologies, in vehicle blind spot perception solutions, the target vehicle's own sensors cannot cover the entire blind spot area and cannot accurately obtain environmental information in specific locations within the blind spot, which can easily lead to incomplete road condition perception. At the same time, existing cross-vehicle data sharing solutions mostly push data indiscriminately, lacking targeted request mechanisms, which can easily lead to data redundancy, consume communication resources, and fail to establish a sharing decision mechanism relative to vehicles, which may cause data privacy leaks or unnecessary resource consumption.

[0042] In some embodiments, before acquiring the vision compensation data of the target vehicle in the blind spot shared by the relative vehicle via a preset network, the method further includes: If the target vehicle has a blind spot, a data sharing request is sent to the relative vehicle so that the relative vehicle can resolve the data sharing request. The data sharing request includes the target vehicle's identification information and the environmental perception data required for the preset location. The environmental perception data includes image data and / or radar data. If the other vehicle agrees to share, the real-time environmental perception data required for the preset location will be sent to the target vehicle as field-of-view compensation data through the preset network.

[0043] For example, the target vehicle uses an onboard perception system, such as a camera, lidar, or ultrasonic radar, combined with vehicle attitude sensors, such as a steering angle sensor and a speed sensor, to collect real-time data on its surrounding environment and driving status. Based on a preset blind spot detection algorithm, it analyzes and determines the current blind spot range and the preset location corresponding to the blind spot. Subsequently, the target vehicle generates a data sharing request, which encapsulates the target vehicle's unique identification information, such as the VIN code and the onboard terminal IMEI code, the preset location information corresponding to the blind spot, and the required environmental perception data type. The target vehicle then sends this data sharing request to the relative vehicle within the coverage area of ​​the preset location of the blind spot via a vehicle-to-everything (V2X) communication module.

[0044] For example, after the relative vehicle receives a data sharing request through its onboard communication module TBOX, the CDC (i.e., continuous damping control system) uses a preset protocol parsing algorithm to extract the target vehicle identification information, preset orientation information, and the required environmental perception data type from the sharing request. Subsequently, the relative vehicle verifies the legality of the target vehicle identification and determines whether its current position and attitude are within the preset orientation coverage area in the request and whether its onboard sensors have the capability to collect the required data type. Combining the preset sharing decision rules, it generates a decision result of agreeing or refusing to share. If sharing is refused, the relative vehicle sends a rejection response to the target vehicle; if sharing is agreed, the subsequent data collection and transmission process begins.

[0045] For example, if the relative vehicle agrees to share, it adjusts the acquisition parameters of its own onboard sensors based on the preset orientation information obtained through parsing; it collects environmental perception data of the preset orientation in real time through the corresponding sensors; it preprocesses the collected data, encapsulates the preprocessed real-time environmental perception data into field-of-view compensation data, and adds a timestamp and its own identification information; it sends the field-of-view compensation data to the target vehicle through a preset network; after receiving the data, the target vehicle sends back a data reception confirmation signal; if no confirmation is received, the relative vehicle retransmits the data.

[0046] By using the above methods, environmental perception data from preset locations is requested in a targeted manner, avoiding redundancy caused by indiscriminate data sharing. This ensures that the target vehicle only obtains the core data required for blind spots, improving the accuracy of vision compensation and reducing communication resource consumption. Through target vehicle identification verification and relative vehicle sharing decision-making mechanisms, illegal requests are effectively filtered, balancing the contradictions between data sharing, privacy protection, and resource consumption, thereby improving the compliance and reliability of the solution.

[0047] In related technologies, there are technical issues such as how to establish a unified global coordinate system, achieve accurate fusion of perception data under the local coordinate system of multiple vehicles, and solve the model misalignment problem caused by the inconsistency of coordinate references of multi-source data; and how to improve the accuracy of relative pose relationship calculation and balance the contradiction between the dependence on positioning information and the accuracy of calculation.

[0048] In some embodiments, coordinate transformation is performed based on the positioning information of the target vehicle and the relative vehicle to determine the relative pose relationship. Three-dimensional reconstruction is then performed based on the three-dimensional point cloud and the relative pose relationship in the real-time data stream at the same time to obtain a three-dimensional environment model reflecting the road conditions of the target vehicle, including: Obtain the local coordinate system formed by each vehicle with itself as the origin, as well as the positioning information of the target vehicle; The relative pose relationship is determined by performing coordinate transformation based on the positioning information of the target vehicle and the relative vehicle. A global coordinate system is constructed based on the environment of all vehicles in the preset network, and the three-dimensional point cloud of each real-time data stream in the global coordinate system is determined according to the relative pose relationship. Align and fuse the 3D point clouds at the same time, and obtain a 3D environment model that reflects the road conditions of the target vehicle through global optimization.

[0049] In some embodiments, before determining the relative pose relationship by performing coordinate transformation based on the positioning information of the target vehicle and the relative vehicle, the method further includes: Extract invariant feature points from consecutive image frames in a real-time data stream, match invariant feature points at the same time, and determine key environmental features. The motion information of the relative vehicle and the calibration parameters of the environmental acquisition equipment are determined. Based on the motion information and calibration parameters, the key environmental features are calculated in depth, and a three-dimensional environmental model of the relative vehicle in the local coordinate system is obtained through three-dimensional reconstruction.

[0050] In some embodiments, each 3D point cloud is derived from the target vehicle and / or the relative vehicle.

[0051] For example, each vehicle determines its own attitude reference through onboard attitude sensors, such as inertial measurement units (IMUs) and gyroscopes, and establishes a local coordinate system with the vehicle's center of mass as the origin. Subsequently, the target vehicle collects its own real-time positioning information, including latitude and longitude, altitude, and positioning reliability, through an onboard positioning module, such as a GPS / BeiDou high-precision positioning module, and sends a positioning information acquisition request to the relative vehicle through a preset network. The relative vehicle responds to the request and feeds back its own real-time positioning information to the target vehicle. The target vehicle verifies the validity of all received positioning information and completes the positioning information of the target vehicle and the relative vehicle in the local coordinate system of each vehicle.

[0052] For example, the target vehicle converts its own and the relative vehicle's positioning information (latitude, longitude, altitude) into spatial rectangular coordinates in the same Earth coordinate system. Based on a coordinate transformation algorithm, such as a rigid body transformation algorithm, using the target vehicle's spatial rectangular coordinates as a reference, the translation vector (displacement difference in the X, Y, and Z axes) and rotation matrix (rotation angles around the X, Y, and Z axes) of the relative vehicle relative to the target vehicle are calculated. The translation vector and rotation matrix are combined to obtain the relative pose relationship between the target vehicle and the relative vehicle. The accuracy of the calculated relative pose relationship is verified. If the error is greater than a preset threshold, the coordinate transformation calculation is repeated until the accuracy requirements are met. Based on the positioning information of all vehicles in the preset network, the target vehicle uses a global coordinate system construction algorithm to select the position of the vehicle with the highest positioning accuracy as the origin of the global coordinate system. The vehicle's forward direction is taken as the global X-axis, the horizontal leftward direction is taken as the global Y-axis, and the vertical upward direction is taken as the global Z-axis. A global coordinate system covering the environment of all vehicles in the preset network is established. Subsequently, based on the obtained relative pose relationship, combined with the local coordinate system of each vehicle and its own positioning information, the 3D point cloud in the real-time data stream of each vehicle is transformed from its local coordinate system to the global coordinate system through a coordinate transformation formula to obtain the spatial coordinates of each 3D point cloud in the global coordinate system.

[0053] For example, the target vehicle extracts 3D point cloud data of each vehicle in the global coordinate system at the same time stamp, and uses a point cloud alignment algorithm to finely align the point clouds, eliminating point cloud misalignment caused by residual errors in coordinate transformation; the aligned point cloud data is then fused to remove redundant and noisy points; subsequently, a global optimization algorithm is used to globally adjust the fused point cloud, correcting local fusion deviations and ensuring the global consistency of the model; finally, based on the optimized point cloud data, a 3D reconstruction algorithm is used to construct a 3D environment model reflecting the road conditions of the target vehicle. The 3D environment model includes key environmental features such as roads, obstacles, and traffic signs.

[0054] For example, the relative vehicle extracts continuous image frames from the real-time data stream, and uses an invariant feature point extraction algorithm to extract feature points with rotation invariance and scale invariance from each image frame; feature descriptors are generated for the extracted feature points, and feature matching algorithms are used to match feature points (feature descriptors) of different image frames at the same timestamp; a random sampling consensus algorithm is used to remove outliers in the matching and retain inliers; based on the feature points corresponding to the inliers, key environmental features are determined, and the spatial location association information of the key features is recorded. The relative vehicle collects its own motion information through onboard motion sensors; it obtains preset calibration parameters of the environmental acquisition equipment, such as camera intrinsic parameters, camera and vehicle extrinsic parameters, and radar and camera extrinsic parameters; based on the motion information and calibration parameters, a depth calculation algorithm, such as a stereo matching depth calculation algorithm based on binocular vision, is used; depth calculation is performed on the determined key environmental features to obtain the three-dimensional coordinates of each key feature in the relative vehicle's local coordinate system; based on the three-dimensional coordinates of all key features, a three-dimensional reconstruction algorithm is used to construct a three-dimensional environmental model of the relative vehicle in the local coordinate system; the accuracy of the reconstructed model is verified, and if the model deviation is greater than a preset threshold, depth calculation and reconstruction are performed again.

[0055] By employing the above methods, the problem of inconsistent local coordinate references for multiple vehicles is solved through the construction and transformation of a global coordinate system. This enables the effective fusion of perception data from all vehicles within the pre-defined network, breaking through the limitations of a single vehicle's perception field of view and constructing a comprehensive 3D environment model covering the road conditions of the target vehicle. Through relative pose optimization, point cloud alignment and fusion, and global optimization, errors in coordinate transformation and data fusion processes are effectively reduced. Furthermore, through feature point extraction and depth calculation optimization, the quality of local data is further improved.

[0056] In related technologies, existing vehicle control solutions based on environmental perception mostly rely on judging the presence or absence of obstacles in 3D environmental models, lacking precise classification of obstacle types, resulting in insufficient targeting of control strategies. A single control strategy is difficult to adapt to different obstacle scenarios, such as bumpy roads, static obstacles in front, and rear collision risks, and is prone to over-control or under-control, affecting driving safety and comfort. At the same time, existing solutions do not establish a clear matching logic between obstacle types and control strategies, resulting in delayed control decision response, and the driving state maintenance strategy in obstacle-free scenarios lacks optimization, failing to balance driving efficiency and stability.

[0057] In some embodiments, a control strategy is generated using a three-dimensional environment model to control the target vehicle, including: If there are obstacles in the 3D environment model, determine the type of obstacle. If the target obstacle in the 3D environment model is a bumpy road surface, then a first control strategy for deceleration is generated to control the target vehicle. If the target obstacle in the 3D environment model is an obstacle ahead, then a second lane-changing control strategy is generated to control the target vehicle. If the target obstacle in the 3D environment model is a rear collision, then a third control strategy of acceleration or a fourth control strategy of lane change is generated to control the target vehicle. If there are no target obstacles in the 3D environment model, a fifth control strategy to maintain driving is generated to control the target vehicle.

[0058] For example, a constructed 3D environment model reflecting the road conditions of the target vehicle is acquired, including 3D spatial information such as roads, surrounding objects, and terrain. Obstacle detection algorithms, such as point cloud clustering-based obstacle detection algorithms and 3D model semantic segmentation-based obstacle recognition algorithms, are used to traverse and analyze the 3D environment model, extracting abnormal areas in the model that differ from normal road conditions and identifying them as target obstacles. Feature parameters of the target obstacles are extracted, such as the obstacle's spatial location, size, shape, motion state, and relative distance / relative speed to the target vehicle. Based on preset obstacle classification rules and the feature parameters, the type of target obstacle is determined.

[0059] If an obstacle is located within the target vehicle's driving surface, has an irregular, protruding shape, and shows no obvious movement characteristics, it is identified as a road bump. If the obstacle is located in front of the target vehicle in its direction of travel, and its relative speed to the target vehicle is zero or slowly approaching, it is identified as a forward obstacle. If the obstacle is located behind the target vehicle, and its relative speed to the target vehicle is greater than a preset threshold and the relative distance is continuously decreasing, it is identified as a rear collision risk. If no such abnormal areas are detected in the 3D environment model, it is determined that there is no target obstacle. When the target obstacle is identified as a road bump, a first control strategy is ultimately formed, which includes deceleration speed, deceleration timing, and suspension adjustment parameters.

[0060] For example, when the target obstacle is determined to be an obstacle ahead, the position, size, and relative distance of the obstacle to the target vehicle are extracted, and the traffic status of the surrounding lanes in the 3D environment model is obtained. Based on the lane change safety judgment model, the feasibility of lane change is verified: if there are no obstacles in the adjacent lane and the width is sufficient, the lane change trajectory, lane change speed, and lane change timing are calculated; a second control strategy is generated, including lane change direction, lane change trajectory parameters, speed adjustment command, and turn signal control command; if the adjacent lane cannot be changed, a deceleration and stopping command can be superimposed to ensure safe stopping before collision. For example, when no target obstacle is detected in the 3D environment model, the current driving state parameters of the target vehicle are extracted; a fifth control strategy is generated to maintain the current driving state: if the vehicle is in constant speed driving mode, the current set speed is maintained, and the engine output power is kept stable; if the vehicle is in non-constant speed driving mode, the driving speed corresponding to the current accelerator / brake pedal control force is maintained; at the same time, the steering system parameters are adjusted through the lane keeping assist algorithm to ensure that the vehicle drives stably in the current lane; changes in road conditions ahead are monitored in real time to reserve decision time for subsequent obstacle response.

[0061] For example, the corresponding control strategy described above is converted into a control signal that can be recognized by the vehicle actuator; the control signal is sent to the corresponding actuator via the vehicle bus; the actuator responds to the control signal and completes operations such as speed adjustment, lane change, and suspension adjustment; at the same time, the feedback data of the actuator is collected in real time and compared with the preset parameters of the control strategy. If there is a deviation, the control signal is corrected through a closed-loop control algorithm to ensure that the control effect meets expectations.

[0062] By accurately classifying target obstacles in the 3D environment model, the system matches appropriate control strategies to different obstacle scenarios, avoiding the limitations of a single control strategy. This effectively reduces safety risks in scenarios such as road bumps, obstacles ahead, and rear collisions, achieving accurate obstacle type identification and differentiated control, thus improving driving safety. Optimizing control parameters for different obstacle types, such as smooth deceleration and lane changes, reduces driving discomfort caused by over-control. The steady-state maintenance strategy in obstacle-free scenarios balances stability and driving efficiency, improving the overall driving experience. It ensures rapid response in obstacle identification, strategy generation, and execution, reducing decision-making delays while correcting control deviations and guaranteeing the stability of control effects.

[0063] Optionally, in some embodiments, after completing the vehicle-mounted visual blind spot control, the method further includes: If the target vehicle generates a sharing termination request according to the trigger command, the target vehicle's connection in the preset network will be disconnected to exit information sharing.

[0064] By responding to multi-source trigger commands, the target vehicle can terminate the connection in a timely manner when sharing is no longer needed, avoiding the waste of communication resources and vehicle energy caused by continuous connection, and improving the overall communication efficiency of the preset network and the vehicle's range. Through standardized termination request generation, bidirectional command interaction and thorough resource release process, risks such as connection residue and data leakage are eliminated. At the same time, the hierarchical processing of cached data balances the timeliness of termination and data integrity, improving the security and reliability of the solution.

[0065] Please see Figure 3 This is a schematic diagram of screen sharing for the vehicle-mounted visual blind spot control method provided in this application embodiment; please refer to... Figure 4 This is a schematic diagram of video sharing between vehicle fleets provided in an embodiment of this application; The fleet shares vehicle information in a local area network mode. Vehicles A and B are lined up to form a fleet. No additional special equipment is needed inside the vehicles. For example, the vehicles include external front, rear, left and right camera modules, TBOX, CDC control interface, and camera data sharing and processing terminal.

[0066] The front, rear, left, and right camera modules collect external images, share them with other vehicles on the local area network via the camera data sharing and processing terminal and the TBOX local area wireless network.

[0067] The TBOX is connected to the camera data sharing and processing terminal. The camera data sharing terminal sends the shared data stream to the TBOX of the target vehicle in a point-to-point manner using the RTP (Real-Time Streaming) protocol. The target TBOX receives the video data stream and forwards it to the vehicle's CDC for display.

[0068] The CDC control interface is used to control camera sharing connection requests, sharing connection reception, and sharing camera image display.

[0069] The camera data sharing and processing terminal receives sharing control commands from the CDC and is used to control the camera to share data streams.

[0070] This system employs a high degree of fusion between visual SLAM technology and inter-vehicle positioning information. Utilizing only onboard cameras and positioning data, it constructs an accurate 3D environment model and enables dynamic obstacle avoidance and early warning. Through continuous image data streams and information exchange between vehicles, it generates a real-time map of the surrounding environment and predicts potential collision risks. The implementation process of visual SLAM technology is as follows: The vehicle-mounted camera is used to capture front, side, and rear view images in real time. Image features are extracted from the front, side, and rear view images using either SURF (Speeded Up Robust Features) or ORB (Oriented FAST and Rotated BRIEF) algorithms. The feature points and descriptors extracted from each view are then standardized to form a feature dataset in a unified format.

[0071] For example, firstly, the time reference of all images is unified to eliminate the problem of spatiotemporal misalignment of different frames and consecutive frame sequences of multi-view images at the same physical moment. Based on the time-aligned images, accurate feature matching is completed across viewpoints or between frames, and stable and effective key feature points are selected. By continuously tracking the key feature points, an uninterrupted temporal trajectory is formed. The tracked key feature points in the pixel domain are converted into three-dimensional spatial coordinates to construct a lightweight sparse local map, i.e., a map in a local coordinate system.

[0072] Using either binocular or monocular visual SLAM algorithms, with camera intrinsic parameters as the mapping benchmark between pixels and space, and motion information from the SLAM algorithm as the basis for viewpoint association, the binocular camera directly estimates depth by calculating depth through disparity when the poses of the left and right cameras are known at the same moment through triangulation. Alternatively, the monocular camera calculates depth through multi-frame triangulation when the poses of the cameras at different moments are known. Then, the depth points are aggregated into a point cloud to obtain a 3D point cloud of continuous frame images. By extracting environmental geometry and obstacle depth through structured methods, the geometric structure of the vehicle's surrounding environment and the depth information of obstacles can be obtained.

[0073] Each vehicle obtains its global positioning information through its onboard GPS system and shares the location information through a shared local area network communication link; vehicles transmit data with each other through the onboard local area network to ensure real-time synchronization of positioning data, including speed, heading angle, and position coordinates.

[0074] Based on the location information of vehicles within the local area network, Kalman filtering technology is used to transform and fuse the local coordinate system and the global coordinate system. Combined with the relative position and attitude data of the vehicle body obtained by visual SLAM, the positioning error of each vehicle is accurately corrected, ensuring a high degree of consistency between the map and the environment model.

[0075] By preprocessing the local maps generated by visual SLAM for each vehicle, a standardized local map is obtained. This standardized local map is then aligned and merged with the standardized local maps of other vehicles in the fleet to eliminate initial deviations between vehicles. A global optimization algorithm is used to minimize all constraint errors. Specifically, by ensuring that the coordinates of all nodes (vehicle poses, feature points) in the global map satisfy all constraints, global coordinate consistency is achieved. The point clouds are merged and dynamically updated to generate a shared global environment map for the fleet, allowing all vehicles in the local area network to share an accurate global environment map.

[0076] Please see Figure 5 The diagram below illustrates the data interaction of the vehicle-mounted visual blind spot control method provided in this embodiment of the application. 1) In a convoy of vehicles, the driver observes the surroundings to determine if there are blind spots. If so, the driver clicks a shared camera search request. The CDC (Digital Control Center) communicates via CAN or CANFD bus to transmit this signal to the TBOX (Tank Optimizer). The TBOX processes the signal and distributes it to other vehicles within a 10-meter radius of the location via the vehicle positioning system. Simultaneously, the signal is displayed in the CDC of the requesting vehicle. The driver selects the vehicle to be shared from the displayed images and sends a request for a local area network connection. This request is then distributed to the vehicle to be shared via the cloud, and the driver also turns on their vehicle's Wi-Fi hotspot. The vehicle to be shared receives the distributed sharing request signal and clicks to agree to share or refuse to share. When the driver refuses to share, this signal is distributed to the requesting vehicle via the cloud. When the driver agrees to share, this signal is forwarded to the camera data sharing and processing terminal via CAN or CANFD bus, and the TBOX is notified to establish a Wi-Fi connection. 2) The camera data sharing and processing terminal receives data streams from the upper / lower / left / right cameras respectively, based on the different shared signal values, as shown in Figure 1. It forwards the data streams to the TBOX via the RTP protocol, and the TBOX transmits the data to the TBOX of the connected vehicle via the local area network. At the same time, it sends its own real-time positioning data to the TBOX of the connected vehicle via the local area network Ethernet communication.

[0077] 3) After the vehicle's TBOX receives the RTP data stream, it transfers the data stream to the CDC. The CDC parses the data and displays it on the screen. Based on the image displayed on the screen, it also receives the location data of the vehicle being shared. By combining the target vehicle's own location data and the video orientation information, it uses visual SLAM technology to render the geographic information around the two vehicles, marking concave and convex obstacles in prominent colors so that the driver can see them clearly at a glance.

[0078] 4) The rendered geographic location information screen is broadcast and shared to all vehicles via the local area network. After receiving the address location information screens from vehicles A and B, vehicle C will also receive the location data from vehicles A and B, and combine it with its own location data to mark its own location on the location information screen.

[0079] 5) When the shared data stream needs to end, the requesting vehicle / shared vehicle clicks the "Shared Terminal Terminate" button in the CDC to disconnect and terminate the sharing.

[0080]

[0081] By defining different signal values, camera data from other (relative) vehicles at different locations can be transmitted to the target vehicle.

[0082] In some embodiments, an in-vehicle vision blind spot control system is provided, which is used to execute the in-vehicle vision blind spot control method provided in any of the above embodiments. Please refer to Figure 3 This is a schematic diagram of a vehicle-mounted visual blind spot control system provided in an embodiment of this application, which can be applied to a target vehicle. The vehicle-mounted visual blind spot control system includes: The data acquisition module 610 is used to acquire vision compensation data of the target vehicle in the blind spot shared by the relative vehicle through a preset network. The vision compensation data includes the positioning information of the relative vehicle when acquiring real-time data streams. The data parsing module 620 is used to parse the field-of-view compensation data and determine the real-time data stream and positioning information of the relative vehicle. The visual blind spot filling module 630 is used to perform coordinate transformation based on the positioning information of the target vehicle and the relative vehicle to determine the relative pose relationship. Based on the 3D point cloud and the relative pose relationship in the real-time data stream at the same time, it performs 3D reconstruction to obtain a 3D environment model reflecting the road conditions of the target vehicle, so as to complete the vehicle-mounted visual blind spot filling control.

[0083] Based on the above embodiments, the vehicle-mounted visual blind spot control system also includes: a vehicle control module 640, which is used to generate control strategies through a three-dimensional environment model to control the target vehicle.

[0084] Through the above methods, the vehicle-mounted visual blind spot control system of this application acquires the visual field compensation data of the target vehicle in the blind spot area shared by the relative vehicle through a preset network, analyzes the visual field compensation data, determines the real-time data stream and positioning information of the relative vehicle, performs coordinate transformation based on the positioning information of the target vehicle and the relative vehicle to determine the relative pose relationship, and performs three-dimensional reconstruction based on the three-dimensional point cloud and the relative pose relationship in the real-time data stream at the same time to obtain a three-dimensional environment model reflecting the road conditions of the target vehicle, so as to complete the vehicle-mounted visual blind spot control. First, by sharing the perception resources of the target vehicle in real time through a pre-set network, the blind spots of the target vehicle's field of vision are effectively covered, resulting in low cost and low maintenance cost. This effectively avoids damage to the vehicle from small road obstacles and improves the comprehensiveness and reliability of the vehicle's perception in complex road conditions. Second, by achieving spatiotemporal alignment of multi-source data through coordinate transformation and combining it with 3D reconstruction technology to construct a high-precision 3D environment model of the blind spot, decision-making errors caused by road condition perception deviations are avoided. Third, based on the accurate 3D environment model and the real-time status of the vehicle, targeted control strategies are generated to achieve precise adjustment of steering and speed, effectively reducing the risk of accidents such as blind spot collisions and scrapes, and improving vehicle driving safety and ride comfort.

[0085] Specific limitations regarding the vehicle-mounted visual blind spot control system can be found in the limitations of the vehicle-mounted visual blind spot control method described above, and will not be repeated here. Each module in the aforementioned vehicle-mounted visual blind spot control system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the electronic device in hardware form or independent of it, or stored in the memory of the electronic device in software form, so that the processor can call and execute the corresponding operations of each module.

[0086] In this embodiment, the vehicle-mounted visual blind spot control system is essentially set up with multiple modules to execute the vehicle-mounted visual blind spot control method in any of the above embodiments. The specific functions and technical effects can be referred to the above embodiments, and will not be repeated here.

[0087] In one embodiment, an electronic device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, the electronic device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. The computer program is executed by the processor to implement the functions or steps of the server-side method described above.

[0088] In one embodiment, an electronic device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 8 As shown, the electronic device includes a processor, memory, network interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with an external server via a network connection. The computer program is executed by the processor to implement the functions or steps of the client side of the above method.

[0089] In one embodiment, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0090] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0091] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or electronic device described above can be referred to the relevant descriptions in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0092] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), direct memory bus RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0093] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the above-described device or system can be divided into different functional units or modules to complete all or part of the functions described above.

[0094] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A vehicle-mounted visual blind spot filling control method, characterized by, The method is applied to a target vehicle, and the method comprises the following steps: Obtaining field of view compensation data of the target vehicle in a blind area of the field of view shared by a relative vehicle through a preset network, wherein the field of view compensation data comprises positioning information corresponding to the relative vehicle when collecting a real-time data stream; Analyzing the field of view compensation data to determine the real-time data stream of the relative vehicle and the positioning information; Converting coordinates according to the positioning information of the target vehicle and the relative vehicle respectively to determine a relative pose relationship, performing three-dimensional reconstruction based on three-dimensional point clouds in the real-time data stream at the same time and the relative pose relationship to obtain a three-dimensional environment model reflecting the road conditions of the target vehicle, so as to complete the control of the vehicle-mounted visual compensation blind area.

2. The in-vehicle vision blind spot compensation control method of claim 1, wherein, The preset network comprises a local area network and at least a 5G communication network, if the distance between the target vehicle and the relative vehicle is less than a preset communication distance, the vehicles communicate with each other through the local area network; if the distance between the target vehicle and the relative vehicle is greater than the preset communication distance, the vehicles communicate with each other through at least a 5G communication network, and the local area network is a WIFI network or a V2X direct connection network.

3. The in-vehicle vision blind spot compensation control method of claim 1, wherein, Before obtaining the field of view compensation data of the target vehicle in the blind area of the field of view shared by the relative vehicle through the preset network, the method further comprises the following steps: If the target vehicle has a blind area of the field of view, a data sharing request is sent to the relative vehicle to make the relative vehicle analyze the data sharing request, wherein the data sharing request comprises identification information of the target vehicle and environmental perception data required at a preset position, and the environmental perception data comprises image data or / and radar data; 4. The in-vehicle vision blind spot compensation control method of claim 1, wherein, If the relative vehicle agrees to share, real-time environmental perception data required at a preset position is sent to the target vehicle as the field of view compensation data through the preset network. According to the positioning information of the target vehicle and the relative vehicle respectively, the coordinates are converted to determine the relative pose relationship, and based on the three-dimensional point clouds in the real-time data stream at the same time and the relative pose relationship, three-dimensional reconstruction is performed to obtain a three-dimensional environment model reflecting the road conditions of the target vehicle, which comprises the following steps: Obtaining a local coordinate system formed by each vehicle taking itself as the origin, and positioning information of the target vehicle; According to the positioning information of the target vehicle and the relative vehicle respectively, the coordinates are converted to determine the relative pose relationship; A global coordinate system is constructed according to the environment of all vehicles in the preset network, and three-dimensional point clouds of each real-time data stream in the global coordinate system are determined according to the relative pose relationship; 5. The in-vehicle vision blind spot compensation control method of claim 4, wherein, Aligning and fusing each three-dimensional point cloud at the same time to obtain a three-dimensional environment model reflecting the road environment of the target vehicle through global optimization. Before converting coordinates according to the positioning information of the target vehicle and the relative vehicle respectively to determine the relative pose relationship, the method further comprises the following steps: Extracting invariant feature points of continuous image frames in the real-time data stream, matching the invariant feature points at the same time to determine environmental key features; The relative vehicle motion information is determined, and a calibration parameter of an environment acquisition device is determined; a depth calculation is performed on the environment key feature according to the motion information and the calibration parameter; a three-dimensional environment model of the relative vehicle in a local coordinate system is obtained through three-dimensional reconstruction, and each three-dimensional point cloud is derived from the target vehicle or the relative vehicle.

6. The in-vehicle vision blind spot compensation control method of claim 1, wherein, After the vehicle visual blind supplement control is completed, a control strategy is generated through the three-dimensional environment model to control the target vehicle.

7. The in-vehicle vision blind spot compensation control method of claim 6, wherein, The control strategy is generated through the three-dimensional environment model to control the target vehicle, including: If the three-dimensional environment model has a target obstacle, the type of the target obstacle is determined; If the target obstacle in the three-dimensional environment model is a road bump, a first control strategy of deceleration is generated to control the target vehicle; If the target obstacle in the three-dimensional environment model is a front obstacle, a second control strategy of lane change is generated to control the target vehicle; If the target obstacle in the three-dimensional environment model is a rear collision, a third control strategy of acceleration or a fourth control strategy of lane change is generated to control the target vehicle; If the three-dimensional environment model does not have a target obstacle, a fifth control strategy of keeping driving is generated to control the target vehicle.

8. The in-vehicle blind spot view control method according to any one of claims 1 to 6, characterized by, After the vehicle visual blind supplement control is completed, if the target vehicle generates a sharing termination request according to a trigger instruction, the connection of the target vehicle in the preset network is disconnected to exit the information sharing.

9. A vehicle-mounted vision blind-filling control system, characterized by comprising: The system is applied to a target vehicle, and includes: A data acquisition module is configured to acquire visual field compensation data of the target vehicle in a visual field blind area shared by a relative vehicle through a preset network, wherein the visual field compensation data includes positioning information corresponding to the relative vehicle when collecting a real-time data stream; A data analysis module is configured to analyze the visual field compensation data to determine the real-time data stream and the positioning information of the relative vehicle; A visual blind supplement module is configured to perform coordinate conversion according to the positioning information of the target vehicle and the relative vehicle to determine a relative pose relationship, perform three-dimensional reconstruction on three-dimensional point clouds in the real-time data stream and the relative pose relationship at the same time, and obtain a three-dimensional environment model reflecting the road conditions of the target vehicle to complete vehicle visual blind supplement control.

10. A vehicle characterized by comprising: The vehicle adopts the method of any one of claims 1 to 8.

Images