Reversing auxiliary alarm method and related equipment

Abstract

The invention discloses a reversing auxiliary alarm method and related equipment, and relates to the technical field of vehicle auxiliary driving, and the method comprises the steps: obtaining first side video stream data, second side video stream data and wide-angle video stream data of a target vehicle; based on the first image recognition model, determining a first target reference object and first position information of the first target reference object, and a second target reference object and second position information of the second target reference object; calculating a first reference distance and a second reference distance based on the first position information, the second position information, the pre-calibrated reference size and a monocular distance measurement algorithm; determining a to-be-detected obstacle and a pixel position of the to-be-detected obstacle based on the second image recognition model; based on the first reference distance, the second reference distance and the pixel position of the to-be-detected obstacle, determining the actual distance from the to-be-detected obstacle to the vehicle tail; and based on the actual distance and a preset distance range, determining an alarm signal grade and outputting alarm prompt information corresponding to the alarm signal grade.

Description

Technical Field

[0001] The present application relates to the field of vehicle assisted driving technology, and in particular to a reversing auxiliary alarm method and related equipment. Background Art

[0002] With the development of intelligent automotive technology, reversing assistance systems have gradually become an important feature for improving driving safety. Traditional reversing assistance solutions rely on ultrasonic radar or millimeter-wave radar for obstacle detection. Such solutions require the installation of multiple radar sensors on the vehicle body, which has high hardware costs and is susceptible to inclement weather or complex environments, resulting in false alarms or missed alarms. In addition, radar sensors have a limited detection range and blind spots, especially behind the vehicle or in areas with low obstacles, which limits their practical application. Therefore, a reversing assistance alarm method is urgently needed to solve the above-mentioned technical problems. Summary of the Invention

[0003] The Summary of the Invention introduces a series of simplified concepts that will be further described in the Detailed Description of the Invention. The Summary of the Invention of this application is not intended to limit the key features and essential technical features of the claimed technical solution, nor is it intended to determine the scope of protection of the claimed technical solution.

[0004] In a first aspect, the present application provides a reverse assist alarm method, comprising:

[0005] Acquire first side video stream data, second side video stream data, and wide-angle video stream data of a target vehicle;

[0006] Analyzing the first side video stream data and the wide-angle video stream data based on the first image recognition model to determine a first target reference object in a first public field of view area and first position information of the first target reference object, and analyzing the second side video stream data and the wide-angle video stream data to determine a second target reference object in a second public field of view area and second position information of the second target reference object;

[0007] Calculate a first reference distance from the first target reference object to the rear of the vehicle based on the first position information, the pre-calibrated reference size, and the monocular ranging algorithm; and calculate a second reference distance from the second target reference object to the rear of the vehicle based on the second position information, the pre-calibrated reference size, and the monocular ranging algorithm;

[0008] Based on the second image recognition model, the wide-angle video stream data is recognized to determine the obstacle to be detected and the pixel position of the obstacle to be detected in the wide-angle video stream data;

[0009] Determine the actual distance from the obstacle to be detected to the rear of the vehicle based on the first reference distance, the second reference distance, and the pixel position of the obstacle to be detected;

[0010] Based on the actual distance and the preset distance range, the alarm signal level is determined and the alarm prompt information corresponding to the alarm signal level is output.

[0011] In some embodiments, the first image recognition model includes a feature extraction module, a region segmentation module, and a key point detection module. Based on the first image recognition model, the first side video stream data and the wide-angle video stream data are analyzed to determine the first target reference object in the first public field of view area and the first position information of the first target reference object, including:

[0012] Based on the feature extraction module, feature extraction is performed on the first side video stream data and the wide-angle video stream data, and an overlapping area of ​​the field of view of the first electronic rearview mirror lens and the ultra-wide-angle reversing camera is determined as a first common field of view area, wherein the first electronic rearview mirror lens is used to obtain the first side video stream data, and the ultra-wide-angle reversing camera is used to obtain the wide-angle video stream data;

[0013] Based on the region segmentation module, semantic segmentation is performed on the first side video stream data or the wide-angle video stream data of the first public field of view area to generate a segmentation mask containing the first candidate reference object;

[0014] Based on the key point detection module, geometric feature matching is performed on the first candidate reference object in the segmentation mask to determine the bounding box coordinates and spatial position of the first candidate reference object;

[0015] According to the bounding box coordinates and the spatial position, a first candidate reference object that meets the preset geometric constraints is screened out as a first target reference object in the first public field of view area, and first position information of the first target reference object is output.

[0016] In some embodiments, calculating a first reference distance from the first target reference object to the rear of the vehicle based on the first position information, a pre-calibrated reference size, and a monocular ranging algorithm includes:

[0017] generating a first monocular ranging scale coefficient based on a ratio of a pre-calibrated reference size to a pixel size of the pre-calibrated reference size in the first side video stream data or the wide-angle video stream data, wherein the pre-calibrated reference size is a calibrated distance between a front door handle and a rear door handle;

[0018] Based on the first monocular ranging scale coefficient and the pixel distance from the first target reference object to the rear of the vehicle, a first reference distance from the first target reference object to the rear of the vehicle is calculated using a monocular ranging algorithm, wherein the pixel distance is determined based on first position information of the first target reference object.

[0019] In some embodiments, the second image recognition model includes a feature extraction module, a region segmentation module, and a key point detection module. Based on the second image recognition model, the wide-angle video stream data is identified, and the obstacle to be detected and the pixel position of the obstacle to be detected in the wide-angle video stream data are determined, including:

[0020] Based on the feature extraction module, feature extraction is performed on the wide-angle video stream data to generate a target feature map containing candidate obstacle areas;

[0021] Based on the region segmentation module, semantic segmentation is performed on the target feature map to determine the candidate obstacle regions in the wide-angle video stream data and generate segmentation masks for the candidate obstacle regions;

[0022] Based on the key point detection module, geometric key point detection is performed on the candidate obstacle area in the segmentation mask to generate the bounding box coordinates and center point coordinates of the candidate obstacle area;

[0023] According to the bounding box coordinates and the center point coordinates, the candidate obstacle areas that meet the preset confidence threshold are screened out as the obstacles to be detected in the wide-angle video stream data, and the pixel positions of the obstacles to be detected are output.

[0024] In some embodiments, determining the actual distance from the obstacle to be detected to the rear of the vehicle based on the first reference distance, the second reference distance, and the pixel position of the obstacle to be detected includes:

[0025] Calculating a root mean square value of the first reference distance and the second reference distance based on the first reference distance and the second reference distance;

[0026] Generate a baseline scaling factor for the wide-angle video stream data based on the RMS value;

[0027] Determine the pixel distance from the obstacle to the rear of the vehicle based on the pixel position of the rear of the vehicle and the pixel position of the obstacle to be detected in the wide-angle video stream data;

[0028] The pixel distance is proportionally converted according to the reference scale factor to generate the actual distance from the obstacle to be measured to the rear of the vehicle.

[0029] In some embodiments, the preset distance range includes a first preset distance, a second preset distance, and a third preset distance, and the alarm signal level includes a first level alarm signal, a second level alarm signal, and a third level alarm signal. Based on the actual distance and the preset distance range, the alarm signal level is determined and the alarm prompt information corresponding to the alarm signal level is output, including:

[0030] When the actual distance is less than or equal to the first preset distance and greater than the second preset distance, a first-level alarm signal is triggered, the annular light strip at the edge of the display screen is controlled to flash in a first preset mode and a first buzzer prompt message is output, wherein the first preset distance is greater than the second preset distance;

[0031] When the actual distance is less than or equal to the second preset distance and greater than the third preset distance, a secondary alarm signal is triggered, the annular light strip is controlled to flash in a second preset mode and a second buzzer prompt message is output, wherein the second preset distance is greater than the third preset distance;

[0032] When the actual distance is less than or equal to the third preset distance, a third-level alarm signal is triggered, the annular light strip is controlled to be always on and a third buzzer prompt message is output.

[0033] In some embodiments, further comprising:

[0034] Obtaining ambient light status signals and vehicle speed signals;

[0035] determining light compensation parameters based on the ambient light status signal;

[0036] Determine the alarm response time threshold based on the vehicle speed signal;

[0037] Based on the light compensation parameters and the alarm response time threshold, the flashing frequency of the ring light strip and the volume levels of the first beep prompt information, the second beep prompt information, and the third beep prompt information are dynamically adjusted.

[0038] In a second aspect, the present application proposes a reverse assist alarm device, comprising:

[0039] A video stream data acquisition unit, configured to acquire first side video stream data, second side video stream data, and wide-angle video stream data of a target vehicle;

[0040] a target reference object determining unit, which analyzes the first side video stream data and the wide-angle video stream data based on the first image recognition model to determine a first target reference object in a first public field of view area and first position information of the first target reference object, and analyzes the second side video stream data and the wide-angle video stream data to determine a second target reference object in a second public field of view area and second position information of the second target reference object;

[0041] a reference distance generating unit, which calculates a first reference distance from a first target reference object to a rear end of the vehicle based on the first position information, a pre-calibrated reference size, and a monocular ranging algorithm, and calculates a second reference distance from a second target reference object to a rear end of the vehicle based on the second position information, the pre-calibrated reference size, and the monocular ranging algorithm;

[0042] a unit for determining an obstacle to be detected, which identifies the wide-angle video stream data based on the second image recognition model, and determines the obstacle to be detected and the pixel position of the obstacle to be detected in the wide-angle video stream data;

[0043] an obstacle distance determination unit, which determines an actual distance from the obstacle to be detected to the rear of the vehicle based on the first reference distance, the second reference distance, and the pixel position of the obstacle to be detected;

[0044] The alarm signal prompt unit determines the alarm signal level based on the actual distance and the preset distance range and outputs alarm prompt information corresponding to the alarm signal level.

[0045] In a third aspect, an electronic device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to implement the steps of any one of the reversing assistance alarm methods of the first aspect when executing the computer program stored in the memory.

[0046] In a fourth aspect, the present application proposes a computer-readable storage medium having a computer program stored thereon, which implements the reversing assistance alarm method of any one of the first aspects when the computer program is executed by a processor.

[0047] In summary, this application obtains multi-channel video stream data and uses image recognition models to analyze reference objects in the public field of view area, combines pre-calibrated dimensions with monocular ranging algorithms to calculate the reference distance, and then determines the actual distance of the obstacle and the alarm level. This application does not need to rely on reversing radar, which significantly reduces hardware costs; through multi-camera collaboration and root mean square value calibration, it effectively reduces blind spots and improves ranging accuracy; based on the dynamic alarm signal classification mechanism, the alarm prompt form can be adjusted in real time according to the distance to the obstacle, enhancing the driver's perception of dangerous conditions. In addition, this application achieves stable detection in complex environments through the combination of algorithm models and calibrated reference objects, providing an efficient and reliable solution for vehicle reversing safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present description. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0049] Figure 1 A flowchart of a reverse assist alarm method provided in an embodiment of the present application;

[0050] Figure 2 A schematic diagram of the camera arrangement principle provided in an embodiment of the present application;

[0051] Figure 3 A schematic structural diagram of a reverse parking assist alarm device provided in an embodiment of the present application;

[0052] Figure 4A schematic structural diagram of a reverse assist alarm electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or that are inherent to these processes, methods, products or devices. The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present application, not all of the embodiments.

[0054] See also Figure 1 , which is a flowchart of a reverse assist alarm method provided in an embodiment of the present application, may specifically include:

[0055] S110, obtaining first side video stream data, second side video stream data, and wide-angle video stream data of a target vehicle;

[0056] Exemplarily, step S110 collects first side video stream data, second side video stream data, and wide-angle video stream data through the target vehicle's first (left) electronic rearview mirror camera, second (right) electronic rearview mirror camera, and ultra-wide-angle reversing camera, respectively. The left and right electronic rearview mirror cameras use wide-angle lenses (e.g., 120° or more), which, after distortion correction, cover a field of view of approximately 90° to the side and rear of the vehicle; the ultra-wide-angle reversing camera (e.g., 170°) covers a large area behind the vehicle, aiming to minimize the rear blind spot. The three video streams are transmitted to the control module in real time via communication protocols such as LVDS or Ethernet, providing a basic data source for subsequent obstacle detection and ranging.

[0057] The purpose of acquiring video stream data is to collaboratively capture the vehicle's rear and side environments through multiple perspectives, particularly the shared field of view between the left and right electronic rearview mirrors and the ultra-wide-angle backup camera. Data fusion from these areas provides key input for subsequent reference object recognition, baseline distance calculation, and obstacle location, thus supporting the core logic of the entire reverse parking assist warning system.

[0058] S120: Analyze the first side video stream data and the wide-angle video stream data based on the first image recognition model to determine a first target reference object in a first public field of view area and first position information of the first target reference object, and analyze the second side video stream data and the wide-angle video stream data to determine a second target reference object in a second public field of view area and second position information of the second target reference object;

[0059] Exemplarily, step S120 uses a first image recognition model to analyze the first side video stream data (such as the left electronic rearview mirror camera) and the wide-angle video stream data (such as the ultra-wide-angle reversing camera) to determine the first common field of view area where the two fields of view overlap, and screen out the first target reference object with stable geometric features (such as door handles, body boundaries, and other objects of known size). Through feature extraction, semantic segmentation, and key point detection, the model outputs the bounding box coordinates and spatial position information of the first target reference object in the video stream to form the first position information. Similarly, the second side video stream data (such as the right electronic rearview mirror camera) and the wide-angle video stream data are processed in the same way to determine the second target reference object and the second position information in the second common field of view area.

[0060] The core purpose of this step is to provide reliable reference data for subsequent baseline distance calculations. Collaborative analysis of the common field of view ensures consistency in multi-camera data, while the selection of target reference objects is based on preset geometric constraints (such as dimensional stability and positional recognizability) to avoid misselection of dynamic or interfering objects, laying the foundation for the high-precision execution of the monocular ranging algorithm.

[0061] S130, calculating a first reference distance from the first target reference object to the rear of the vehicle based on the first position information, the pre-calibrated reference size, and the monocular ranging algorithm, and calculating a second reference distance from the second target reference object to the rear of the vehicle based on the second position information, the pre-calibrated reference size, and the monocular ranging algorithm;

[0062] Exemplarily, step S130 achieves high-precision calculation of the reference distance by combining a pre-calibrated reference size with a monocular ranging algorithm. Specifically, a pre-calibrated reference size (e.g., the fixed distance between the front and rear door handles) serves as an anchor point for known physical dimensions. A monocular ranging scale factor is generated by analyzing the ratio of the pixel size of the target reference object in the video stream to its actual size. Based on this scale factor and the pixel distance from the target reference object to the rear of the vehicle (determined by the first position information or the second position information), the monocular ranging algorithm is used to map the pixel-level position to the actual physical distance, thereby obtaining the first and second reference distances.

[0063] The core of this step is to ensure the accuracy of the reference distance through multi-camera collaboration and redundant verification of the spatial position of calibration reference objects. For example, the spatial position information of the target reference object in the shared field of view of the left and right electronic rearview mirror cameras and the ultra-wide-angle reversing camera is synchronously calibrated. Combined with statistical methods such as the root mean square (RMS), this further eliminates the perspective error of monocular ranging, providing a reliable benchmark scale for subsequent calculation of the actual obstacle distance.

[0064] S140: Identify the wide-angle video stream data based on the second image recognition model, and determine the obstacle to be detected and the pixel position of the obstacle to be detected in the wide-angle video stream data;

[0065] Exemplarily, step S140 processes the wide-angle video stream data using a second image recognition model to identify potential obstacles in the environment behind the vehicle and locate their pixel positions within the image. Based on feature extraction, region segmentation, and keypoint detection techniques, the second image recognition model selects candidate obstacle regions that meet pre-defined criteria, generates the bounding box coordinates and center point coordinates of the obstacles, and provides precise pixel-level position input for subsequent ranging.

[0066] This step ensures accurate obstacle recognition through multi-level feature analysis and geometric constraint verification, avoiding false detections caused by complex backgrounds or changing lighting. By combining obstacle location with a baseline scale factor, it lays the data foundation for actual distance calculation and alarm logic triggering, supporting the real-time and reliability of the entire reverse assist system.

[0067] S150: Determine the actual distance from the obstacle to be detected to the rear of the vehicle based on the first reference distance, the second reference distance, and the pixel position of the obstacle to be detected;

[0068] Exemplarily, step S150 determines the actual distance from the obstacle to the rear of the vehicle by fusing the reference distances calculated by the left and right cameras with the pixel position of the obstacle captured by the wide-angle camera. Specifically, a reference scaling factor for the wide-angle camera is generated based on the reference distances of reference objects in the left and right common field of view. The pixel distance from the obstacle to the rear of the vehicle in the wide-angle image is then proportionally converted to the actual physical distance, achieving accurate distance measurement.

[0069] This step utilizes a multi-camera collaborative calibration mechanism to dynamically correlate the spatial information of the calibration reference object with the wide-angle camera's real-time image, ensuring the reliability of the ranging results. Combined with a preset baseline scaling factor, the system quickly maps the pixel position of obstacles to their actual distance, providing precise input for subsequent graded alarms, effectively improving reverse parking safety while reducing hardware reliance.

[0070] S160: Determine an alarm signal level based on the actual distance and the preset distance range and output alarm prompt information corresponding to the alarm signal level.

[0071] Exemplarily, step S160 dynamically classifies the warning level by comparing the actual distance to the obstacle with multiple preset distance thresholds. The preset distance range is typically configured as a multi-level interval, with each level corresponding to a different degree of danger. Based on the interval within which the actual distance to the obstacle falls, the corresponding alarm signal level is automatically triggered, ensuring that the driver receives differentiated warning feedback based on the distance change.

[0072] Different warning signal levels correspond to specific visual and auditory prompts. For example, long-range vehicles trigger low-frequency flashing lights and intermittent beeps, medium-range vehicles switch to high-frequency flashing and continuous beeps, and close-range vehicles use steady lights and rapid beeps. These graded warnings allow drivers to intuitively perceive the proximity of obstacles, enabling them to brake or steer promptly to effectively avoid collisions.

[0073] In summary, the embodiment of the present application obtains the first side, second side and wide-angle video stream data of the target vehicle, uses the first image recognition model to determine the target reference object in the public field of view and obtains its position information, combines the pre-calibrated reference size and the monocular ranging algorithm to calculate the reference distance, and then uses the second image recognition model to identify the obstacle to be measured and the pixel position in the wide-angle video stream, determines the actual distance based on the reference distance and the pixel position, and finally outputs a graded alarm prompt information according to the actual distance and the preset range. The reversing auxiliary alarm function can be realized without relying on the reversing radar, which significantly reduces the vehicle hardware cost; the public field of view area is analyzed collaboratively by multiple cameras and the reference distance is calibrated using the root mean square value , effectively reducing blind spots and improving obstacle ranging accuracy; setting multi-level alarm signal levels based on actual distance, and different flashing modes of the ring light strip on the display screen and differentiated output of buzzer prompt information, so that the driver can intuitively perceive the changes in obstacle distance, enhancing the real-time perception of dangerous conditions; at the same time, the flashing frequency of the light strip and the buzzer volume level are dynamically adjusted in combination with the ambient light status signal and the vehicle speed signal, thereby improving the system's adaptability in complex environments; in addition, updating the convolutional neural network data through the OTA component and continuously optimizing the image recognition model can reduce false alarms, missed alarms, and erroneous alarms, providing an efficient, reliable and intelligent solution for vehicle reversing safety.

[0074] For example, the assisted navigation warning system of an embodiment of the present application includes a left electronic rearview mirror display, a right electronic rearview mirror display, an electronic rearview mirror control module, an OTA (Over-the-Air Download) component, a body control module, a left (first) electronic rearview mirror camera, a right (second) electronic rearview mirror camera, an ultra-wide-angle backup camera, an RLS (Road Light Sensor), and an ABS (Anti-lock Braking System). By coordinating multiple video streams with sensor data, the communication architecture of the embodiment of the present application is that the left and right electronic rearview mirror cameras each use a wide-angle lens (greater than or equal to 120°), which, after image correction, covers approximately 90° of the vehicle's side and rear field of view. The video stream is transmitted in real time to the electronic rearview mirror control module via the LVDS (Low-Voltage Differential Signaling) protocol. The ultra-wide-angle backup camera (approximately 170°) is responsible for covering a large area at the rear of the vehicle to minimize blind spots, and its video stream is also transmitted to the control module via LVDS. The electronic rearview mirror control module and the body control module use CANFD (Controller Area Network Flexible Data-Rate) for high-speed communication. The body control module routes the real-time vehicle speed signal provided by the ABS to the electronic rearview mirror control module in a 20ms cycle, and at the same time forwards the ambient light conditions (such as daytime, nighttime, dusk) and weather information (such as rainy days) detected by the RLS sensor for dynamic light compensation and algorithm optimization of the video stream data. In addition, the electronic rearview mirror control module is connected to the OTA component via Ethernet, supporting remote updating of the parameters of the first image recognition model and the second image recognition model. The first image recognition model and the second image recognition model are both convolutional neural network (CNN) models, which continuously improve the obstacle recognition accuracy and reduce the false alarm rate. The system also integrates a LAN (Local Area Network) for auxiliary data transmission between modules to ensure efficient synchronization of multi-sensor information. Through the collaborative design of the above hardware and communication protocols, the system realizes the full-link functions of electronic rearview mirror display, multi-camera fusion ranging, environmental adaptive compensation and remote algorithm upgrade, providing a reliable foundation for a cost-effective reversing assistance solution that does not rely on reversing radar.

[0075] See also Figure 2 , Figure 2The schematic diagram of the camera arrangement principle provided in the embodiment of the present application adopts a multi-camera collaborative architecture to achieve all-round environmental perception. The left electronic rearview mirror camera and the right electronic rearview mirror camera are both configured with wide-angle lenses. After distortion correction, they cover a field of view of approximately 90° on the left and right sides of the vehicle respectively, and transmit the video stream to the electronic rearview mirror control module in real time via LVDS; the ultra-wide-angle reversing camera is deployed at the rear of the vehicle to cover a large area at the rear, significantly reducing the blind spots of the traditional reversing camera (such as Figure 2 (shown in the middle gray area). The fields of view of the left and right electronic rearview mirror cameras partially overlap with those of the ultra-wide-angle backup camera, defined as the left and right public field of view areas, respectively. In the left public field of view, the system uses a first image recognition model to identify a first target reference object with stable geometric features (such as a door handle, vehicle body boundary, or other known objects of known dimensions). A monocular ranging algorithm is then used to calculate the first reference distance from the object to the rear of the vehicle. Similarly, the same process is used to determine the second target reference object and second reference distance in the right public field of view. The pixel position of the obstacle to be detected in the wide-angle video stream captured by the ultra-wide-angle backup camera is combined with the RMS value of the left and right reference distances to generate a reference scaling factor, ultimately achieving a highly accurate mapping of the actual obstacle distance. By complementing the multi-camera fields of view and redundant verification of calibration reference objects, the system effectively overcomes the perspective bias inherent in monocular ranging. Furthermore, utilizing CANFD communication between the electronic rearview mirror control module and the body control module, the system integrates ABS vehicle speed signals and RLS ambient light data in real time, dynamically optimizing the ranging algorithm and alarm response logic. This results in a fully visual backup assistance solution that is independent of radar hardware.

[0076] In some examples, the first image recognition model includes a feature extraction module, a region segmentation module, and a key point detection module. Based on the first image recognition model, the first side video stream data and the wide-angle video stream data are analyzed to determine a first target reference object in the first public field of view area and first position information of the first target reference object, including:

[0077] Based on the feature extraction module, feature extraction is performed on the first side video stream data and the wide-angle video stream data, and an overlapping area of ​​the field of view of the first electronic rearview mirror lens and the ultra-wide-angle reversing camera is determined as a first common field of view area, wherein the first electronic rearview mirror lens is used to obtain the first side video stream data, and the ultra-wide-angle reversing camera is used to obtain the wide-angle video stream data;

[0078] Based on the region segmentation module, semantic segmentation is performed on the first side video stream data or the wide-angle video stream data of the first public field of view area to generate a segmentation mask containing the first candidate reference object;

[0079] Based on the key point detection module, geometric feature matching is performed on the first candidate reference object in the segmentation mask to determine the bounding box coordinates and spatial position of the first candidate reference object;

[0080] According to the bounding box coordinates and the spatial position, a first candidate reference object that meets the preset geometric constraints is screened out as a first target reference object in the first public field of view area, and first position information of the first target reference object is output.

[0081] Exemplarily, the first image recognition model realizes the analysis of the first side video stream data and the wide-angle video stream data through the collaborative work of the feature extraction module, the region segmentation module and the key point detection module, and performs feature extraction on the first side video stream data (collected by the left electronic rearview mirror camera) and the wide-angle video stream data (collected by the ultra-wide-angle reversing camera) based on the feature extraction module. The feature extraction module adopts a pre-trained CNN model to extract multi-scale features in the video stream through multiple layers of convolutional layers and pooling layers, including edge features (such as vehicle body boundaries, lane lines), texture features (such as road surface materials, obstacle surface details) and spatial relationship features (such as the relative position of objects). Among them, the shallow convolution layer captures local detail features, and the deep convolution layer extracts global semantic information to form a multi-dimensional feature map. In view of the heterogeneous perspective differences between the left electronic rearview mirror camera and the ultra-wide-angle reversing camera, the feature extraction module uses an adaptive feature fusion algorithm to correlate the feature maps of the two video streams across perspectives to enhance the consistency of common area features.

[0082] Subsequently, the feature points of the two video streams are matched based on a feature matching algorithm (such as the ORB algorithm). The ORB algorithm extracts corner point features in the video stream through the oFAST key point detector and generates a rotation-invariant rBRIEF binary descriptor to describe the local structural information of the feature points. The similarity of the feature descriptors in the two video streams is calculated by the Hamming distance, and feature point pairs with a matching degree higher than a preset matching threshold (such as 85%) are screened out. According to the coordinate distribution of the matching feature point pairs, a random sampling consistency algorithm is used to estimate the homography matrix, eliminate mismatched point pairs, and achieve precise geometric alignment of the two video streams. Based on the homography matrix, the video stream of the left electronic rearview mirror camera is perspective transformed to align it with the perspective of the ultra-wide-angle reversing camera, and then the overlapping area of ​​the field of view of the two cameras is determined, that is, the first common field of view area.

[0083] The first public field of view area covers the overlapping field of view of the side and rear of the vehicle, which is determined by coordinate mapping and bounding box intersection and union calculation. For example, the field of view of the left electronic rearview mirror camera covers approximately 90° of the left rear side after correction, and the ultra-wide-angle reversing camera covers approximately 170° of the rear of the vehicle. The overlapping area of ​​the two is mapped to the same reference system through coordinate transformation to generate a public field of view mask covering the area from the left rear of the vehicle to the middle of the rear of the vehicle. The video stream data in this area integrates multi-view information, providing high-confidence input data for subsequent reference object screening. Through the above-mentioned feature extraction and alignment process, the field of view fragmentation caused by differences in camera perspectives is effectively overcome, ensuring that the position information of objects in the public field of view has spatial consistency, and providing a reliable reference framework for the monocular ranging algorithm.

[0084] Based on the region segmentation module, semantic segmentation is performed on the first side video stream data or wide-angle video stream data within the first public field of view. The region segmentation module adopts an improved U-Net semantic segmentation network, and its network architecture includes an encoder, a decoder and a jump connection structure. The encoder part extracts multi-scale features through the ResNet-50 backbone network, uses dilated convolution to expand the receptive field, and captures the global contextual information of the side, rear and rear areas of the vehicle; the decoder part gradually upsamples the feature map through the deconvolution layer, combines the jump connection to fuse the shallow detail features and deep semantic features, and realizes pixel-level classification. The network output is a multi-channel probability map, and each channel corresponds to a preset object category (such as the front door handle, the edge of the rearview mirror, the vehicle body boundary, the road surface, dynamic objects, etc.).

[0085] During the model inference phase, the input video stream data is preprocessed (including normalization and resizing to a preset pixel size) and then the initial semantic segmentation result is generated through the U-Net network. In response to the screening requirements of candidate reference objects, the system thresholds the probability map through a post-processing algorithm: only pixel areas belonging to the preset reference object category (such as the front door handle, the body boundary) and with a confidence level higher than the preset confidence threshold (such as 90%) are retained to generate a binary segmentation mask. The segmentation mask smoothes edge noise through morphological operations, and uses the connected domain analysis algorithm to extract independent areas of the candidate reference objects, eliminating interference fragments with an area smaller than the preset minimum threshold (such as 50 pixels).

[0086] The region segmentation module outputs a segmentation mask containing the precise pixel range of candidate reference objects and maps it to the original video stream resolution. This segmentation mask serves as input to the subsequent keypoint detection module, restricting the processing area to candidate reference objects with stable geometric features and known dimensions. This ensures high reliability of the input data for the ranging algorithm and provides an accurate data foundation for geometric feature matching and baseline distance calculation.

[0087] Based on the key point detection module, geometric feature matching is performed on the first candidate reference object in the segmentation mask. The key point detection module uses the SURF (Speeded-Up Robust Features) algorithm, combined with multi-scale spatial extreme value detection and direction normalization technology, to extract the boundary feature points of the candidate reference object. Specifically, the extreme points of the local area of ​​the image are first calculated through the Hessian matrix to screen out feature points with high contrast and stability; then, the main direction is calculated within the neighborhood of the feature point through the Haar wavelet response to generate a rotation-invariant feature descriptor. For the binary area of ​​the candidate reference object in the segmentation mask, the module only performs key point detection on the pixel area covered by the mask to avoid background interference. After the detection is completed, the boundary feature point set of the candidate reference object is output, and its bounding box coordinates (including the upper left and lower right pixel coordinates) and geometric center point coordinates (calculated by the mean of the feature point coordinates) are calculated based on the distribution of the feature points.

[0088] After obtaining the pixel position of the candidate reference object, its pixel position is mapped to the three-dimensional space coordinate system, and the monocular ranging scale factor is calculated based on the pre-calibrated reference size and its pixel size in the video stream. This scale factor represents the actual distance corresponding to the unit pixel under the camera's viewing angle. According to the coordinates of the geometric center point of the candidate reference object and the pixel position of the rear end of the car in the image (determined by the pre-calibrated rear end bounding box), the pixel distances ΔX and ΔY between the two in the horizontal direction (X axis) and the vertical direction (Y axis) are calculated, and the total pixel distance D_pixel = √(ΔX 2 +ΔY 2 Based on the monocular ranging scale factor K, the pixel distance is converted to the actual longitudinal distance D_real = K × D_pixel. Combined with camera parameters (such as camera installation height h and pitch angle θ), the perspective projection model is used to calculate the lateral offset (ΔX_real = K × ΔX) and height coordinate (H = h - (D_real × tanθ)) of the candidate reference object to generate complete spatial position information.

[0089] Based on the bounding box coordinates and spatial position information of the candidate reference objects, a multi-level screening process is performed using preset geometric constraints to ensure the stability, geometric feature consistency, and position repeatability of the target reference objects in a dynamic environment. The specific screening process includes:

[0090] In order to effectively filter out dynamic objects (such as pedestrians, moving vehicles, etc.) and background interference (such as swaying branches on the roadside, fluttering billboards, etc.), a temporal consistency check mechanism is adopted. Specifically, based on the segmentation results of continuous video frames, the position stability and size consistency of the candidate reference object area are calculated. In terms of position stability, the offset of the center point of the candidate reference object between adjacent frames is required to be less than 5 pixels, so as to ensure that the position of the object in the picture is relatively stable rather than changing frequently. In terms of size consistency, the area fluctuation of the specified area is less than 10% to avoid misjudgment due to large changes in object size. Only candidate reference objects that meet the above stability constraints will be retained for subsequent screening.

[0091] Verify the structural features of candidate reference objects through geometric shape constraints. Geometric invariants such as the contour moment and edge straightness of the candidate reference objects need to match the pre-trained shape template. For structural parts with straight line features, the edge straightness error is required to be no more than 15°. For example, the straight edge of the candidate reference object is detected by Hough transform, and its angular deviation from the ideal straight line is calculated. If the deviation exceeds the threshold, it is regarded as an irregular object (such as a bump on the ground or a tree branch) and directly excluded. That is, irregular bumps or depressions in the candidate area that do not meet the shape constraints will be filtered out to ensure that the selected reference object has regular and stable shape features.

[0092] The position of the candidate reference object in the vehicle body coordinate system should remain fixed and reproducible. By performing time series analysis on multi-frame video stream data, the position change rate of the candidate reference object in consecutive frames is calculated. In the corresponding vehicle low-speed reversing scenario, its three-dimensional coordinate change in the vehicle body coordinate system is required to be less than 0.05 m / s, and it must remain visible in at least 80% of the consecutive frames. Once this range is exceeded or the position fluctuation exceeds the set threshold, the object will be excluded. For example, if a candidate reference object suddenly changes position or disappears frequently due to visual occlusion during the reversing process, it is determined to be an unreliable reference object (such as a temporary stop sign) and is filtered out.

[0093] The candidate reference object selected by the above-mentioned triple constraints of stability, shape, and position will be determined as the first target reference object in the first public field of view. Its bounding box coordinates and spatial position information are integrated into the first position information, covering the pixel position in the image coordinate system (including the bounding box coordinates and the center point coordinates) and the physical position in the three-dimensional space coordinate system (such as the longitudinal distance and the lateral offset). Subsequently, the first position information is transmitted to the electronic rearview mirror control module via the CANFD communication protocol, providing accurate reference input for the subsequent monocular ranging algorithm, helping to achieve accurate reversing assist ranging function.

[0094] Similarly, based on the first image recognition model, the second side video stream data (collected by the right electronic rearview mirror camera) and the wide-angle video stream data (collected by the ultra-wide-angle reversing camera) are analyzed to determine the overlapping area of ​​the two fields of view as the second common field of view. The feature extraction module extracts multi-scale features of the two video streams (such as the vehicle body boundary, the rearview mirror edge), and the feature points are matched using the ORB algorithm. The homography matrix is ​​estimated to achieve perspective alignment and generate a second common field of view mask covering the right rear end of the vehicle to the middle of the rear end. The video stream data in the second common field of view is semantically segmented based on the region segmentation module, and the segmentation masks belonging to the pre-calibrated reference object category (such as the front door handle, the vehicle body boundary) with a confidence level higher than 90% are screened out. The independent regions of the candidate reference objects are extracted through morphological operations and connected domain analysis. The key point detection module uses the SURF algorithm to match the geometric features of candidate reference objects, generating bounding box coordinates and spatial position information. Combined with temporal stability verification (center point offset between adjacent frames less than 5 pixels and regional area fluctuation less than 10%) and geometric shape constraints (edge ​​straightness error no more than 15°), it selects a second target reference object that meets the preset conditions. The final output is the second position information of the second target reference object, including the bounding box coordinates and center point coordinates in the image coordinate system, as well as the longitudinal distance and lateral offset in the three-dimensional spatial coordinate system. This provides accurate input for the right-hand reference distance calculation, ensuring redundant verification and system robustness of multi-camera collaborative ranging.

[0095] In some examples, calculating a first reference distance from the first target reference object to the rear of the vehicle based on the first position information, a pre-calibrated reference size, and a monocular ranging algorithm includes:

[0096] generating a first monocular ranging scale coefficient based on a ratio of a pre-calibrated reference size to a pixel size of the pre-calibrated reference size in the first side video stream data or the wide-angle video stream data, wherein the pre-calibrated reference size is a calibrated distance between a front door handle and a rear door handle;

[0097] Based on the first monocular ranging scale coefficient and the pixel distance from the first target reference object to the rear of the vehicle, a first reference distance from the first target reference object to the rear of the vehicle is calculated using a monocular ranging algorithm, wherein the pixel distance is determined based on first position information of the first target reference object.

[0098] Exemplarily, the pre-calibrated reference size is a fixed physical distance between the front door handle and the rear door handle, which is pre-written into the system through vehicle model parameter calibration (for example, the calibration distance is L meters). In the first side video stream data or the wide-angle video stream data, the pixel positions of the front door handle and the rear door handle are extracted through the image recognition model, and the pixel distance between the two in the picture is calculated (for example, the pixel length is P). Based on the ratio of the pre-calibrated reference size L to the pixel distance P, a first monocular ranging scale coefficient (K1=L / P) is generated, which represents the actual physical distance corresponding to the unit pixel (unit: meter / pixel). The generation of the scale coefficient relies on the calibrated sizes of the front and rear door handles as known anchor points to ensure the benchmark accuracy of the ranging algorithm.

[0099] When calculating the first reference distance based on the first monocular ranging scale factor and the pixel distance from the first target reference object to the rear of the vehicle, first extract the center point coordinates (X1, Y1) of the first target reference object in the image coordinate system from the first position information of the first target reference object, and determine the reference point coordinates (X2, Y2) of the rear of the vehicle in the image using the pre-calibrated rear bounding box coordinates. The pixel distance between the two is calculated using the Euclidean distance formula D_pixel = √[(X1-X2) 2 +(Y1-Y2) 2 ]. Then, using the monocular ranging algorithm, the pixel distance D_pixel is multiplied by the first monocular ranging ratio coefficient K1 to obtain the first reference distance D1 = K1 × D_pixel from the first target reference object to the rear of the vehicle. The above calculation process forms a closed-loop ranging process through the dynamic combination of preset calibration parameters and image data, which not only ensures the repeatability of the algorithm, but also avoids the error accumulation caused by perspective deviation or environmental interference, providing technical support for the high-precision ranging of the reversing assistance alarm system. In addition, when there are multiple first target reference objects, the above steps are performed independently for each reference object, and the reference distance from the second target reference object to the rear of the vehicle is calculated separately according to the pixel position information of each reference object.

[0100] Similarly, based on the second position information, the pre-calibrated reference size, and the monocular ranging algorithm, a second reference distance from the second target reference object to the rear of the vehicle is calculated, including:

[0101] Exemplarily, a second monocular ranging scale factor is generated based on the ratio of a pre-calibrated reference size to the pixel size of a second target reference object in the second side video stream or wide-angle video stream. The pre-calibrated reference size is the calibrated distance between the front and rear door handles. The first image recognition model extracts the second position information of the second target reference object, obtains its center point coordinates in the image coordinate system, and calculates the pixel distance between the two using the Euclidean distance formula. Combined with the second monocular ranging scale factor, the pixel distance is mapped to the actual physical distance using a monocular ranging algorithm to generate a second reference distance from the second target reference object to the rear of the vehicle. This process verifies the spatial position of redundant calibration reference objects, ensuring the accuracy of the second reference distance and providing multi-view data support for the subsequent fusion calculation of the actual obstacle distance. Furthermore, when there are multiple second target reference objects, the above steps are performed independently for each reference object, namely, the reference distance from the second target reference object to the rear of the vehicle is calculated separately based on the pixel position information of each reference object.

[0102] In some examples, the second image recognition model includes a feature extraction module, a region segmentation module, and a key point detection module. Based on the second image recognition model, the wide-angle video stream data is recognized, and the obstacle to be detected and the pixel position of the obstacle to be detected in the wide-angle video stream data are determined, including:

[0103] Based on the feature extraction module, feature extraction is performed on the wide-angle video stream data to generate a target feature map containing candidate obstacle areas;

[0104] Based on the region segmentation module, semantic segmentation is performed on the target feature map to determine the candidate obstacle regions in the wide-angle video stream data and generate segmentation masks for the candidate obstacle regions;

[0105] Based on the key point detection module, geometric key point detection is performed on the candidate obstacle area in the segmentation mask to generate the bounding box coordinates and center point coordinates of the candidate obstacle area;

[0106] According to the bounding box coordinates and the center point coordinates, the candidate obstacle areas that meet the preset confidence threshold are screened out as the obstacles to be detected in the wide-angle video stream data, and the pixel positions of the obstacles to be detected are output.

[0107] For example, when extracting features from wide-angle video stream data based on the feature extraction module, the second image recognition model can use the same or different convolutional neural network (CNN) architecture as the first image recognition model (e.g., ResNet, YOLO series, or custom CNN). The feature extraction module uses multiple convolutional and pooling layers to extract multi-scale features from the pixel data in the wide-angle video stream, covering edge features (e.g., obstacle outlines), texture features (e.g., rough walls, smooth floors), and motion features (e.g., optical flow of dynamic objects). For static obstacles, the focus is on extracting stable geometric features; for dynamic obstacles, optical flow features or temporal difference features are also extracted. During the feature extraction process, batch normalization can be used to reduce internal covariate shift, and activation functions (e.g., ReLU) can be used to enhance nonlinear representation capabilities, ultimately generating a target feature map containing candidate obstacle regions. If the second image recognition model is the same as the first, its feature extraction parameters are reused. If they are different, the feature extraction module is trained separately for obstacle detection requirements, for example, to increase feature sensitivity for low obstacles (e.g., curbs, steps) or transparent objects (e.g., glass).

[0108] When semantically segmenting the target feature map using the region segmentation module, the module can use the same U-Net architecture as the first image recognition model, or a modified version (such as DeepLab), but its output categories focus on obstacle-related categories (such as vehicles, pedestrians, fixed obstacles, and ground objects). The model extracts global semantic information through the encoder and performs pixel-level classification through the decoder, outputting a multi-channel probability map, with each channel corresponding to a preset obstacle category. Subsequently, a binary segmentation mask is generated through thresholding (e.g., setting a category confidence threshold greater than or equal to 0.9), retaining only pixel regions belonging to the obstacle category. To eliminate false detections, morphological operations are used to fill holes in the segmentation mask and smooth edges. Connected domain analysis is then used to remove discrete regions with an area smaller than a preset minimum threshold (e.g., 100 pixels, adjusted based on camera resolution). This ultimately generates a segmentation mask that contains the complete candidate obstacle regions. If the region segmentation module of the first image recognition model already covers obstacle categories, the second image recognition model can reuse it. For higher detection accuracy, a separate segmentation model for obstacles can be trained, optimizing the loss function to balance the category imbalance.

[0109] When the keypoint detection module detects geometric keypoints for candidate obstacle regions in the segmentation mask, it can employ the same SURF algorithm, ORB algorithm, or deep learning keypoint detection model (CenterNet) as the first image recognition model. However, the detection targets shift from stable geometric features of fixed reference objects to keypoints of the obstacle's outline (e.g., vertices, inflection points) or center of mass. Taking CenterNet as an example, the coordinates of the obstacle center are predicted using a keypoint heatmap on the feature map, and a bounding box is generated by combining offset and size predictions. Traditional algorithms detect points of interest using the Hessian matrix, calculate their principal orientations, and generate rotation-invariant descriptors. Feature matching is then used to eliminate background noise points, ultimately outputting the bounding box coordinates of the candidate obstacle region. If the keypoint detection module of the first image recognition model is suitable for obstacle outline detection, the second image recognition model can be directly reused. For higher real-time performance, a lightweight keypoint detection algorithm, such as Lightweight OpenPose, can be used to accommodate the computational resource constraints of in-vehicle embedded systems.

[0110] When screening candidate obstacle areas based on bounding box coordinates and center point coordinates, low-confidence areas are filtered out by pre-set confidence thresholds (such as category confidence greater than or equal to 0.85, and bounding box overlap rate IoU greater than or equal to 0.7). The specific screening logic includes category confidence filtering, size filtering, and position filtering. Category confidence filtering only retains candidate areas whose category confidence output by the semantic segmentation module is higher than the threshold; size filtering excludes areas whose bounding box size is less than the preset minimum obstacle size (such as width less than 0.1 meters, height less than 0.1 meters, which can be adjusted according to the actual vehicle model) to avoid false detection of noise; position filtering eliminates candidate areas located in the blurred area at the edge of the wide-angle video stream (such as 5% pixel range at the edge of the picture) to reduce the effect of distortion. Candidate obstacle areas that meet the screening conditions are determined as obstacles to be measured, and their pixel positions are output in the form of bounding box coordinates and center point coordinates for subsequent distance calculations. Unlike the first image recognition model that screens stable reference objects, the second image recognition model focuses more on real-time detection of obstacles. Therefore, a temporal consistency check can be introduced (such as requiring obstacles to be detected in several consecutive frames) to reduce false alarms in dynamic environments. At the same time, the screening strategies of the first and second image recognition models can be independently configured according to the differences in detection targets.

[0111] It should be noted that in the embodiment of the present application, the first image recognition model and the second image recognition model can select the same or different architectures according to actual needs, and the convolutional neural network (CNN) structure, training parameters and functional focus of the two can be set independently; for example, the two can reuse the same feature extraction module, region segmentation module and key point detection module to simplify the system design; different network layers, activation functions or loss functions can also be configured for different needs of reference object recognition and obstacle detection (such as the first image recognition model focuses on the robust extraction of stable geometric features, and the second image recognition model focuses on the real-time detection of dynamic obstacles), and even different types of neural networks can be used (such as the first image recognition model uses ResNet and the second image recognition model uses YOLO), so as to flexibly adapt to the algorithm optimization needs in multiple scenarios while ensuring the ranging accuracy.

[0112] In some examples, determining the actual distance from the obstacle to be detected to the rear of the vehicle based on the first reference distance, the second reference distance, and the pixel position of the obstacle to be detected includes:

[0113] Calculating a root mean square value of the first reference distance and the second reference distance based on the first reference distance and the second reference distance;

[0114] Generate a baseline scaling factor for the wide-angle video stream data based on the RMS value;

[0115] Determine the pixel distance from the obstacle to the rear of the vehicle based on the pixel position of the rear of the vehicle and the pixel position of the obstacle to be detected in the wide-angle video stream data;

[0116] The pixel distance is proportionally converted according to the reference scale factor to generate the actual distance from the obstacle to be measured to the rear of the vehicle.

[0117] For example, when calculating the root mean square value based on the first reference distance and the second reference distance, the first reference distance D1 and the second reference distance D2 calculated by the first image recognition model are first obtained. The root mean square value RMS is calculated based on the root mean square formula, where RMS=√(D1 2 +D2 2 ) / 2, this value is used to balance the potential deviation of the reference distance on the left and right sides, eliminating the error caused by the difference in viewing angle in monocular ranging. For example, if D1 = 2.0 meters and D2 = 2.2 meters, then RMS = √(2.0 2 +2.2 2 ) / 22≈2.102 meters. The RMS calibration can improve the stability and reliability of the reference distance and provide a more accurate benchmark for subsequent distance measurements.

[0118] Based on the RMS value and the pre-calibrated pixel size of the rear of the car in the ultra-wide-angle reversing camera image, when generating the benchmark scale factor of the wide-angle video stream data, the RMS value is used as the benchmark reference of the actual physical distance, and the scale mapping is performed in combination with the pixel size of the pre-calibrated reference object in the wide-angle video stream. Specifically, the pixel size of the pre-calibrated reference object (pre-calibrated rear boundary) in the wide-angle video stream is P ref Pixels, whose actual physical distance is W tail Meters, then the base scale factor K base According to the formula K base =RMS×W tail / P ref This coefficient represents the actual physical distance (unit: meter / pixel) corresponding to the unit pixel in the wide-angle video stream data. For example, if RMS = 2.102 meters, P ref =100 pixels, W tail =1.2 m, then K base ≈0.0252 m / pixel, used to convert pixel distance to actual distance.

[0119] When determining the pixel distance between the obstacle to be detected and the rear of the vehicle based on the wide-angle video stream data, the pixel position of the obstacle to be detected (such as the coordinates of the center point of the bounding box (X0, Y0)) output by the second image recognition model and the pre-calibrated pixel position of the rear of the vehicle (such as the coordinates of the center point of the bottom edge of the rear bounding box (X0, Y0)) are first calculated. r ,Y r )), calculate the pixel distance D between the two pixel =√(X0-X r ) 2 +(Y0-Y r ) 2 For example, if the coordinates of the obstacle center point are (300,400) pixels and the coordinates of the rear reference point are (320,500) pixels, then the pixel distance D pixel =√(300-320) 2 +(400-500) 2 =400+10000=10400≈102 pixels. This pixel distance reflects the relative position of the obstacle and the rear of the vehicle in the image coordinate system.

[0120] When the pixel distance is converted according to the base scale factor, the pixel distance D pixel Proportional coefficient K with reference base Multiply them to get the actual distance D from the obstacle to the rear of the vehicle. real =D pixel ×K base Taking the above data as an example, if D pixel =102 pixels, K base= 0.0252 meters / pixel, then the actual distance Dreal = 102 × 0.0252 = 2.57 meters. This conversion process combines the reference distance from multi-camera collaborative calibration with the real-time pixel data of the wide-angle video stream. It eliminates monocular ranging errors through the root mean square value and uses the reference scale factor to achieve a linear mapping from pixel space to physical space. Ultimately, it generates an accurate actual obstacle distance, providing a reliable basis for reverse assist warnings.

[0121] In some examples, the preset distance range includes a first preset distance, a second preset distance, and a third preset distance, and the alarm signal level includes a first level alarm signal, a second level alarm signal, and a third level alarm signal. Based on the actual distance and the preset distance range, the alarm signal level is determined and the alarm prompt information corresponding to the alarm signal level is output, including:

[0122] When the actual distance is less than or equal to the first preset distance and greater than the second preset distance, a first-level alarm signal is triggered, the annular light strip at the edge of the display screen is controlled to flash in a first preset mode and a first buzzer prompt message is output, wherein the first preset distance is greater than the second preset distance;

[0123] When the actual distance is less than or equal to the second preset distance and greater than the third preset distance, a secondary alarm signal is triggered, the annular light strip is controlled to flash in a second preset mode and a second buzzer prompt message is output, wherein the second preset distance is greater than the third preset distance;

[0124] When the actual distance is less than or equal to the third preset distance, a third-level alarm signal is triggered, the annular light strip is controlled to be always on and a third buzzer prompt message is output.

[0125] For example, the preset distance range is set according to the critical distance for safe reversing of the vehicle, wherein the first preset distance is 1.8 meters, the second preset distance is 1.0 meters, and the third preset distance is 0.5 meters; the preset distance can be adjusted according to the wheelbase of the vehicle model and the camera installation parameters; when the actual distance from the obstacle to be detected to the rear of the vehicle is less than or equal to the first preset distance (for example, 1.8 meters) and greater than the second preset distance (for example, 1.0 meters), a first-level alarm signal is triggered. At this time, the electronic rearview mirror control module sends a command to the body control module via the CANFD protocol to control the annular light strip on the edge of the electronic rearview mirror display screen (such as Figure 2The black area of ​​the display screen flashes slowly (e.g., at a frequency of 1Hz) in a first preset pattern (e.g., green light), and simultaneously triggers the instrument buzzer to output a first beep prompt (e.g., an intermittent beep every 2 seconds at a volume of 40-50 decibels). The first preset pattern uses a combination of color and frequency to visually alert the driver to an obstacle at a relatively long warning distance. This gentle visual and audio prompt reminds the driver to pay attention to the rear environment, avoiding inattention caused by the long distance. The first preset distance is set based on the vehicle's safe braking distance when reversing and the driver's reaction time, ensuring early warning when an obstacle enters this range.

[0126] When the actual distance is further shortened to less than or equal to the second preset distance (for example, 1.0 meters) and greater than the third preset distance (for example, 0.5 meters), the secondary alarm signal is triggered. The electronic rearview mirror control module switches the flashing mode of the annular light strip to the second preset mode (for example, yellow light), and increases the flashing frequency to medium speed (such as 2Hz), while triggering the second buzzer prompt information (for example, continuous buzzing, sounding once every 1 second, with a volume of 50-60 decibels). The second preset mode prompts the driver to be alert and prepare to brake by enhancing the visual flashing frequency and auditory prompt density. Yellow light has a longer wavelength and is highly recognizable under different lighting conditions. The medium-speed flashing mode can avoid visual fatigue and effectively attract attention. The setting of the second preset distance is combined with the minimum safe distance when the vehicle is reversing at low speed to ensure that more urgent warning feedback is provided when obstacles approach.

[0127] When the actual distance is less than or equal to the third preset distance (for example, 0.5 meters), a third-level alarm signal is triggered. At this time, the annular light strip switches to a constant red light, cancels the flashing and maintains a continuous high-brightness display, and triggers the third beep prompt information (for example, a rapid continuous beep, with a frequency of 3Hz and a volume of 60-70 decibels). Red as a warning color has the strongest visual impact. The constant light mode eliminates the visual interference caused by flashing, and the continuous beeping forces the driver to stop immediately. The driver perceives that the obstacle is at a very close and dangerous distance and needs to brake immediately to avoid a collision. The setting of the third preset distance is based on the physical contact threshold between the rear of the vehicle and the obstacle to ensure that the highest level of alarm is provided before the critical point of collision risk.

[0128] This multi-level alarm mechanism precisely maps obstacle distance to warning intensity, avoiding false alarms or delayed responses caused by a single threshold. Drivers can quickly determine the location and severity of obstacles through intuitive changes in the ring light color and beep patterns, enhancing the safety of reverse maneuvers.

[0129] In some instances, this also includes:

[0130] Obtaining ambient light status signals and vehicle speed signals;

[0131] determining light compensation parameters based on the ambient light status signal;

[0132] Determine the alarm response time threshold based on the vehicle speed signal;

[0133] Based on the light compensation parameters and the alarm response time threshold, the flashing frequency of the ring light strip and the volume levels of the first beep prompt information, the second beep prompt information, and the third beep prompt information are dynamically adjusted.

[0134] For example, when determining light compensation parameters based on ambient light status signals, the ambient light intensity, spectral distribution, and weather conditions (such as daytime, nighttime, dusk, and rainy days) are collected in real time through the RLS, and the signals are transmitted to the electronic rearview mirror control module via CANFD communication. The control module matches the corresponding light compensation parameters based on the preset light threshold range (such as light intensity greater than 5000 lux during the day and less than 50 lux at night); in low-light environments, the brightness and contrast adjustment parameters of the video stream data are automatically increased to enhance the visibility of obstacle edge features; in strong light or rainy environments, noise reduction filtering is enabled to reduce the interference of overexposure or water marks on image recognition. The light compensation parameters are applied to the video stream data in real time through the CNN preprocessing layer to ensure the stability of feature extraction under different lighting conditions.

[0135] When determining the alarm response time threshold based on the vehicle speed signal, real-time vehicle speed data is obtained from the ABS and updated every 20ms. Based on the linear relationship between vehicle speed and braking distance, a multi-level response time threshold is preset. When the vehicle speed is less than or equal to 5km / h, the response time threshold is set to 200ms. When the vehicle speed is greater than 5km / h, the response time threshold is set to 200ms - (vehicle speed - 5) × 10ms (for example, at a speed of 10km / h, the response time threshold is 200ms - 5 × 10 = 150ms). The response time threshold controls the delay before the alarm signal is triggered. Higher vehicle speeds result in lower thresholds, allowing more time for braking. Combined with light compensation parameters, the flashing frequency and beep volume of the ring light strip are dynamically adjusted: in low-light or high-speed scenarios, the flashing frequency of the first-level alarm is increased from 1Hz to 1.5Hz, and the beep volume is increased by 5-10 decibels; the second-level alarm frequency is increased from 2Hz to 3Hz, and the volume is increased by 10-15 decibels; the third-level alarm remains on and the volume is increased to 70-80 decibels, compensating for environmental interference by enhancing visual and auditory stimulation to ensure timely driver response.

[0136] See also Figure 3 , is a schematic structural diagram of a reverse parking assist alarm device provided in an embodiment of the present application, comprising:

[0137] The video stream data acquisition unit 21 is used to acquire the first side video stream data, the second side video stream data and the wide-angle video stream data of the target vehicle;

[0138] a target reference object determining unit 22 that analyzes the first side video stream data and the wide-angle video stream data based on the first image recognition model to determine a first target reference object in a first public field of view area and first position information of the first target reference object, and analyzes the second side video stream data and the wide-angle video stream data to determine a second target reference object in a second public field of view area and second position information of the second target reference object;

[0139] The reference distance generating unit 23 calculates a first reference distance from the first target reference object to the rear of the vehicle based on the first position information, the pre-calibrated reference size, and the monocular ranging algorithm, and calculates a second reference distance from the second target reference object to the rear of the vehicle based on the second position information, the pre-calibrated reference size, and the monocular ranging algorithm;

[0140] The obstacle to be detected determining unit 24 identifies the wide-angle video stream data based on the second image recognition model, and determines the obstacle to be detected and the pixel position of the obstacle to be detected in the wide-angle video stream data;

[0141] The obstacle distance determination unit 25 determines the actual distance between the obstacle to be detected and the rear of the vehicle based on the first reference distance, the second reference distance, and the pixel position of the obstacle to be detected;

[0142] The alarm signal prompt unit 26 determines the alarm signal level based on the actual distance and the preset distance range and outputs alarm prompt information corresponding to the alarm signal level.

[0143] See also Figure 4 An embodiment of the present application also provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor. When the processor 320 executes the computer program 311, the steps of any method of reversing assist alarm are implemented.

[0144] Since the electronic device introduced in this embodiment is a device used to implement a reversing auxiliary alarm device in the embodiment of the present application, based on the method introduced in the embodiment of the present application, technical personnel in this field can understand the specific implementation method of the electronic device of this embodiment and its various variations. Therefore, how the electronic device implements the method in the embodiment of the present application is no longer introduced in detail here. As long as the equipment used by technical personnel in this field to implement the method in the embodiment of the present application falls within the scope of protection to be protected by this application.

[0145] During the specific implementation process, when the computer program 311 is executed by the processor, any implementation method of the embodiments corresponding to the first aspect can be implemented.

[0146] It should be noted that, in the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0147] Those skilled in the art will appreciate that the embodiments of the present application may provide methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-readable program code.

[0148] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0149] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0150] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0151] The present application also provides a computer program product, which includes computer software instructions. When the computer software instructions are executed on a processing device, the processing device executes Figure 1The process of a reversing auxiliary alarm method in the corresponding embodiment.

[0152] A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired or wireless means. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium can be a magnetic medium, an optical medium or a semiconductor medium, etc.

[0153] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0154] In the several embodiments provided in this application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0155] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0156] In addition, the functional units in the various embodiments of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The above-mentioned integrated units may be implemented in the form of hardware and / or software functional units.

[0157] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk, or an optical disk.

[0158] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

[0159] Although the preferred embodiments of this specification have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of this specification.

[0160] Obviously, those skilled in the art may make various changes and modifications to this specification without departing from the spirit and scope of this specification. Thus, if such changes and modifications fall within the scope of the claims of this specification and their equivalents, this specification is intended to include such changes and modifications.

Claims

1. A reverse assist alarm method, characterized in that: include: Acquire first side video stream data, second side video stream data, and wide-angle video stream data of a target vehicle; Analyzing the first side video stream data and the wide-angle video stream data based on a first image recognition model to determine a first target reference object in a first public field of view area and first position information of the first target reference object, and analyzing the second side video stream data and the wide-angle video stream data to determine a second target reference object in a second public field of view area and second position information of the second target reference object; Calculate a first reference distance from the first target reference object to the rear of the vehicle based on the first position information, the pre-calibrated reference size, and the monocular ranging algorithm; and calculate a second reference distance from the second target reference object to the rear of the vehicle based on the second position information, the pre-calibrated reference size, and the monocular ranging algorithm; Identify the wide-angle video stream data based on a second image recognition model, and determine an obstacle to be detected in the wide-angle video stream data and a pixel position of the obstacle to be detected; Determining an actual distance from the obstacle to be detected to the rear of the vehicle based on the first reference distance, the second reference distance, and a pixel position of the obstacle to be detected; Based on the actual distance and the preset distance range, an alarm signal level is determined and an alarm prompt information corresponding to the alarm signal level is output.

2. The method according to claim 1, characterized in that The first image recognition model includes a feature extraction module, a region segmentation module, and a key point detection module. The first side video stream data and the wide-angle video stream data are analyzed based on the first image recognition model to determine a first target reference object in a first public field of view area and first position information of the first target reference object, including: Based on the feature extraction module, feature extraction is performed on the first side video stream data and the wide-angle video stream data, and an overlapping area of ​​the field of view of a first electronic rearview mirror lens and an ultra-wide-angle reversing camera is determined as the first common field of view area, wherein the first electronic rearview mirror lens is used to obtain the first side video stream data, and the ultra-wide-angle reversing camera is used to obtain the wide-angle video stream data; Based on the region segmentation module, semantic segmentation is performed on the first side video stream data or the wide-angle video stream data of the first public field of view area to generate a segmentation mask containing a first candidate reference object; Based on the key point detection module, perform geometric feature matching on the first candidate reference object in the segmentation mask to determine the bounding box coordinates and spatial position of the first candidate reference object; According to the bounding box coordinates and the spatial position, a first candidate reference object that meets preset geometric constraints is screened out as a first target reference object in the first public field of view area, and first position information of the first target reference object is output.

3. The method according to claim 1, characterized in that The calculating, based on the first position information, the pre-calibrated reference size, and the monocular ranging algorithm, a first reference distance from the first target reference object to the rear of the vehicle includes: generating a first monocular ranging scale coefficient based on a ratio of a pre-calibrated reference size to a pixel size of the pre-calibrated reference size in the first side video stream data or the wide-angle video stream data, wherein the pre-calibrated reference size is a calibrated distance between a front door handle and a rear door handle; Based on the first monocular ranging scale coefficient and the pixel distance from the first target reference object to the rear of the vehicle, a first reference distance from the first target reference object to the rear of the vehicle is calculated using a monocular ranging algorithm, wherein the pixel distance is determined based on first position information of the first target reference object.

4. The method according to claim 1, wherein The second image recognition model includes a feature extraction module, a region segmentation module, and a key point detection module. The wide-angle video stream data is identified based on the second image recognition model, and the obstacle to be detected and the pixel position of the obstacle to be detected in the wide-angle video stream data are determined, including: Based on the feature extraction module, feature extraction is performed on the wide-angle video stream data to generate a target feature map containing candidate obstacle areas; Based on the region segmentation module, semantic segmentation is performed on the target feature map to determine candidate obstacle regions in the wide-angle video stream data, and a segmentation mask of the candidate obstacle regions is generated; Based on the key point detection module, geometric key point detection is performed on the candidate obstacle area in the segmentation mask to generate the bounding box coordinates and center point coordinates of the candidate obstacle area; According to the bounding box coordinates and the center point coordinates, candidate obstacle regions that meet a preset confidence threshold are screened out as obstacles to be detected in the wide-angle video stream data, and pixel positions of the obstacles to be detected are output.

5. The method according to claim 1, wherein The determining, based on the first reference distance, the second reference distance, and the pixel position of the obstacle to be detected, an actual distance from the obstacle to the rear of the vehicle includes: Calculating a root mean square value of the first reference distance and the second reference distance based on the first reference distance and the second reference distance; generating a reference scaling factor for the wide-angle video stream data based on the root mean square value; Determining a pixel distance from the obstacle to be detected to the rear of the vehicle based on a pixel position of the rear of the vehicle and a pixel position of the obstacle to be detected in the wide-angle video stream data; The pixel distance is proportionally converted according to the reference proportional coefficient to generate the actual distance from the obstacle to be detected to the rear of the vehicle.

6. The method according to claim 1, characterized in that The preset distance range includes a first preset distance, a second preset distance, and a third preset distance, the alarm signal level includes a first-level alarm signal, a second-level alarm signal, and a third-level alarm signal, and determining the alarm signal level based on the actual distance and the preset distance range and outputting the alarm prompt information corresponding to the alarm signal level includes: When the actual distance is less than or equal to the first preset distance and greater than the second preset distance, the first level alarm signal is triggered, the annular light strip at the edge of the display screen is controlled to flash in a first preset mode and a first buzzer prompt message is output, wherein the first preset distance is greater than the second preset distance; When the actual distance is less than or equal to the second preset distance and greater than the third preset distance, the secondary alarm signal is triggered, the annular light strip is controlled to flash in a second preset mode and a second buzzer prompt message is output, wherein the second preset distance is greater than the third preset distance; When the actual distance is less than or equal to the third preset distance, the third-level alarm signal is triggered, the annular light strip is controlled to be always on and a third buzzer prompt message is output.

7. The method according to claim 6, characterized in that Also includes: Obtaining ambient light status signals and vehicle speed signals; determining a light compensation parameter based on the ambient light state signal; determining an alarm response time threshold based on the vehicle speed signal; Based on the light compensation parameter and the alarm response time threshold, the flashing frequency of the annular light strip and the volume levels of the first beep prompt information, the second beep prompt information, and the third beep prompt information are dynamically adjusted.

8. A reverse auxiliary alarm device, characterized in that: include: A video stream data acquisition unit, configured to acquire first side video stream data, second side video stream data, and wide-angle video stream data of a target vehicle; a target reference object determining unit, which analyzes the first side video stream data and the wide-angle video stream data based on a first image recognition model to determine a first target reference object in a first public field of view area and first position information of the first target reference object, and analyzes the second side video stream data and the wide-angle video stream data to determine a second target reference object in a second public field of view area and second position information of the second target reference object; a benchmark reference distance generating unit, which calculates a first benchmark distance from the first target reference object to the rear of the vehicle based on the first position information, a pre-calibrated reference size, and a monocular ranging algorithm, and calculates a second benchmark distance from the second target reference object to the rear of the vehicle based on the second position information, the pre-calibrated reference size, and the monocular ranging algorithm; a detection obstacle determination unit, which identifies the wide-angle video stream data based on a second image recognition model, and determines the detection obstacle and the pixel position of the detection obstacle in the wide-angle video stream data; an obstacle distance determining unit, configured to determine an actual distance from the obstacle to be detected to the rear of the vehicle based on the first reference distance, the second reference distance, and a pixel position of the obstacle to be detected; The alarm signal prompt unit determines an alarm signal level based on the actual distance and a preset distance range and outputs alarm prompt information corresponding to the alarm signal level.

9. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to implement the steps of the reversing assistance alarm method as claimed in any one of claims 1 to 7 when executing the computer program stored in the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the reversing assistance alarm method according to any one of claims 1 to 7 is implemented.

Images