Driving early warning method, device and equipment and storage medium

Abstract

The application discloses a driving prewarning method, device and equipment and a storage medium. The application collects environment data from multiple angles, obtains initial image data, performs image splicing and image matching on the initial image data, and obtains target image data; an overlapping area of the target image data and the initial image data is obtained, and a fusion route is obtained; image fusion is performed through the fusion route, fusion image data is obtained, and the collected images are spliced, matched and fused to obtain more accurate and comprehensive images; a target recognition algorithm is used to perform target recognition on the fusion image data, and each target in the image can be accurately recognized; a ranging algorithm is used to calculate the distance between the recognized target and the vehicle, and an initial distance is obtained; an optimal ranging algorithm is used to optimize the initial distance, and a target distance is obtained; and the vehicle is prewarned based on the target distance, the accuracy of distance calculation is improved, the vehicle can be accurately prewarned, the driver is assisted to drive safely, and the safety factor is improved.

Description

Technical Field

[0001] This invention relates to the field of vehicle technology, and in particular to a driving warning method, device, equipment, and storage medium. Background Technology

[0002] In recent years, the level of intelligence in commercial vehicles has been continuously improving. Before the widespread adoption of related technologies, driving safety in commercial vehicles relied primarily on the driver. Due to the significant differences between commercial vehicles and passenger cars in terms of size, weight, and visibility, drivers initially faced greater safety pressures, and the severity of commercial vehicle accidents was incomparable to that of ordinary passenger vehicles. With the development of sensor technologies such as cameras, the driving safety of commercial vehicles has been greatly improved. Drivers have shifted from traditionally observing the surroundings through rearview mirrors to gaining a preliminary understanding of the surroundings through images captured by cameras, resulting in a certain degree of improvement in safety.

[0003] Existing driving warning systems use cameras to capture images and display them on a screen to warn drivers. However, the warnings are not comprehensive enough. Drivers need to switch between the rearview mirror and the in-vehicle display to understand their surroundings, which distracts them and can easily lead to accidents. Summary of the Invention

[0004] The main objective of this invention is to provide a driving warning method, device, equipment, and storage medium, aiming to solve the technical problem that the existing driving warning technology is not comprehensive enough, leading to safety accidents.

[0005] To achieve the above objectives, the present invention provides a driving warning method, the method comprising the following steps:

[0006] While the vehicle is in motion, environmental data is collected from multiple angles to obtain initial image data;

[0007] The initial image data is stitched together and matched to obtain the target image data;

[0008] Obtain the overlapping area between the target image data and the initial image data to obtain the fusion route;

[0009] The initial image data and the target image data are fused using the fusion route to obtain fused image data;

[0010] The fused image data is used to identify targets using a target recognition algorithm to obtain the identified targets;

[0011] The distance between the identified target and the vehicle is calculated using a ranging algorithm to obtain the initial distance;

[0012] The initial distance is optimized using an optimal ranging algorithm to obtain the target distance;

[0013] The vehicle is given a driving warning based on the target distance.

[0014] Optionally, obtaining the overlapping area between the target image data and the initial image data to obtain the fusion route includes:

[0015] The pixel grayscale values ​​within the overlapping region are obtained based on the overlapping region.

[0016] The pixel grayscale values ​​are compared to obtain the comparison matching degree;

[0017] When the comparison matching degree is greater than the preset matching degree threshold, the pixel points corresponding to the comparison matching degree are regarded as pixel pairs;

[0018] The fusion path is obtained through the pixel pairs.

[0019] Optionally, optimizing the initial distance using a preferred ranging algorithm to obtain the target distance includes:

[0020] Acquire initial distance data collected within three consecutive cycles, wherein the three consecutive cycles include a first cycle, a second cycle, and a third cycle.

[0021] The first average distance for the first period, the second average distance for the second period, and the third average distance for the third period are calculated using the initial distance data.

[0022] The first average distance and the third average distance are compared to obtain the comparison similarity.

[0023] When the comparison similarity is greater than or equal to a preset similarity threshold, the average of the second distances is taken as the target distance.

[0024] Optionally, after comparing the first average distance and the third average distance to obtain the comparison similarity, the method further includes:

[0025] When the comparison similarity is less than a preset similarity threshold, a weighting coefficient is obtained;

[0026] The first weighted distance is calculated based on the weighting coefficient and the average of the second distance.

[0027] Calculate the average of the first average distance and the third average distance to obtain the average distance;

[0028] The second weighted distance is calculated based on the weighting coefficients and the average distance.

[0029] The target distance is obtained by using the first weighted distance and the second weighted distance.

[0030] Optionally, the step of performing target recognition on the fused image data using a target recognition algorithm to obtain the recognized target includes:

[0031] The fused image data is sliced ​​using a target recognition algorithm to obtain slice feature maps;

[0032] The slice feature map is convolved to obtain a convolutional feature map;

[0033] The convolutional feature map is sampled to obtain the corresponding feature vector;

[0034] Based on the feature vectors, the predicted bounding box, predicted target type, and confidence level are obtained;

[0035] Get the initial annotation box;

[0036] Calculate the loss value based on the initial bounding box and the predicted bounding box;

[0037] When the loss value is less than a preset loss threshold and the confidence level is greater than a preset confidence threshold, the recognition target representing the object's location and type is obtained through the predicted bounding box and the predicted target type.

[0038] Optionally, the step of calculating the distance between the identified target and the vehicle using a ranging algorithm to obtain the initial distance includes:

[0039] Obtain the location of the target to be identified, the external parameters of the camera, and the internal parameters of the camera;

[0040] The first coordinates of the identified target in the world coordinate system are obtained based on the position of the identified target.

[0041] Based on the camera's external parameters, a rigid body transformation is performed on the first coordinates to obtain the second coordinates of the target in the camera coordinate system.

[0042] Based on the camera's intrinsic parameters, perspective projection is performed on the second coordinates to obtain the image coordinates of the identified target in the image coordinate system;

[0043] Obtain the pixel width and pixel height, the horizontal length and vertical length of the pixels of the image corresponding to the target being identified;

[0044] The pixel coordinates of the target in the pixel plane are calculated using the horizontal length of the pixel, the vertical length of the pixel, the pixel width and pixel height of the image, and the image coordinates.

[0045] Obtain the vehicle's pixel coordinates in the pixel plane, the camera's height above the ground, and the camera's pitch angle;

[0046] Calculate the coordinate difference based on the vehicle pixel coordinates and the pixel coordinates of the identified target in the pixel plane;

[0047] The distance between the target and the vehicle is calculated using the coordinate difference, the camera's intrinsic parameters, the camera's ground elevation, and the camera's pitch angle to obtain the initial distance.

[0048] Optionally, the step of acquiring initial image data by collecting environmental data from multiple angles while the vehicle is in motion includes:

[0049] Environmental data is collected from multiple angles to obtain raw image data;

[0050] Feature points in the original image data are detected to obtain pixel coordinates;

[0051] Obtain the world coordinates of each point in the raw image data in the world coordinate system;

[0052] Calculate the camera intrinsic parameter matrix and camera extrinsic parameter matrix based on the relationship between pixel coordinates and world coordinates;

[0053] The distorted pixel coordinates are calculated using the camera intrinsic parameter matrix and the camera extrinsic parameter matrix.

[0054] The distortion parameters are calculated using the pixel coordinates and the distorted pixel coordinates.

[0055] The distortion parameters are used to correct the distortion of the original image data to obtain the initial image data.

[0056] Furthermore, to achieve the above objectives, the present invention also proposes a driving warning device, the driving warning device comprising:

[0057] The data acquisition module is used to acquire environmental data from multiple angles while the vehicle is in motion, and to obtain initial image data.

[0058] The processing module is used to perform image stitching and image matching on the initial image data to obtain the target image data;

[0059] The acquisition module is used to acquire the overlapping area between the target image data and the initial image data to obtain the fusion route;

[0060] The fusion module is used to perform image fusion on the initial image data and the target image data through the fusion route to obtain fused image data;

[0061] The recognition module is used to perform target recognition on the fused image data using a target recognition algorithm to obtain the recognized target;

[0062] The calculation module is used to calculate the distance between the identified target and the vehicle using a ranging algorithm to obtain an initial distance;

[0063] The optimization module is used to optimize the initial distance using a preferred ranging algorithm to obtain the target distance;

[0064] The warning module is used to provide driving warnings to the vehicle based on the target distance.

[0065] Furthermore, to achieve the above objectives, the present invention also proposes a driving warning device, which includes: a memory, a processor, and a driving warning program stored in the memory and executable on the processor, wherein the driving warning program is configured to implement the steps of the driving warning method described above.

[0066] In addition, to achieve the above objectives, the present invention also proposes a storage medium storing a driving warning program, wherein the driving warning program, when executed by a processor, implements the steps of the driving warning method as described above.

[0067] This invention collects environmental data from multiple angles while the vehicle is in motion to obtain initial image data; it then performs image stitching and matching on the initial image data to obtain target image data; it acquires the overlapping area between the target image data and the initial image data to obtain a fusion route; it then fuses the initial image data and the target image data using the fusion route to obtain fused image data; it performs image stitching, matching, and fusion on the collected images to obtain a more accurate and comprehensive image; it then uses a target recognition algorithm to identify the target in the fused image data; it can accurately distinguish various targets in the image and determine the type of obstacles around the vehicle; it calculates the distance between the identified target and the vehicle using a ranging algorithm to obtain an initial distance; it then optimizes the initial distance using an optimal ranging algorithm to obtain a target distance; and it provides driving warnings to the vehicle based on the target distance, improving the accuracy of distance calculation and thus providing accurate warnings to assist the driver in safe driving and improving driving safety. Attached Figure Description

[0068] Figure 1 This is a schematic diagram of the structure of a vehicle warning device in the hardware operating environment involved in the embodiments of the present invention;

[0069] Figure 2 This is a flowchart illustrating the first embodiment of the vehicle warning method of the present invention;

[0070] Figure 3 This is a hardware architecture diagram of a driving warning method according to one embodiment of the present invention.

[0071] Figure 4 This is a schematic diagram of the overall process of driving warning in one embodiment of the driving warning method of the present invention;

[0072] Figure 5 This is a flowchart illustrating the second embodiment of the vehicle warning method of the present invention;

[0073] Figure 6 This is a flowchart illustrating the third embodiment of the vehicle warning method of the present invention;

[0074] Figure 7 This is a flowchart illustrating the fourth embodiment of the vehicle warning method of the present invention;

[0075] Figure 8 This is a structural block diagram of the first embodiment of the vehicle warning device of the present invention.

[0076] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0077] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0078] Reference Figure 1 , Figure 1 This is a schematic diagram of the vehicle warning device structure in the hardware operating environment involved in the embodiments of the present invention.

[0079] like Figure 1 As shown, the driving warning device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0080] Those skilled in the art will understand that Figure 1The structure shown does not constitute a limitation on the vehicle warning device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0081] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a vehicle warning program.

[0082] exist Figure 1 In the driving warning device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the driving warning device of the present invention can be set in the driving warning device. The driving warning device calls the driving warning program stored in the memory 1005 through the processor 1001 and executes the driving warning method provided in the embodiment of the present invention.

[0083] This invention provides a driving warning method, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the vehicle warning method of the present invention.

[0084] In this embodiment, the driving warning method includes the following steps:

[0085] Step S10: While the vehicle is in motion, environmental data is collected from multiple angles to obtain initial image data.

[0086] It should be noted that the execution subject in this embodiment is the controller of the driving warning system, but it can also be other devices or controllers that can achieve the same or similar functions. This embodiment does not limit this, and this embodiment takes the controller of the driving warning system as an example for explanation.

[0087] In practice, in order to complete the driving warning, it is necessary to install data collection and warning devices on the vehicle. By installing cameras at the front, rear, left and right of the vehicle body for environmental perception, environmental data can be collected from multiple angles while the vehicle is driving, thereby obtaining initial image data representing multiple angles around the vehicle.

[0088] It should be understood that the vehicle adopts an intelligent cockpit domain, which integrates several cockpit functions. Data processing is performed through the controller in the intelligent cockpit domain, and the vehicle's operating system provides voice alerts via a voice assistant to warn of the surrounding environment upon entering the vehicle. Traditional vehicle warning architectures use ECUs (Electronic Control Units) connected to a bus. Adding new functions requires adding new ECUs, leading to high bus load, decreased transmission efficiency, and poor processing capacity when there are too many functions. In contrast, the intelligent cockpit domain controller in this solution uses a combination of a System-on-Chip (SOC) and a Microcontroller Unit (MCU) chip. When data collected by the camera is transmitted to the domain controller, it is processed by the SOC chip to achieve image data stitching, fusion, recognition, and ranging. Figure 3 As shown, Figure 3 This diagram illustrates the hardware architecture for implementing driving warning in this embodiment. Image data is acquired from multiple angles via a camera and transmitted to the smart cockpit domain. The domain controller of the smart cockpit domain uses a SOC chip and a bus module with an MCU chip to process the image data, obtaining processed data. This processed data is then transmitted to the in-vehicle voice assistant for driving warning. The SOC chip is also connected to a 5G module and a WiFi module, while the MCU chip is connected to a T-BOX module, enabling remote vehicle system upgrades and in-vehicle entertainment features.

[0089] Furthermore, the steps of acquiring environmental data from multiple angles to obtain initial image data while the vehicle is in motion specifically include: acquiring environmental data from multiple angles to obtain raw image data; detecting feature points in the raw image data to obtain pixel coordinates; obtaining the world coordinates of each point in the raw image data in the world coordinate system; calculating the camera intrinsic parameter matrix and camera extrinsic parameter matrix based on the relationship between pixel coordinates and world coordinates; calculating the distorted pixel coordinates using the camera intrinsic parameter matrix and the camera extrinsic parameter matrix; calculating distortion parameters using the pixel coordinates and the distorted pixel coordinates; and correcting the distortion of the raw image data using the distortion parameters to obtain the initial image data.

[0090] It should be understood that the camera in this embodiment can be a fisheye camera, thereby acquiring more comprehensive image data. Since the image data acquired by the fisheye camera requires distortion correction, environmental data can be acquired from multiple angles using cameras located at various positions on the vehicle, resulting in uncorrected image data, i.e., the original image data. After acquiring the original image data, an image detection algorithm can be used to detect feature points in the original image data, obtaining the pixel coordinates of each feature point, i.e., pixel coordinates.u , v Using Zhang Zhengyou's calibration method, the world coordinate system is fixed on this original image data. Therefore, the world coordinate W of any point in the original image data in the world coordinate system is 0. By dividing the original image data, the world coordinates of each point in the original image data in the world coordinate system can be calculated. U , V , W ).

[0091] It should be noted that the relationship between the image coordinate system and the world coordinate system is as follows: sm = A [ RT ] M Where A is the camera's intrinsic parameter, [ RT [This refers to the camera's external parameters.] m For the image coordinate system, M Using the world coordinate system, s The scale factor can be set as needed. The relationship between pixel coordinates and world coordinates can be obtained through the relationship between the image coordinate system and the world coordinate system. s ( u , v )= A [ RT ]( U , V , W Therefore, the camera intrinsic parameter matrix can be calculated based on the relationship between pixel coordinates and world coordinates. A and camera extrinsic parameter matrix [ RT The distorted pixel coordinates are calculated based on the camera's intrinsic and extrinsic parameter matrices. u' , v' ), and through the pixel coordinates ( u , v ) and the distorted pixel coordinates ( u' , v' The distortion parameters are calculated using the following formula:

[0092] u = u' +( u'-u 0)( k 1r 2 + k 2r 4 ), v = v' +( v'-v 0)( k 1r 2 + k 2r 4 (Formula 1)

[0093] In Equation 1, ( u 0, v 0) is the center point of the pixel coordinate system. k 1, k 2, r The distortion parameter can be calculated using Equation 1 above. Therefore, distortion correction can be performed on the camera using the distortion parameter, improving the accuracy of the acquired image data and obtaining initial image data, which is the image data after distortion correction.

[0094] Step S20: Perform image stitching and image matching on the initial image data to obtain the target image data.

[0095] It should be noted that after obtaining the initial image data, cylindrical projection is required to resolve the structural issues between the image and the actual object. For example, if the initial image points are... B ( x , y The projected image points are B' ( x 1, y 1) Obtain the pixel image width C and height H and the camera's horizontal angle θ The camera focal length can be obtained using the following formula 2. f :

[0096] f = C / (2tan( θ / 2))(Formula 2)

[0097] In equation 2 above, θ For the camera's horizontal perspective, C The width of the pixel image is used to obtain the camera's focal length. f The coordinate relationship between the initial image and the projected image is as follows: Equation 3:

[0098]

[0099] (Equation 3)

[0100] The projected image data can be calculated using Equation 3 above. The projected image data is then used as the initial image data. Image stitching and matching are performed on the initial image data. The image stitching and matching uses the SIFT (Scale-invariant feature transform) algorithm. By stitching and matching images of the same scene taken from different angles, a panoramic image is formed.

[0101] Specifically, the SIFT algorithm is used to generate the scale space through Gaussian blur, as shown in Equation 4 below:

[0102] (Equation 4)

[0103] in:

[0104] (Equation 5)

[0105] In equations 4 and 5 above, G( x , y , σ ) represents the Gaussian kernel with varying scale. I ( x , y () represents image coordinates, * represents convolution. σ As a scale factor, L ( x , y , σ ) represents the scale space, using the difference of Gaussians function and the multiples of adjacent scale spaces. k Perform neighborhood extremum detection as shown in Equation 6:

[0106] (Equation 6)

[0107] The extreme points can be calculated using Equation 6 above. These detected extreme points are then selected as feature points, and a feature vector is generated for each feature point. A feature vector with a diameter of 3× is selected around each feature point. σ A neighborhood of size ×16 is generated, and then the neighborhood is rotated as a whole using the feature vector generated in the previous step. Rotation can be achieved through polar coordinate transformation; if the coordinates of a point are (… x , y The coordinates after rotation are () x' , y' The calculation process is shown in Equation 7:

[0108] (Equation 7)

[0109] The stitched image is obtained through Equation 7 above. Since the sampling spaces of the same object taken from different perspectives are in different coordinate systems, it is necessary to convert the different coordinate systems into a single coordinate system. Geometric relationships are then established between images of the same scene. The steps are as follows: First, the spatial relationships between images are identified. The adjacency relationship between images is determined by the number of matching feature points. Then, the pixel correspondence between adjacent images is determined. Features detected from the images are used for image registration, typically using Euclidean distance for matching. Image registration requires estimation of 3... 3. Global homography matrix: The Euclidean distance matching features contain a large number of outliers. The RANSAC (Random Sample Consensus) algorithm is used to remove mismatches. The global homography transformation matrix of the image is as follows: Equation 8:

[0110] (Equation 8)

[0111] In Equation 8, ( x , y ) are the initial image coordinates, ( x' , y' The coordinates of the target image are the coordinates of the stitched and matched images, thus obtaining the target image data.

[0112] Step S30: Obtain the overlapping area between the target image data and the initial image data to obtain the fusion route.

[0113] It should be noted that after obtaining the target image data, image fusion can be performed on the target image data and the initial image data to obtain more data information. Then, the overlapping area between the target image data and the initial image data can be obtained, and the fusion route can be obtained based on the overlapping area.

[0114] Step S40: Perform image fusion on the initial image data and the target image data through the fusion route to obtain fused image data.

[0115] In practice, once the fusion route is obtained, the initial image data and the target image data can be fused using the fusion route to obtain fused image data.

[0116] Step S50: Perform target recognition on the fused image data using a target recognition algorithm to obtain the recognized target.

[0117] In practical implementation, after obtaining the fused image data, targets in the fused image data can be identified using a target recognition algorithm. This algorithm can be YOLOv5, or other target recognition algorithms; this embodiment does not limit this, and will use YOLOv5 as an example. By performing target recognition using the YOLOv5 algorithm, the type and location of the identified target object can be obtained.

[0118] Further, the target recognition steps specifically include: slicing the fused image data using a target recognition algorithm to obtain slice feature maps; convolving the slice feature maps to obtain convolutional feature maps; sampling the convolutional feature maps to obtain corresponding feature vectors; obtaining predicted bounding boxes, predicted target types, and confidence scores based on the feature vectors; obtaining initial bounding boxes; calculating loss values ​​based on the initial bounding boxes and the predicted bounding boxes; and obtaining a recognized target representing the object's location and type using the predicted bounding boxes and the predicted target types when the loss value is less than a preset loss threshold and the confidence score is greater than a preset confidence threshold.

[0119] It should be noted that the YOLOv5 algorithm's network structure includes an input layer, a backbone network, a neck network, and an output layer. The fused image data is preprocessed at the input layer, including adaptive scaling and adaptive anchor calculation, with optimal values ​​calculated during algorithm training. The backbone network consists of four modules: FOCUS, CSP, CBL, and SPP. In the FOCUS structure, the fused image data is sliced ​​to obtain slice feature maps, which are then convolved to obtain convolutional feature maps. The neck network generates a feature pyramid to recognize the relevance of the same object. The output layer samples the convolutional feature maps, using three different sampling ratios to generate corresponding feature vectors. These feature vectors are then used to predict image features, i.e., to generate predicted bounding boxes, predicted object types, and confidence scores.

[0120] Specifically, the intersection-union ratio (IUGR) is used to calculate the loss value. This is achieved by obtaining the initial bounding boxes and then calculating the loss value based on both the initial bounding boxes and the predicted bounding boxes. The calculation is shown in Equation 9:

[0121]

[0122] LOSS =1- IOU ( A , B )

[0123] (Equation 9)

[0124] In equation 9 above, A This is the initial annotation box. B To predict the bounding box, IOU ( A , B () represents intersection, union, and ratio. LOSS This represents the loss value. A smaller loss value indicates a better prediction result. However, the above calculation cannot guarantee all overlapping cases, therefore, a different approach is used. GIOU replace IOU The calculation is as follows: Equation 10:

[0125]

[0126] GIOU ( LOSS )=1- GIOU

[0127] (Equation 10)

[0128] In Equation 10 above, D is the smallest bounding rectangle containing the predicted bounding box and the initial labeled box. The final loss value is calculated using Equation 10 above. GIOU ( LOSS The loss value is compared with a preset loss threshold, which can be set as needed, such as 0.2 or 0.1. This embodiment does not impose any restrictions on this. The confidence threshold can also be set as needed, such as 0.8 or 0.9. When the loss value is less than the preset loss threshold and the confidence level is greater than the preset confidence threshold, the identified target representing the object's location and type is obtained through the predicted bounding box and the predicted target type. When the confidence level is greater than the preset confidence threshold and the loss value is less than the preset loss threshold, the corresponding predicted bounding box is taken as the target's location, and the type of the target object is obtained through the corresponding predicted target type, thereby obtaining the vehicles and pedestrians around the vehicle.

[0129] Step S60: Calculate the distance between the identified target and the vehicle using a ranging algorithm to obtain the initial distance.

[0130] It should be understood that once the target is identified, the distance between the target and the vehicle can be calculated. Specifically, the two-dimensional information of the target's position is transformed into a coordinate system through a ranging algorithm to obtain information about the actual environment, thereby calculating the initial distance.

[0131] Step S70: Optimize the initial distance using an optimal ranging algorithm to obtain the target distance.

[0132] In practice, the accuracy of the measured distance cannot be guaranteed due to the complexity of actual road conditions. Therefore, the optimal distance measurement algorithm can be used to optimize the initial distance, improve the distance accuracy, and obtain the optimized distance, which is the target distance.

[0133] Step S80: Provide a driving warning to the vehicle based on the target distance.

[0134] It should be noted that after obtaining the target distance, it can be compared with a preset distance threshold, such as 5m or 10m. This embodiment does not impose any restrictions on this, and the specific value can be adjusted according to the vehicle model and size. By comparing the target distance with the preset distance threshold, when the target distance is less than the preset distance threshold, a warning message is generated to the vehicle controller. The vehicle's voice assistant then issues a voice warning to the driver, alerting them to potential driving hazards and prompting them to slow down or stop. The warning ends when the vehicle stops. When the target distance is greater than or equal to the preset distance threshold, the controller continues to operate until a hazard requires a warning. The controller stops operating when the car stops or reaches a certain stable speed. In addition, the SOC chip will also handle functions related to WiFi, GPS, virtualization, and LCD display. The intelligent cockpit domain controller also features an MCU for CAN communication and power management, and will also include a 5G module and a T-BOX module for remote vehicle system upgrades and in-vehicle entertainment, enhancing the user experience.

[0135] like Figure 4 As shown, Figure 4 This is a schematic diagram of the overall process of the driving warning system in this embodiment. Initial image data is obtained through image acquisition and filtering. This initial image data is then stitched, fused, and target identified to measure the distance between the target and the vehicle. When the target is within a safe distance, it is determined whether the vehicle is stopped or traveling at a stable speed. If the vehicle is not stopped or not traveling at a stable speed, image acquisition and processing continue. The warning ends when the vehicle stops or travels at a stable speed. When the target is not within a safe distance, a warning is broadcast via the voice assistant on the display screen.

[0136] In this embodiment, environmental data is collected from multiple angles while the vehicle is in motion to obtain initial image data. Image stitching and matching are then performed on the initial image data to obtain target image data. The overlapping area between the target image data and the initial image data is obtained to determine a fusion route. The initial image data and the target image data are then fused using the fusion route to obtain fused image data. Image stitching, matching, and fusion are performed on the collected images to obtain a more accurate and comprehensive image. A target recognition algorithm is used to identify the target in the fused image data, thus identifying the target. This allows for accurate differentiation of various targets in the image and determination of the types of obstacles around the vehicle. A distance measurement algorithm is used to calculate the distance between the identified target and the vehicle to obtain an initial distance. This initial distance is then optimized using a selective distance measurement algorithm to obtain a target distance. Based on this target distance, a driving warning is issued to the vehicle, improving the accuracy of distance calculation and thus providing accurate warnings to assist the driver in safe driving and increasing driving safety.

[0137] refer to Figure 5 , Figure 5 This is a flowchart illustrating the second embodiment of the vehicle warning method of the present invention.

[0138] Based on the first embodiment described above, step S30 of the driving warning method in this embodiment specifically includes:

[0139] Step S301: Obtain the pixel grayscale values ​​within the overlapping area based on the overlapping area.

[0140] It should be noted that after obtaining the overlapping area, the pixel grayscale value within the overlapping area can be obtained. Specifically, in order to improve the transition effect, the average grayscale value of the pixel can be used as the pixel grayscale value.

[0141] Step S302: Compare the pixel grayscale values ​​to obtain the comparison matching degree.

[0142] By comparing the grayscale values ​​of each pixel, the contrast matching degree between two pixels is obtained. The contrast matching degree is calculated as follows: Equation 11:

[0143] (Equation 11)

[0144] In Equation 11, the left side of the equation represents the comparison matching degree. β 1 represents the grayscale value of the first pixel. β 2 represents the grayscale value of the second pixel.

[0145] Step S303: When the comparison matching degree is greater than the preset matching degree threshold, the pixel points corresponding to the comparison matching degree are taken as pixel pairs.

[0146] In practice, a preset matching threshold of 99.999% is used. The comparison matching score is compared with this threshold. When the comparison matching score is greater than the threshold, the first and second pixels corresponding to these scores are considered pixel pairs. These pixel pairs are then aggregated to obtain a pixel pair set. A pixel pair is... φ =( β 1, β 2).

[0147] Step S304: Obtain the fusion route through the pixel pairs.

[0148] It should be understood that the fusion route is L = ( φ 1, φ 2, φ 3, ... φ n This allows us to obtain a fusion path based on pixel pairs, and then fuse the images using the fusion path.

[0149] In this embodiment, the pixel grayscale values ​​within the overlapping region are obtained based on the overlapping region; the pixel grayscale values ​​are compared to obtain a comparison matching degree; when the comparison matching degree is greater than a preset matching degree threshold, the pixel points corresponding to the comparison matching degree are taken as pixel pairs; a fusion route is obtained through the pixel pairs; and the optimal fusion route is obtained by using the comparison matching route selection to improve the image fusion effect.

[0150] refer to Figure 6 , Figure 6 This is a flowchart illustrating the third embodiment of the vehicle warning method of the present invention.

[0151] Based on the first embodiment described above, step S70 of the driving warning method in this embodiment specifically includes:

[0152] Step S701: Obtain initial distance data collected within three consecutive cycles, wherein the three consecutive cycles include a first cycle, a second cycle, and a third cycle.

[0153] It should be noted that by using the three-cycle optimal ranging method to optimize the obtained initial distance data, initial distance data collected within three consecutive cycles can be obtained. The three consecutive cycles include the first cycle, the second cycle, and the third cycle. The initial distance is collected twice in each cycle, and the two ranging results are considered as one cycle.

[0154] Step S702: Calculate the first average distance for the first period, the second average distance for the second period, and the third average distance for the third period using the initial distance data.

[0155] In practice, the three consecutive cycles are respectively T 1. T 2. T 3. The average of the two distance measurements in the first period is the average of the first distance. L 1. The mean of the second distance in the second period is L 2. The mean of the third distance in the third period is L 3.

[0156] Step S703: Compare the first average distance and the third average distance to obtain the comparison similarity.

[0157] It should be understood that the second distance mean, falling between the first and third distance means, has a lower probability of being random and is therefore more reliable. Therefore, the first and third distance means can be compared to obtain a comparison similarity score.

[0158] Step S704: When the comparison similarity is greater than or equal to the preset similarity threshold, the average of the second distance is taken as the target distance.

[0159] It should be noted that the preset similarity threshold is set to 98%. When the comparison similarity is greater than or equal to 98%, it means that the difference between the distances of the three collections is very small. In this case, the average of the second distances will be used as the final target distance for optimization.

[0160] Further, after comparing the first average distance and the third average distance to obtain a comparison similarity, the method further includes: when the comparison similarity is less than a preset similarity threshold, obtaining a weight coefficient; calculating a first weighted distance based on the weight coefficient and the second average distance; calculating the average of the first average distance and the third average distance to obtain a distance average; calculating a second weighted distance based on the weight coefficient and the distance average; and obtaining the target distance through the first weighted distance and the second weighted distance.

[0161] It should be noted that when the comparison similarity is less than the preset similarity threshold of 98%, the distance mean needs to be further calculated, and a weighting coefficient can be set. ω Weighting coefficient ω The range is between (0,1), and it can generally be set above 0.8. The first weighted distance and the second weighted distance are calculated using weighting coefficients. The first weighted distance is calculated as follows: (Equation 12)

[0162] S 1= ω * L 2 (Equation 12)

[0163] In Equation 12, S 1 represents the first weighted distance. L 2 represents the mean of the second distance. ω These are the weighting coefficients.

[0164] The average distance is obtained by using the first and third average distances, and then the second weighted distance is calculated using the average distance and the weighting coefficient, as shown in Equation 13:

[0165] (Equation 13)

[0166] In Equation 13, S 2 represents the first weighted distance. The second weighted distance is obtained by combining the average distance and the weighting coefficient. The target distance can be obtained by adding the first weighted distance and the second weighted distance. Therefore, the target distance is calculated as follows: Equation 14:

[0167] L best = S 1+ S 2 (Equation 14)

[0168] This embodiment acquires initial distance data collected over three consecutive periods, including a first period, a second period, and a third period. The first average distance for the first period, the second average distance for the second period, and the third average distance for the third period are calculated using the initial distance data. The first average distance and the third average distance are compared to obtain a similarity score. When the similarity score is greater than or equal to a preset similarity threshold, the second average distance is used as the target distance. The calculated initial distance is collected over three consecutive periods, and the average distance for each period is calculated and compared to obtain a similarity score. The target distance is selected based on the similarity score. When the similarity score is greater than the preset similarity threshold, the second average distance is used as the target distance. The second average distance is located between the first and third average distances, indicating a lower probability of randomness and higher reference value, thus improving the accuracy of the calculated distance.

[0169] refer to Figure 7 , Figure 7 This is a flowchart illustrating the fourth embodiment of the vehicle warning method of the present invention.

[0170] Based on the first embodiment described above, step S60 of the driving warning method in this embodiment specifically includes:

[0171] Step S601: Obtain the location of the target to be identified, the external parameters of the camera, and the internal parameters of the camera.

[0172] It should be noted that once the target is identified, its position can be obtained, along with the camera's intrinsic and extrinsic parameters. The camera's intrinsic parameters include focal length, principal point coordinates, and distortion parameters, while its extrinsic parameters include rotation matrix and translation vector.

[0173] Step S602: Obtain the first coordinates of the target in the world coordinate system based on the position of the target.

[0174] By identifying the target's location, we obtain the target's first coordinates in the world coordinate system. X x , Y y , Z z ).

[0175] Step S603: Perform a rigid body transformation on the first coordinates based on the camera's extrinsic parameters to obtain the second coordinates of the target in the camera coordinate system.

[0176] It should be noted that the camera's extrinsic parameters include the rotation matrix and translation vector. A rigid body transformation is performed on the first coordinate system using the rotation matrix and translation vector to obtain the second coordinate system of the target in the camera coordinate system. The calculation process is shown in Equation 15 below:

[0177] (Equation 15)

[0178] In Equation 15, ( X x , Y y , Z z Let R be the first coordinate, R be the rotation matrix, and T be the translation vector. X s , Y s , Z s () is the second coordinate in the camera coordinate system.

[0179] Step S604: Perform perspective projection on the second coordinates based on the camera intrinsic parameters to obtain the image coordinates of the identified target in the image coordinate system.

[0180] It should be noted that after obtaining the second coordinate, perspective projection is performed on the second coordinate using the camera's intrinsic parameters to obtain the image coordinates of the target in the image coordinate system. The calculation process is shown in Equation 16 below:

[0181] (Equation 16)

[0182] In Equation 16, ( T x , T y () represents the image coordinates. f The focal length is a parameter in the camera's intrinsic parameters. X s , Y s , Z s () is the second coordinate in the camera coordinate system.

[0183] Step S605: Obtain the pixel width and pixel height, the horizontal length of the pixels and the vertical length of the pixels in the image corresponding to the target to be identified.

[0184] It should be noted that after obtaining the pixel width and pixel height of the image corresponding to the target, half of the pixel width is calculated as the x-coordinate of the center point of the pixel coordinate system, and half of the pixel height is calculated as the y-coordinate of the center point of the pixel coordinate system, thus obtaining the center point of the pixel coordinate system. m' ,n' ).

[0185] Step S606: Calculate the pixel coordinates of the target in the pixel plane using the horizontal length of the pixel, the vertical length of the pixel, the pixel width and pixel height of the image, and the image coordinates.

[0186] In practical implementation, the calculation is as follows: Equation 17:

[0187] (Equation 17)

[0188] In Equation 17, ( m , n () represents the pixel coordinates in the pixel plane. D x The horizontal length of the image's pixels. D y The vertical length of the image pixels is ( ), and the center point of the pixel coordinate system is ( ). m' , n' The pixel coordinates of the target in the pixel plane are calculated using Equation 17 above.

[0189] Step S607: Obtain the vehicle pixel coordinates, camera height above the ground, and camera pitch angle in the pixel plane.

[0190] In practice, since the camera has both a pitch angle of 0 and a certain pitch angle, the camera's pitch angle and the vehicle's pixel coordinates in the pixel plane can be obtained.

[0191] Step S608: Calculate the coordinate difference based on the vehicle pixel coordinates and the pixel coordinates of the identified target in the pixel plane.

[0192] coordinate difference l It can be calculated using the vehicle pixel coordinates and the pixel coordinates of the target in the pixel plane.

[0193] Step S609: Calculate the distance between the target and the vehicle using the coordinate difference, the camera intrinsic parameters, the camera height above the ground, and the camera pitch angle to obtain the initial distance.

[0194] It should be noted that when the camera pitch angle is 0, the calculation process for the initial distance is as follows: Equation 18:

[0195] (Equation 18)

[0196] In Equation 18, d The initial distance, f The focal length is a parameter in the camera's intrinsic parameters. h 1 represents the camera's height above the ground.

[0197] When the camera pitch angle is not 0, the initial distance is calculated as follows: Equation 19:

[0198] (Equation 19)

[0199] In Equation 19, θ The camera's pitch angle is given, and the initial distance data can be calculated using Equation 18 or Equation 19 above.

[0200] This embodiment acquires the position of the target to be identified, camera extrinsic parameters, and camera intrinsic parameters; obtains the first coordinates of the target in the world coordinate system based on the target's position; performs a rigid body transformation on the first coordinates based on the camera extrinsic parameters to obtain the second coordinates of the target in the camera coordinate system; performs perspective projection on the second coordinates based on the camera intrinsic parameters to obtain the image coordinates of the target in the image coordinate system; acquires the pixel width and pixel height, and the horizontal and vertical lengths of the pixels in the image corresponding to the target; calculates the pixel coordinates of the target in the pixel plane using the horizontal length, vertical length, pixel width and height, and image coordinates; acquires the vehicle's pixel coordinates in the pixel plane, camera's ground clearance, and camera's pitch angle; calculates the coordinate difference between the vehicle's pixel coordinates and the target's pixel coordinates in the pixel plane; calculates the distance between the target and the vehicle using the coordinate difference, camera intrinsic parameters, camera's ground clearance, and camera pitch angle to obtain an initial distance. This allows for quick and accurate calculation of the distance between the target and the vehicle, improving the early warning effect.

[0201] Reference Figure 8 , Figure 8 This is a structural block diagram of the first embodiment of the vehicle warning device of the present invention.

[0202] like Figure 8 As shown, the driving warning device proposed in this embodiment of the invention includes:

[0203] The acquisition module 10 is used to acquire environmental data from multiple angles while the vehicle is in motion, and obtain initial image data.

[0204] The processing module 20 is used to perform image stitching and image matching on the initial image data to obtain target image data.

[0205] The acquisition module 30 is used to acquire the overlapping area between the target image data and the initial image data to obtain the fusion route.

[0206] The fusion module 40 is used to perform image fusion on the initial image data and the target image data through the fusion route to obtain fused image data.

[0207] The recognition module 50 is used to perform target recognition on the fused image data through a target recognition algorithm to obtain the recognized target.

[0208] The calculation module 60 is used to calculate the distance between the identified target and the vehicle through a ranging algorithm to obtain an initial distance.

[0209] The optimization module 70 is used to optimize the initial distance using a preferred ranging algorithm to obtain the target distance.

[0210] The warning module 80 is used to provide driving warnings to the vehicle based on the target distance.

[0211] In this embodiment, environmental data is collected from multiple angles while the vehicle is in motion to obtain initial image data. Image stitching and matching are then performed on the initial image data to obtain target image data. The overlapping area between the target image data and the initial image data is obtained to determine a fusion route. The initial image data and the target image data are then fused using the fusion route to obtain fused image data. Image stitching, matching, and fusion are performed on the collected images to obtain a more accurate and comprehensive image. A target recognition algorithm is used to identify the target in the fused image data, thus identifying the target. This allows for accurate differentiation of various targets in the image and determination of the types of obstacles around the vehicle. A distance measurement algorithm is used to calculate the distance between the identified target and the vehicle to obtain an initial distance. This initial distance is then optimized using a selective distance measurement algorithm to obtain a target distance. Based on this target distance, a driving warning is issued to the vehicle, improving the accuracy of distance calculation and thus providing accurate warnings to assist the driver in safe driving and increasing driving safety.

[0212] In one embodiment, the acquisition module 30 is further configured to obtain the pixel grayscale values ​​within the overlapping area based on the overlapping area; compare the pixel grayscale values ​​to obtain a comparison matching degree; when the comparison matching degree is greater than a preset matching degree threshold, take the pixel points corresponding to the comparison matching degree as pixel pairs; and obtain a fusion route through the pixel pairs.

[0213] In one embodiment, the optimization module 70 is further configured to acquire initial distance data collected within three consecutive periods, wherein the three consecutive periods include a first period, a second period, and a third period; calculate a first distance mean for the first period, a second distance mean for the second period, and a third distance mean for the third period using the initial distance data; compare the first distance mean and the third distance mean to obtain a comparison similarity; and when the comparison similarity is greater than or equal to a preset similarity threshold, use the second distance mean as the target distance.

[0214] In one embodiment, the optimization module 70 is further configured to: obtain a weight coefficient when the comparison similarity is less than a preset similarity threshold; calculate a first weight distance based on the weight coefficient and the second distance mean; calculate the average of the first distance mean and the third distance mean to obtain an average distance; calculate a second weight distance based on the weight coefficient and the average distance; and obtain the target distance by using the first weight distance and the second weight distance.

[0215] In one embodiment, the recognition module 50 is further configured to slice the fused image data using a target recognition algorithm to obtain slice feature maps; convolve the slice feature maps to obtain convolutional features; sample the convolutional feature maps to obtain corresponding feature vectors; obtain predicted bounding boxes, predicted target types, and confidence scores based on the feature vectors; obtain initial bounding boxes; calculate loss values ​​based on the initial bounding boxes and the predicted bounding boxes; and when the loss value is less than a preset loss threshold and the confidence score is greater than a preset confidence threshold, obtain a recognition target representing the object's location and type using the predicted bounding boxes and the predicted target type.

[0216] In one embodiment, the calculation module 60 is further configured to: acquire the position of the target to be identified, camera extrinsic parameters, and camera intrinsic parameters; obtain the first coordinates of the target to be identified in the world coordinate system based on the position of the target; perform a rigid body transformation on the first coordinates based on the camera extrinsic parameters to obtain the second coordinates of the target to be identified in the camera coordinate system; perform perspective projection on the second coordinates based on the camera intrinsic parameters to obtain the image coordinates of the target to be identified in the image coordinate system; acquire the pixel width and pixel height, the horizontal length and vertical length of the pixels of the image corresponding to the target to be identified; calculate the pixel coordinates of the target to be identified in the pixel plane using the horizontal length, vertical length, pixel width and pixel height, and image coordinates; acquire the vehicle pixel coordinates, camera ground height, and camera pitch angle in the pixel plane; calculate the coordinate difference between the vehicle pixel coordinates and the target pixel coordinates in the pixel plane; and calculate the distance between the target to be identified and the vehicle using the coordinate difference, the camera intrinsic parameters, the camera ground height, and the camera pitch angle to obtain an initial distance.

[0217] In one embodiment, the acquisition module 10 is further configured to acquire environmental data from multiple angles to obtain raw image data; detect feature points in the raw image data to obtain pixel coordinates; acquire the world coordinates of each point in the raw image data in the world coordinate system; calculate the camera intrinsic parameter matrix and the camera extrinsic parameter matrix based on the relationship between pixel coordinates and world coordinates; calculate the distorted pixel coordinates using the camera intrinsic parameter matrix and the camera extrinsic parameter matrix; calculate distortion parameters using the pixel coordinates and the distorted pixel coordinates; and perform distortion correction on the raw image data using the distortion parameters to obtain initial image data.

[0218] Furthermore, to achieve the above objectives, the present invention also proposes a driving warning device, which includes: a memory, a processor, and a driving warning program stored in the memory and executable on the processor, wherein the driving warning program is configured to implement the steps of the driving warning method described above.

[0219] Since this vehicle warning device adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be repeated here.

[0220] Furthermore, this embodiment of the invention also proposes a storage medium storing a driving warning program, which, when executed by a processor, implements the steps of the driving warning method described above.

[0221] Since this storage medium adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be repeated here.

[0222] It should be understood that the above are merely illustrative examples and do not constitute any limitation on the technical solutions of the present invention. In specific applications, those skilled in the art can make settings as needed, and the present invention does not impose any restrictions on this.

[0223] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this invention. In practical applications, those skilled in the art can select some or all of the workflow to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0224] In addition, for technical details not described in detail in this embodiment, please refer to the driving warning method provided in any embodiment of the present invention, which will not be repeated here.

[0225] Furthermore, it should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0226] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0227] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0228] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A driving warning method, characterized in that, The driving warning method includes: While the vehicle is in motion, environmental data is collected from multiple angles to obtain initial image data; The initial image data is stitched together and matched to obtain the target image data; Obtain the overlapping area between the target image data and the initial image data to obtain the fusion route; The initial image data and the target image data are fused using the fusion route to obtain fused image data; The fused image data is used to identify targets using a target recognition algorithm to obtain the identified targets; The distance between the identified target and the vehicle is calculated using a ranging algorithm to obtain the initial distance; The initial distance is optimized using an optimal ranging algorithm to obtain the target distance; Vehicle driving warnings are issued based on the target distance; The step of optimizing the initial distance using a preferred ranging algorithm to obtain the target distance includes: Acquire initial distance data collected within three consecutive cycles, wherein the three consecutive cycles include a first cycle, a second cycle, and a third cycle. The first average distance for the first period, the second average distance for the second period, and the third average distance for the third period are calculated using the initial distance data. The first average distance and the third average distance are compared to obtain the comparison similarity. When the comparison similarity is greater than or equal to a preset similarity threshold, the average of the second distances is taken as the target distance; After comparing the first average distance and the third average distance to obtain the comparison similarity, the method further includes: When the comparison similarity is less than a preset similarity threshold, a weighting coefficient is obtained; The first weighted distance is calculated based on the weighting coefficient and the average of the second distance. Calculate the average of the first average distance and the third average distance to obtain the average distance; The second weighted distance is calculated based on the weighting coefficients and the average distance. The target distance is obtained by using the first weighted distance and the second weighted distance, wherein the first weighted distance is calculated as follows: S 1= ω * L 2, S 1 represents the first weighted distance. L 2 represents the mean of the second distance. ω The first weight is the weighting coefficient; the second weight distance is calculated as follows: , S 2 represents the first weighted distance, L1 represents the first average distance, and L3 represents the third average distance. The target distance is calculated as follows: L best = S 1+ S 2.

2. The driving warning method as described in claim 1, characterized in that, The step of obtaining the overlapping area between the target image data and the initial image data to obtain the fusion route includes: The pixel grayscale values ​​within the overlapping region are obtained based on the overlapping region. The pixel grayscale values ​​are compared to obtain the comparison matching degree; When the comparison matching degree is greater than the preset matching degree threshold, the pixel points corresponding to the comparison matching degree are regarded as pixel pairs; The fusion path is obtained through the pixel pairs.

3. The driving warning method as described in claim 1, characterized in that, The step of performing target recognition on the fused image data using a target recognition algorithm to obtain the recognized target includes: The fused image data is sliced ​​using a target recognition algorithm to obtain slice feature maps; The slice feature map is convolved to obtain a convolutional feature map; The convolutional feature map is sampled to obtain the corresponding feature vector; Based on the feature vectors, the predicted bounding box, predicted target type, and confidence level are obtained; Get the initial annotation box; Calculate the loss value based on the initial bounding box and the predicted bounding box; When the loss value is less than a preset loss threshold and the confidence level is greater than a preset confidence threshold, the recognition target representing the object's location and type is obtained through the predicted bounding box and the predicted target type.

4. The driving warning method as described in claim 1, characterized in that, The step of calculating the distance between the identified target and the vehicle using a ranging algorithm to obtain the initial distance includes: Obtain the location of the target to be identified, the external parameters of the camera, and the internal parameters of the camera; The first coordinates of the identified target in the world coordinate system are obtained based on the position of the identified target. Based on the camera's external parameters, a rigid body transformation is performed on the first coordinates to obtain the second coordinates of the target in the camera coordinate system. Based on the camera's intrinsic parameters, perspective projection is performed on the second coordinates to obtain the image coordinates of the identified target in the image coordinate system; Obtain the pixel width and pixel height, the horizontal length and vertical length of the pixels in the image corresponding to the target being identified; The pixel coordinates of the target in the pixel plane are calculated using the horizontal length of the pixel, the vertical length of the pixel, the pixel width and pixel height of the image, and the image coordinates. Obtain the vehicle's pixel coordinates in the pixel plane, the camera's height above the ground, and the camera's pitch angle; Calculate the coordinate difference based on the vehicle pixel coordinates and the pixel coordinates of the identified target in the pixel plane; The distance between the target and the vehicle is calculated using the coordinate difference, the camera's intrinsic parameters, the camera's ground elevation, and the camera's pitch angle to obtain the initial distance.

5. The driving warning method as described in any one of claims 1 to 4, characterized in that, The process of collecting environmental data from multiple angles while the vehicle is in motion to obtain initial image data includes: Environmental data is collected from multiple angles to obtain raw image data; Feature points in the original image data are detected to obtain pixel coordinates; Obtain the world coordinates of each point in the raw image data in the world coordinate system; Calculate the camera intrinsic parameter matrix and camera extrinsic parameter matrix based on the relationship between pixel coordinates and world coordinates; The distorted pixel coordinates are calculated using the camera intrinsic parameter matrix and the camera extrinsic parameter matrix. The distortion parameters are calculated using the pixel coordinates and the distorted pixel coordinates. The distortion parameters are used to correct the distortion of the original image data to obtain the initial image data.

6. A driving warning device, characterized in that, The driving warning device includes: The data acquisition module is used to acquire environmental data from multiple angles while the vehicle is in motion, and to obtain initial image data. The processing module is used to perform image stitching and image matching on the initial image data to obtain the target image data; The acquisition module is used to acquire the overlapping area between the target image data and the initial image data to obtain the fusion route; The fusion module is used to perform image fusion on the initial image data and the target image data through the fusion route to obtain fused image data; The recognition module is used to perform target recognition on the fused image data using a target recognition algorithm to obtain the recognized target; The calculation module is used to calculate the distance between the identified target and the vehicle using a ranging algorithm to obtain an initial distance; The optimization module is used to optimize the initial distance using a preferred ranging algorithm to obtain the target distance; The early warning module is used to provide driving warnings to the vehicle based on the target distance; The optimization module is further configured to acquire initial distance data collected over three consecutive periods, wherein the three consecutive periods include a first period, a second period, and a third period; calculate the first average distance value of the first period, the second average distance value of the second period, and the third average distance value of the third period using the initial distance data; compare the first average distance value and the third average distance value to obtain a comparison similarity; when the comparison similarity is greater than or equal to a preset similarity threshold, use the second average distance value as the target distance; The optimization module is further configured to: obtain a weight coefficient when the comparison similarity is less than a preset similarity threshold; calculate a first weighted distance based on the weight coefficient and the second average distance; calculate the average of the first average distance and the third average distance to obtain an average distance; calculate a second weighted distance based on the weight coefficient and the average distance; and obtain the target distance using the first weighted distance and the second weighted distance, wherein the first weighted distance is calculated as follows: S 1= ω * L 2, S 1 represents the first weighted distance. L 2 represents the mean of the second distance. ω The first weight is the weighting coefficient; the second weight distance is calculated as follows: , S 2 represents the first weighted distance, L1 represents the first average distance, and L3 represents the third average distance. The target distance is calculated as follows: L best = S 1+ S 2.

7. A vehicle warning device, characterized in that, The driving warning device includes: a memory, a processor, and a driving warning program stored in the memory and executable on the processor, the driving warning program being configured to implement the driving warning method as described in any one of claims 1 to 5.

8. A storage medium, characterized in that, The storage medium stores a driving warning program, which, when executed by a processor, implements the driving warning method as described in any one of claims 1 to 5.

Images