Collision early warning method and device, electronic equipment, readable storage medium and program product

Abstract

The invention relates to a collision early warning method and device, electronic equipment, a readable storage medium and a program product. Target detection and tracking are carried out based on image information collected by a vehicle camera, target detection and tracking information is obtained, a second distance between a communication device and the vehicle is obtained based on communication device information, collected by the vehicle, in the environment, matching is carried out on the target object and the communication device according to the first distance and the second distance, and when matching succeeds, the communication device is started. And carrying out fusion processing on the first distance and the second distance to determine a target distance between the target object and the vehicle, and carrying out collision early warning on the vehicle according to the safety early warning area and the target moving track and the target distance of the successfully matched target object. The distance measurement accuracy can be improved due to the fact that the distance measurement of the image information and the distance measurement of the communication equipment information are fused in the target distance, and the early warning accuracy can be improved due to the fact that the safety early warning area of the vehicle is considered in the collision early warning process.

Description

Technical Field

[0001] The present application relates to the field of information processing technology, and in particular to a collision warning method, device, electronic device, readable storage medium and program product. Background Art

[0002] With the rapid economic development, cars and bicycles have become the main means of travel for people, but the complex traffic environment has led to a rapid increase in the incidence of traffic accidents. Especially in the face of "ghost head-on" scenarios (i.e., there are vehicles or obstacles blocking the line of sight in front of the vehicle, that is, the blind spot of vision, and a non-motor vehicle or pedestrian suddenly jumps out from the roadside, causing the driver to avoid it in time), most drivers are unable to respond quickly, resulting in an extremely high accident casualty rate, because the demand for collision warning systems for such scenarios is increasing day by day.

[0003] In traditional technologies, collision warning algorithms and strategies can effectively reduce the accident rate. However, traditional warning strategies are too sensitive or too slow to produce accurate warning results for complex and changing scenarios. Summary of the invention

[0004] Based on this, it is necessary to provide a collision warning method, device, computer equipment, computer-readable storage medium and computer program product that can improve the accuracy of the warning in response to the above-mentioned technical problems.

[0005] In a first aspect, the present application provides a collision warning method, comprising:

[0006] Performing target detection and tracking based on image information collected by a vehicle camera, obtaining a first distance between a target object and the vehicle, a target running trajectory of the target object, and determining a safety warning area of ​​the vehicle;

[0007] Based on the communication device information in the environment collected by the vehicle, obtaining a second distance between the communication device and the vehicle;

[0008] Matching the target object and the communication device according to the first distance and the second distance, and when it is determined that the match is successful, fusing the first distance and the second distance to obtain a target distance between the successfully matched target object and the vehicle;

[0009] A collision warning is performed on the vehicle according to the safety warning area, the target running trajectory of the successfully matched target object, and the target distance.

[0010] In one embodiment, the safety warning area includes a first area and a second area; the collision warning of the vehicle based on the safety warning area, the target running trajectory of the successfully matched target object, and the target distance includes: determining the corresponding safety warning area and the corresponding weight coefficient based on the target running trajectory of the successfully matched target object; determining the warning parameters corresponding to the corresponding safety warning area and the duration of being in the corresponding safety warning area; determining the collision warning value of the vehicle based on the weight coefficient, the warning parameters, the duration, and the target distance; and providing a warning prompt based on the collision warning value.

[0011] In one of the embodiments, the warning prompt based on the collision warning value includes: when it is determined that the collision warning value is greater than or equal to a preset upper warning threshold, prompting that there is a collision hazard; when it is determined that the collision warning value is greater than or equal to a preset lower warning threshold and less than a preset upper warning threshold, prompting to pay attention to the collision risk; when it is determined that the collision warning value is less than the preset lower warning threshold, prompting safety.

[0012] In one embodiment, the collision warning value of the vehicle is determined according to the weight coefficient, the warning parameter, the duration and the target distance, including: F = αR i *t+D i , where F is the collision warning value, R i is the warning parameter corresponding to the safety warning area, t is the time in the safety warning area, D i is the target distance at time i, and α is the weight coefficient, which changes according to the motion trend of the target trajectory of the successfully matched target object, specifically including:

[0013]

[0014] Among them, p0 and p1 are the position information of the successfully matched target object at different times in the target running trajectory.

[0015] In one embodiment, the fusing the first distance and the second distance to obtain the target distance between the successfully matched target object and the vehicle includes:

[0016] D i =β1A i +β2B i

[0017] Among them, D i is the target distance at time i, A i is the first distance at time i, B iis the second distance at time i, β1 and β2 are weight coefficients, and their sum is 1.

[0018] In one of the embodiments, the second distance between the communication device and the vehicle is obtained based on the communication device information in the environment collected by the vehicle, including: collecting the communication device information in the environment of the vehicle based on the communication sensing device in the vehicle, the communication device information including the communication device and the corresponding signal strength RSSI value; filtering the signal strength RSSI value of the communication device using a Dixon test filtering algorithm; converting the RSSI data after filtering into distance data using a logarithmic fitting algorithm; and determining the distance data as the second distance between the communication device and the vehicle.

[0019] In a second aspect, the present application provides a collision warning device, the device comprising:

[0020] An image processing module, used to perform target detection and tracking based on image information collected by a vehicle camera, obtain a first distance between a target object and the vehicle, a target running trajectory of the target object, and determine a safety warning area of ​​the vehicle;

[0021] A communication processing module, configured to obtain a second distance between the communication device and the vehicle based on the communication device information in the environment collected by the vehicle;

[0022] a distance fusion module, configured to match the target object and the communication device according to the first distance and the second distance, and when it is determined that the match is successful, fuse the first distance and the second distance to obtain a target distance between the successfully matched target object and the vehicle;

[0023] The warning module is used to provide a collision warning to the vehicle according to the safety warning area, the target running track of the successfully matched target object and the target distance.

[0024] In a third aspect, the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method described in the first aspect when executing the computer program.

[0025] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in the first aspect above.

[0026] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method described in the first aspect above.

[0027] In the above-mentioned collision warning method, device, computer equipment, computer-readable storage medium and computer program product, the vehicle-mounted equipment performs target detection and tracking based on the image information collected by the vehicle camera, obtains target detection and tracking information, obtains the second distance between the communication equipment and the vehicle based on the communication equipment information in the environment collected by the vehicle, matches the target object and the communication equipment according to the first distance and the second distance, and when it is determined that the match is successful, fuses the first distance and the second distance to obtain the target distance between the successfully matched target object and the vehicle, and performs a collision warning on the vehicle according to the safety warning area, and the target running track and target distance of the successfully matched target object. Since the target distance fuses the ranging of the image information and the ranging of the communication equipment information, the accuracy of the ranging can be improved, and since the safety warning area of ​​the vehicle, the target running track and target distance of the successfully matched target object are taken into consideration during the collision warning, the accuracy of the warning can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments of the present application or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 A diagram showing an application environment of a collision warning method in an embodiment;

[0030] Figure 2 A schematic diagram of a flow chart of a collision warning method in an embodiment;

[0031] Figure 3 A schematic diagram of a flow chart of a collision warning method in another embodiment;

[0032] Figure 4 is a structural block diagram of a collision warning device in an embodiment;

[0033] Figure 5 FIG. 4 is a diagram showing the internal structure of an electronic device in one embodiment. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0035] Since traditional collision warning strategies cannot output relatively accurate warning results for complex and changing scenarios, drivers have low trust in the warning system. In addition, traditional collision warning strategies rely only on single information, which leads to a large impact on the environment. For example, visual information is affected by factors such as weather, lighting, and occlusion, while communication information is affected by factors such as obstacles and signal interference. Relying on single information will lead to poor system robustness and inability to output accurate warning results.

[0036] Based on this, an embodiment of the present application provides a collision warning method, which realizes intelligent collision warning based on multi-information fusion, and adopts a multi-input parameter model to solve the current problem that warning results cannot be output quickly and accurately for sudden dangerous scenarios such as "ghosting". The system can more intelligently handle various complex and changeable scenarios encountered during driving to adapt to various actual environments.

[0037] The collision warning method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablets, vehicle-mounted devices, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, projection devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be an independent physical server, or it can be a server cluster or distributed system composed of multiple physical servers, or it can be a cloud server that provides cloud computing services.

[0038] In one embodiment, Figure 2 As shown, a collision warning method is provided. This embodiment uses the method applied to a vehicle-mounted device as an example for illustration, and may include the following steps:

[0039] Step 202 , performing target detection and tracking based on the image information collected by the vehicle camera to obtain target detection and tracking information.

[0040] Among them, object detection is an important task in the field of computer vision, which aims to detect target objects from images or videos. Tracking refers to the process of continuously monitoring and locating the position of one or more target objects in a sequence of video frames. Object detection focuses on determining and locating objects in a single image or video frame, while object tracking focuses on tracking the dynamic changes of objects in a video sequence.

[0041] The target detection and tracking information is related information obtained by performing target detection and tracking based on image information collected by the vehicle's camera, including the target object located through target detection, and the first distance between the target object and the vehicle obtained based on ranging of the target object, the target movement trajectory of the target object determined based on target tracking of the target object, and the safety warning area of ​​the vehicle determined by performing target detection based on image information.

[0042] In this embodiment, the vehicle can collect image information in real time through the camera during driving, and the on-board equipment can perform target detection and tracking based on the image information collected in real time by the vehicle camera, thereby obtaining the first distance between the target object and the vehicle, the target running trajectory of the target object, and determining the safety warning area of ​​the vehicle.

[0043] Step 204: Acquire a second distance between the communication device and the vehicle based on the communication device information in the environment collected by the vehicle.

[0044] The communication device information may include the communication device and the corresponding signal strength RSSI (Received Signal Strength Indication, ie, signal strength, which refers to a received signal strength indicator, usually used to measure and indicate the strength of a wireless signal received by a receiver) value.

[0045] In this embodiment, the vehicle may also collect communication device information in the environment while driving, and the vehicle-mounted terminal may also process the communication device information in the environment collected by the vehicle to obtain a second distance between the communication device and the vehicle.

[0046] Step 206 , matching the target object and the communication device according to the first distance and the second distance to obtain a target distance between the successfully matched target object and the vehicle.

[0047] Matching is the process of combining a target object and a communication device at the same location. For example, the target object may be a pedestrian, and the communication device that matches it is the communication device carried by the pedestrian, such as a mobile phone, a phone watch, a tablet computer, etc.

[0048] In this embodiment, the vehicle-mounted device can match the target object and the communication device based on the first distance between the target object and the vehicle and the second distance between the communication device and the vehicle. For example, if the distance difference between the two is less than a certain distance threshold, it can be considered that the two are matched successfully. And when it is determined that the match is successful, the first distance and the second distance are fused to obtain the target distance between the successfully matched target object and the vehicle.

[0049] Specifically, the fusion processing can be a weighted sum of two distances, namely the first distance and the second distance, based on a certain weight, so that the final target distance is a distance measurement that combines image information and communication device information, thereby improving the accuracy of distance measurement.

[0050] Step 208 , providing a collision warning to the vehicle based on the safety warning area, and the target running track and target distance of the successfully matched target object.

[0051] Among them, collision warning is a strategy to warn the driver when it is determined that there is a potential collision risk between the vehicle and the target object. In this embodiment, the vehicle-mounted device can issue a collision warning to the vehicle based on the safety warning area determined above, as well as the target running trajectory and target distance of the successfully matched target object, thereby avoiding safety accidents.

[0052] In the above-mentioned collision warning method, the vehicle-mounted device performs target detection and tracking based on the image information collected by the vehicle camera, obtains target detection and tracking information, obtains the second distance between the communication device and the vehicle based on the communication device information in the environment collected by the vehicle, matches the target object and the communication device according to the first distance and the second distance, and when it is determined that the match is successful, fuses the first distance and the second distance to obtain the target distance between the successfully matched target object and the vehicle, and performs a collision warning on the vehicle according to the safety warning area, and the target running track and target distance of the successfully matched target object. Since the target distance fuses the ranging of the image information and the ranging of the communication device information, the accuracy of the ranging can be improved, and since the safety warning area of ​​the vehicle, the target running track and target distance of the successfully matched target object are taken into consideration during the collision warning, the accuracy of the warning can be improved.

[0053] In an exemplary embodiment, Figure 3 As shown, the above collision warning method is further described below, which may specifically include:

[0054] Step 1: Collect video images of the vehicle during driving based on the vehicle camera and perform target detection and tracking.

[0055] Specifically, YOLOv8 (the latest iteration of the YOLO series of real-time object detectors) can be used for target detection to output data such as the category, location, and bounding box of the object in front (that is, the target object). It can be expressed as:

[0056]

[0057] Among them, (c x ,c y ) are the horizontal and vertical coordinates of the upper left corner of the network in the feature map, (pw ,p h ) is the length and width of the anchor box. w is the width of the bounding box, t h is the height of the bounding box, t x is the horizontal coordinate of the center point of the bounding box, t y is the vertical coordinate of the center point of the bounding box. (b x , b y ) are the horizontal and vertical coordinates of the center point of the detected target object, (b w , b h ) are the width and height of the minimum bounding box of the detected target object.

[0058] Step 2: Based on the pinhole imaging algorithm, perform monocular distance measurement on the target object to obtain a first distance between the target object and the vehicle.

[0059] Specifically, the effective relevant data output in step 1 can be used to complete the monocular ranging of the target object based on the pinhole imaging algorithm. The specific ranging formula can be:

[0060]

[0061] Wherein, d is the calculated first distance between the target object and the vehicle, d0 is the distance between the camera installation position and the vehicle head (i.e., the error), h is the distance between the camera installation position and the ground, α is the pitch angle of the camera, f is the focal length of the camera, and (y-y0) is the object imaging width.

[0062] Step 3: Use the Kalman filter algorithm to predict and track the trajectory of the center coordinates of the target object output in step 1.

[0063] Specifically, the steps of the Kalman filter algorithm are as follows:

[0064] Kalman filtering can use the system state at the previous time node k-1 and the system state at the current time node k to calculate data, and then estimate the optimal value of the current time node k. The formula is: k =A k x k-1 +B k u k +w k

[0065] Among them, x k is the state at time node k, x k-1 is the state at time node k-1, u k is the impact of external factors at time node k, A is the state transfer matrix, B is the input control matrix, w k is the internal interference generated at time node k.

[0066] Then we can calculate the optimal value of time node k, which is the observed value z k , the formula is: k =H k x k +v k . Among them, H is the observation matrix, v k is the observation noise at time k, x k is the state at time node k calculated above.

[0067] The tracking process includes prediction and update as follows:

[0068] Step 1: The system starts predicting based on x′ at time node k-1 k-1 Predict the system state x′ at time node k k , at this time x′ k is the predicted value, x′ k Can correspond to p k The value of is:

[0069] x′ k =A k x′ k-1 +B k u k

[0070]

[0071] Among them, p k is the covariance matrix, Q is the prediction noise covariance matrix, A is the state transfer matrix, and B is the input control matrix.

[0072] Step 2: The system starts to update, and the system first calculates K; then the system makes corrections, adding a correction data to the system prediction result, thereby outputting the optimal prediction value; finally, the error matrix is ​​updated to provide support for subsequent iterations. The formula is:

[0073]

[0074] x′' k = x′ k +K'(Z K -H k x′ k )

[0075] p′ k =(IK'H k ) k

[0076] Among them, K is the Kalman gain matrix, K' is the corresponding parameter after K is updated, that is, the updated Kalman gain matrix, H is the observation matrix, Z k is the observation value at time node K, I is the unit matrix, R is the observation noise covariance, x′′ k For x′ k The updated corresponding parameter is the updated system state at time k, p′ k is the updated covariance matrix.

[0077] Step 4: Calculate the safety warning area of ​​the vehicle, i.e., the safety area.

[0078] Specifically, the lane detection technology based on Kalman filtering and other technologies is first used to output the width of the vehicle's driving lane, thereby calculating the lateral distance of the safety warning area; then the minimum emergency braking distance of the vehicle is calculated based on parameters such as human reaction time and friction, thereby calculating the safety distance model to determine the longitudinal distance between the first area and the second area (for example, the red and yellow safety warning areas with different warning levels). The longitudinal distance y of the red area min The vertical distance y from the yellow area max The calculation expressions include:

[0079] y min =D+t rmax u

[0080] y max =(t rmax +TTC c )

[0081] Where D is the actual distance the driver needs to travel from the time he realizes the danger to the time he drives the car to a complete stop, t rmax is the maximum dangerous response time required by the driver under normal circumstances, u is the speed of the vehicle, TTC c Time to Collision (Time to Collision Collision is a key indicator used in autonomous driving and intelligent driving assistance systems to measure the time before a vehicle collides with an obstacle ahead) indicates the critical value of TTC.

[0082] Step 5: Collect communication device information in the vehicle's environment based on the communication sensing device in the vehicle.

[0083] The communication sensing device can be a vehicle-mounted Wifi sniffing device. Specifically, the communication device information in the environment of the vehicle is collected based on the communication sensing device in the vehicle, wherein the communication device information includes the MAC (Media Access Control, which is the media access control address, also called the physical address, hardware address) address of the communication device and the corresponding signal strength RSSI value. The signal strength RSSI value of the communication device is filtered using the Dixon test method filtering algorithm, and then the RSSI data after filtering is converted into distance data, namely RSSI ranging, using a logarithmic fitting algorithm, and the distance data is determined as the second distance between the communication device and the vehicle.

[0084] Among them, the Dixon test filtering algorithm can be implemented through the following four steps:

[0085] Step 5.1, arrange the observed samples of RSSI values ​​in ascending order to obtain rssi1 <rssi2<rssi3<...<rssi n , assuming that the test significance level α = 0.05, the critical value is D(α,n).

[0086] Step 5.2, check the high-end and low-end outliers respectively, the formula is:

[0087]

[0088] Among them, rssi e is the sample data with the largest distance, rssi n For the most recent but not equal sample data, rssi f is the sample data with the smallest distance, and rssi1 is the first sample data value in the sequence.

[0089] Step 5.3, compare the high-end outliers and the low-end outliers. When, and γ 11 >D(α,n), rssi n is an abnormal value; when When, and When , rssi1 is an abnormal value, and other cases indicate that no abnormal value occurs.

[0090] Step 5.4: After removing the outliers in the RSSI observation samples, continue to repeat the above steps until the outliers no longer appear.

[0091] Then, the RSSI data collected and filtered out of noise is converted into distance data using a logarithmic fitting algorithm, that is, the second distance between the communication device and the vehicle is determined. First, the actual height of the collection device, the path loss factor n and the constant A are calculated to reduce the impact of the actual environment.

[0092] The RSSI value can be expressed as:

[0093]

[0094] Among them, p r (d) represents the signal strength value between the communication device and the communication sensing device when the distance is d, d0 is the reference distance, p r (d0) is the power loss at the reference distance d0, n is the path loss factor, X dBm is a Gaussian random variable with a mean of 0 and a standard deviation in the range of (4,10).

[0095] In general, X dBm The influence of environmental factors can be ignored. If the d0 value is set to 1m, the conversion formula between RSSI and distance can be calculated as follows:

[0096] [RSSI] dBm =A-10nlgd

[0097] Wherein, A is the signal strength value when the value is 1m, that is, when the distance between the receiving node and the transmitting node is 1m. Based on this, the corresponding RSSI value can be converted into distance data to determine the second distance between the communication device and the vehicle.

[0098] Step 6, matching the target object and the communication device according to the first distance and the second distance, and when it is determined that the match is successful, fusing the first distance and the second distance to obtain the target distance between the successfully matched target object and the vehicle.

[0099] Specifically, the target object in front is captured by using the camera of the device in front of the vehicle, and the communication data such as RSSI and MAC value are captured by using the vehicle-mounted Wifi sniffing device. Then, the visual information and communication information are combined by building a visual-wireless positioning fingerprint library, and a unique MAC value is matched for the detected target object (that is, the target object is bound to the MAC value of the communication device), laying the foundation for the subsequent combination of visual ranging results and communication ranging.

[0100] In this embodiment, the first distance and the second distance obtained above can be used for dynamic weighted distance measurement formula calculation to obtain the target distance value. The dynamic weighted distance measurement formula is expressed as:

[0101] D i =β1A i +β2B i

[0102] Among them, D i is the target distance at time i, A i is the first distance at time i, Bi is the second distance at time i, β1 and β2 are weight coefficients, and their sum is 1.

[0103] If the vehicle is traveling in a scene with a good environment, such as sufficient light and good air (i.e., conducive to the use of visual information), the system determines the time coefficient of the vehicle staying in this warning area and increases the weight coefficient β1 according to the corresponding proportion; similarly, if the vehicle is traveling in a scene with a poor environment but good communication signals, such as the main urban area or near the base station (i.e., conducive to the use of communication information), the system determines the time coefficient and increases the weight coefficient β2 according to the corresponding proportion; the formula is:

[0104] β1=β 10 +0.1tβ2=β 20 +0.1t

[0105] Among them, β 10 and β 20 is the initial default weight value, which can be set to 0.5, and t is the time.

[0106] Step 7: Provide a collision warning to the vehicle based on the safety warning area, the target running trajectory and the target distance of the successfully matched target object.

[0107] Specifically, the safety warning area and the corresponding weight coefficient can be determined according to the target running trajectory of the successfully matched target object; the warning parameters corresponding to the safety warning area and the duration in the safety warning area can be determined; the collision warning value of the vehicle can be determined according to the weight coefficient, warning parameters, duration and target distance; and a warning prompt can be given according to the collision warning value.

[0108] In this embodiment, the safety warning area can be obtained through the above steps, and different warning parameters can be set for different areas (such as the first area and the second area, or the red warning area and the yellow warning area), and the target motion trajectory of the target object obtained by target tracking and the target distance calculated above can be combined to construct a dynamic weight warning calculation formula.

[0109] Specifically, the IMF-FCW algorithm (Forward Collision Warning Algorithm Based on Improved-Multi-Information Fusion) proposed in this embodiment can be used to calculate the collision warning value.

[0110] For example, if the red warning area and the yellow warning area are assigned different warning parameters R1 and R2, respectively, the warning parameter corresponding to the red warning area is greater than the warning parameter corresponding to the yellow warning area. Then, different weight coefficients can be assigned to the warning parameters according to the changes in the target trajectory. Combining the time and target distance, the warning calculation formula F can be obtained as follows:

[0111] F=αR i *t+D i

[0112] Among them, F is the collision warning value, R i is the warning parameter corresponding to the warning area, t is the time value in this area, i.e., the duration, and α is the weight coefficient, which varies according to the predicted target motion trend. Specifically:

[0113]

[0114] Among them, p0 and p1 are the position information of the successfully matched target object at different times in the target running trajectory.

[0115] Then, different levels of warning results can be provided to the driver according to the collision warning value F output by the system, which may specifically include:

[0116]

[0117] Wherein, f1 is greater than f0, f1 is the upper warning threshold, and f0 is the lower warning threshold. For example, when it is determined that the collision warning value is greater than or equal to the preset upper warning threshold, a warning of collision risk may be given; when it is determined that the collision warning value is greater than or equal to the preset lower warning threshold and less than the preset upper warning threshold, a warning of collision risk may be given; when it is determined that the collision warning value is less than the preset lower warning threshold, a warning of safety may be given.

[0118] In the above embodiment, by fusing various types and forms of information, the system-related information parameters are continuously adjusted according to the dynamically changing actual environment, and a dynamic fusion warning strategy is innovatively constructed using a multi-input parameter model. A new IMF-FCW algorithm is proposed, which can select a more appropriate weight coefficient according to the complexity of the scene and the estimated movement direction of the target object, thereby ensuring that the output warning result is more reasonable and reliable, and can minimize the poor robustness of the system caused by the small number of parameters, or the system is too sensitive or too slow to the warning.

[0119] Moreover, compared with relying on only a single piece of information, the ranging method in the proposed IMF-FCW algorithm can effectively integrate multiple information and parameters by utilizing the method of constructing a visual-wireless database, thereby minimizing the problems of slow ranging speed and poor robustness caused by using only a single visual information, and low ranging accuracy and poor stability caused by relying only on communication information, so as to improve accuracy.

[0120] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0121] Based on the same inventive concept, the embodiment of the present application also provides a collision warning device for implementing the above-mentioned collision warning method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more collision warning device embodiments provided below can refer to the limitations on the collision warning method above, and will not be repeated here.

[0122] In an exemplary embodiment, Figure 4 As shown, a collision warning device is provided, including: an image processing module 402, a communication processing module 404, a distance fusion module 406 and a warning module 408, wherein:

[0123] The image processing module 402 is used to perform target detection and tracking based on the image information collected by the vehicle camera, obtain a first distance between the target object and the vehicle, a target running trajectory of the target object, and determine a safety warning area of ​​the vehicle;

[0124] The communication processing module 404 is used to obtain a second distance between the communication device and the vehicle based on the communication device information in the environment collected by the vehicle;

[0125] A distance fusion module 406 is used to match the target object and the communication device according to the first distance and the second distance, and when it is determined that the match is successful, fuse the first distance and the second distance to obtain a target distance between the successfully matched target object and the vehicle;

[0126] The warning module 408 is used to provide a collision warning to the vehicle according to the safety warning area, the target running track of the successfully matched target object, and the target distance.

[0127] In an exemplary embodiment, the safety warning area includes a first area and a second area; the warning module is also used to: determine the safety warning area and the corresponding weight coefficient according to the target running trajectory of the successfully matched target object; determine the warning parameters corresponding to the safety warning area and the duration of being in the safety warning area; determine the collision warning value of the vehicle according to the weight coefficient, the warning parameters, the duration and the target distance; and provide a warning prompt according to the collision warning value.

[0128] In an exemplary embodiment, the warning module is also used to: when it is determined that the collision warning value is greater than or equal to a preset upper warning threshold, prompt the existence of a collision hazard; when it is determined that the collision warning value is greater than or equal to a preset lower warning threshold and less than a preset upper warning threshold, prompt attention to the collision risk; when it is determined that the collision warning value is less than the preset lower warning threshold, prompt safety.

[0129] In an exemplary embodiment, determining the collision warning value of the vehicle according to the weight coefficient, the warning parameter, the duration, and the target distance includes:

[0130] F=αR i *t+D i

[0131] Among them, F is the collision warning value, R i is the warning parameter corresponding to the safety warning area, t is the time in the safety warning area, D i is the target distance at time i, and α is the weight coefficient, which changes according to the motion trend of the target trajectory of the successfully matched target object, specifically including:

[0132]

[0133] Among them, p0 and p1 are the position information of the successfully matched target object at different times in the target running trajectory.

[0134] In an exemplary embodiment, the fusing the first distance and the second distance to obtain the target distance between the successfully matched target object and the vehicle includes:

[0135] D i =β1A i +β2B i

[0136] Among them, D i is the target distance at time i, A i is the first distance at time i, B i is the second distance at time i, β1 and β2 are weight coefficients, and their sum is 1.

[0137] In an exemplary embodiment, the communication processing module is also used to: collect communication device information in the vehicle's environment based on a communication sensing device in the vehicle, the communication device information including the communication device and the corresponding signal strength RSSI value; use a Dixon test filtering algorithm to filter the signal strength RSSI value of the communication device; use a logarithmic fitting algorithm to convert the filtered RSSI data into distance data; and determine the distance data as a second distance between the communication device and the vehicle.

[0138] Each module in the above collision warning device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0139] In an exemplary embodiment, a computer device is provided. The computer device may be a vehicle-mounted device, and its internal structure diagram may be as shown in FIG. Figure 5As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (Near Field Communication, NFC) or other technologies. When the computer program is executed by the processor, a collision warning method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0140] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0141] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0142] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0143] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0144] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0145] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0146] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0147] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A collision warning method, characterized in that: The method comprises: Performing target detection and tracking based on image information collected by a vehicle camera, obtaining a first distance between a target object and the vehicle, a target running trajectory of the target object, and determining a safety warning area of ​​the vehicle; Based on the communication device information in the environment collected by the vehicle, obtaining a second distance between the communication device and the vehicle; Matching the target object and the communication device according to the first distance and the second distance, and when it is determined that the match is successful, fusing the first distance and the second distance to obtain a target distance between the successfully matched target object and the vehicle; A collision warning is performed on the vehicle according to the safety warning area, the target running trajectory of the successfully matched target object, and the target distance.

2. The method according to claim 1, characterized in that: The safety warning area includes a first area and a second area; and performing a collision warning on the vehicle according to the safety warning area, the target running trajectory of the successfully matched target object, and the target distance, includes: Determine the safety warning area and the corresponding weight coefficient according to the target running trajectory of the successfully matched target object; Determine the warning parameters corresponding to the security warning area and the duration of being in the security warning area; Determining a collision warning value of the vehicle according to the weight coefficient, the warning parameter, the duration, and the target distance; A warning prompt is given according to the collision warning value.

3. The method according to claim 2, characterized in that Providing a warning prompt according to the collision warning value includes: When it is determined that the collision warning value is greater than or equal to a preset upper warning threshold, prompting that there is a collision risk; When it is determined that the collision warning value is greater than or equal to a preset lower warning threshold and less than a preset upper warning threshold, a warning is issued to pay attention to the collision risk; When it is determined that the collision warning value is less than a preset lower warning threshold, a safety prompt is given.

4. The method according to claim 2, characterized in that: The determining the collision warning value of the vehicle according to the weight coefficient, the warning parameter, the duration and the target distance includes: F=αR i *t+D i Among them, F is the collision warning value, R i is the warning parameter corresponding to the safety warning area, t is the time in the safety warning area, D i is the target distance at time i, and α is the weight coefficient, which changes according to the motion trend of the target trajectory of the successfully matched target object, specifically including: Among them, p0 and p1 are the position information of the successfully matched target object at different times in the target running trajectory.

5. The method according to any one of claims 1 to 4, characterized in that: The fusing the first distance and the second distance to obtain the target distance between the successfully matched target object and the vehicle includes: D i =β1A i +β2B i Among them, D i is the target distance at time i, A i is the first distance at time i, B i is the second distance at time i, β1 and β2 are weight coefficients, and their sum is 1.

6. The method according to any one of claims 1 to 4, characterized in that: The acquiring a second distance between the communication device and the vehicle based on the communication device information in the environment collected by the vehicle includes: Collecting communication device information in the vehicle's environment based on a communication sensing device in the vehicle, the communication device information including the communication device and the corresponding signal strength RSSI value; Using a Dixon test filtering algorithm to filter the signal strength RSSI value of the communication device; The RSSI data after filtering is converted into distance data using a logarithmic fitting algorithm; The distance data is determined as a second distance between the communication device and the vehicle.

7. A collision warning device, characterized in that: The device comprises: An image processing module, used to perform target detection and tracking based on image information collected by a vehicle camera, obtain a first distance between a target object and the vehicle, a target running trajectory of the target object, and determine a safety warning area of ​​the vehicle; A communication processing module, configured to obtain a second distance between the communication device and the vehicle based on the communication device information in the environment collected by the vehicle; a distance fusion module, configured to match the target object and the communication device according to the first distance and the second distance, and when it is determined that the match is successful, fuse the first distance and the second distance to obtain a target distance between the successfully matched target object and the vehicle; The warning module is used to provide a collision warning to the vehicle according to the safety warning area, the target running track of the successfully matched target object and the target distance.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images