Vehicle early warning method based on animal collision, program product and equipment

Abstract

The invention discloses a vehicle early warning method based on animal collision, a program product and a storage medium. The method comprises the following steps: acquiring image data under a current frame acquired by an image acquisition device of vehicle equipment; performing target detection on the image data under the current frame by using a target detection algorithm to obtain detection data of a current animal target, and obtaining tracking data of the current animal target; calculating collision time corresponding to the current animal target based on the detection data of the current animal target and the tracking data of the current animal target; determining a motion state of the current animal target based on the collision time corresponding to the current animal target; and executing early warning based on the motion state of the current animal target.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a vehicle early warning method based on animal collision, a computer program product and vehicle equipment. Background Art

[0002] Millions of animals are killed in wildlife–vehicle collisions (WVCs) every year, and the resulting traffic accidents pose a serious threat to human and animal lives and road damage. Therefore, in the past few decades, more and more research and prevention measures related to animal-vehicle collisions have emerged. Studies have shown that the risk areas for animal-vehicle collisions are concentrated in places with dim light, poor weather conditions, and where drivers cannot see the road ahead clearly.

[0003] Generally, signs are posted in areas where animals are active, but drivers may not notice these signs and make them ineffective. Setting up fences to force animals to take detours is costly and may interfere with normal animal migration behavior. The regional coverage system can detect large animals within a range of up to 50 meters. The system continuously captures external information (such as heat, sound, and animal movement) and then sends it to a computer for processing and analysis. Compared with traditional fence systems, animals can move freely, but the system has many false alarms under a large amount of vegetation cover. The beam breaking system uses a transmitter and receiver with infrared, laser or micro-radiation. When an animal appears in the beam of the system, a warning will be issued. However, if this method wants to cover a long distance, more sensors must be used.

[0004] Although some current measures can reduce WVCs, there is still a need for a real-time vehicle-mounted animal warning system and display method. In the prior art, in pedestrian and collision schemes, one method is to estimate the depth distance, calculate the relative speed based on the distance of consecutive frames, and thus obtain the collision time. For animals with fast movement speed and a wide variety of species, the scheme based on common height estimation of distance and the scheme based on multi-frame calculation of depth information are invalid. Some animals are in the roadside jungle, and the bottom of the animal detection frame is often not the landing point, so the triangulation based on a single object will also fail. Although the scheme based on binocular vision can estimate the distance, the distance estimation for distant targets is often related to the camera baseline distance, camera focal length, etc., and the binocular vision calibration is complex and the hardware cost is high. Therefore, the collision scheme based on pedestrian detection in the prior art is used for animal target detection, and the warning effect is poor. Summary of the invention

[0005] In order to solve the existing technical problems, the present invention provides a vehicle warning method based on animal collision, a computer program product and a vehicle device, which can improve the accuracy of animal collision warning, thereby improving driving safety.

[0006] In a first aspect, a vehicle warning method based on animal collision is provided, comprising: acquiring image data in a current frame acquired by an image acquisition device of the vehicle equipment; performing target detection on the image data in the current frame using a target detection algorithm to obtain detection data of the current animal target, and acquiring tracking data of the current animal target; calculating a collision time corresponding to the current animal target based on the detection data of the current animal target and the tracking data of the current animal target; determining a motion state of the current animal target based on the collision time corresponding to the current animal target; and executing a warning alert based on the motion state of the current animal target.

[0007] In a second aspect, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the vehicle warning method based on animal collision as described in any embodiment of the present application is implemented.

[0008] In a third aspect, a vehicle device is provided, characterized in that it includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the vehicle warning method based on animal collision as described in any embodiment of the present application.

[0009] The embodiment of the present application collects image data of the vehicle equipment during driving in real time, identifies the current animal target from the image data of the current frame, tracks each animal target during vehicle driving, predicts the collision time of the current animal target relative to the vehicle equipment based on the detection data and tracking data of the current animal target, and specifically analyzes the motion state of the current animal target based on the collision time corresponding to the current animal target, and executes different early warning alerts according to the different motion states of the current animal target, thereby improving the accuracy of animal collision warnings and thus improving driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a diagram of an application environment of a vehicle early warning method based on animal collision in one embodiment;

[0011] Figure 2 is a flow chart of a vehicle early warning method based on animal collision in one embodiment;

[0012] Figure 3 It is a flowchart of calculating the collision time corresponding to the current animal target in the vehicle early warning method based on animal collision in one embodiment;

[0013] Figure 4 It is a flowchart of calculating the collision time corresponding to the current animal target in the vehicle early warning method based on animal collision in another embodiment;

[0014] Figure 5 A flowchart of determining the motion state of a current animal target in a vehicle early warning method based on animal collision in another embodiment;

[0015] Figure 6 A schematic diagram of collision time thresholds at different sensitivity levels in one embodiment;

[0016] Figure 7 A schematic diagram of different alarm warnings in an embodiment;

[0017] Figure 8 is a schematic diagram of a vehicle warning device based on animal collision in one embodiment;

[0018] Fig. 9 Schematic diagram of vehicle equipment in one embodiment. DETAILED DESCRIPTION

[0019] The technical solution of the present invention is further elaborated in detail below in conjunction with the accompanying drawings and specific embodiments of the specification. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the technical field of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the scope of protection of the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0020] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it should be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0021] See also Figure 1 , is an application environment diagram of a vehicle warning method based on animal collision in an embodiment. The vehicle warning method based on animal collision is applied to a vehicle device 10, and the vehicle device 10 includes a display terminal 11, a sensor module 12 and a controller 13. The sensor module 12 is used to collect data during the driving process of the vehicle device 10. The controller 13 predicts the collision time between the vehicle device 10 and the animal target based on the image data collected by the sensor module 12 and the motion data of the vehicle device 10, and displays the warning information on the screen of the vehicle device 10 based on the collision time.

[0022] The sensor module 12 may be a module integrating one or more sensors, and the sensor module 12 may be multiple and may be installed at different positions of the vehicle equipment 10. The sensor module 12 includes but is not limited to a multi-spectral visual sensor, an environmental perception sensor and a motion posture sensor. The multi-spectral sensor includes but is not limited to a combination of one or more of the following sensors: thermal imaging sensor, visible light image sensor, millimeter wave sensor, lidar sensor, infrared thermal imaging sensor and depth sensor. The environmental perception sensor includes but is not limited to a combination of one or more of the following sensors, such as brightness sensor, temperature sensor, haze sensor and other environmental sensors. The motion posture sensor includes but is not limited to a combination of one or more of the following: inertial sensor (Inertial Measurement Unit, IMU), speed sensor, acceleration sensor, gyroscope sensor, geomagnetic sensor, rotation vector sensor, steering wheel angle sensor, level sensor, tilt sensor, vibration sensor, displacement sensor and gravity sensor, etc.

[0023] The screen is the surface on which the display terminal 11 displays images. The screen is arranged in front of the driver so that the driver can view the images. The display terminal 11 includes but is not limited to displays such as liquid crystal displays, projectors with projector screens, etc. The display terminal 11 can be a display located on the central control of the vehicle device 10, or a projector arranged at the rear of the vehicle device 10 to project images onto the projector screen.

[0024] The controller 13 may be one or more. When there are multiple controllers 13, the multiple controllers may be integrated on one chip or independently set on each chip. The vehicle device 10 is a device installed on any type of mobile body, such as a vehicle, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobile device, an airplane, a drone, a ship or a robot, etc. In some embodiments, the controller 13 and the display terminal 11 may be integrated.

[0025] The image acquisition device may be a combination of one or more sensor modules 12. The image acquisition device may be a monocular vision sensor or a multi-vision sensor. For example, it may be a combination of one or more sensors selected from thermal imaging sensors, visible light image sensors, millimeter wave sensors, laser radar sensors, infrared thermal imaging sensors, and depth sensors.

[0026] Common assisted driving sensors include visible light, infrared, millimeter wave radar, lidar, etc. Visible light is easily affected by light, and WVCs often occur in dim light and poor weather conditions. Lidar is not affected by light, but it still misses many small targets and may fail in rainy, snowy and other weather conditions. Millimeter wave radar has low spatial resolution and insufficient target classification capabilities. Infrared thermal imaging detects thermal radiation from objects and is not affected by light. It has a longer detection distance at night, can see through foggy weather, and has higher resolution. Therefore, in an optional implementation, an infrared thermal imaging sensor is used to solve the animal collision warning system in special scenarios such as night, dim light, and fog.

[0027] See also Figure 2 , is a flow chart of a vehicle warning method based on animal collision provided by an embodiment of the present application. The vehicle warning method based on animal collision is applied to vehicle equipment, and the vehicle warning method based on animal collision includes the following steps:

[0028] S11, acquiring image data of the current frame acquired by the image acquisition device of the vehicle equipment.

[0029] In this embodiment, the image acquisition device is arranged in front of the vehicle device 10, and is used to acquire image data in front of the vehicle device 10 during operation. The image acquisition device acquires image data at a preset time interval, and the shorter the time interval, the higher the real-time performance. The acquired real-time video stream data is presented in the form of frame images. The higher the frame rate, the higher the real-time performance.

[0030] S12. Utilize a target detection algorithm to perform target detection on the image data of the current frame, obtain detection data of the current animal target, and obtain tracking data of the current animal target.

[0031] In this embodiment, a target detection algorithm can be used to identify the target from the image data and obtain the detection data of the target. The identified current animal target can be one or more, wherein the detection data includes but is not limited to: the center point coordinates of the detection frame, the size of the detection frame, the length, width and height of the target, etc. The target detection model algorithm includes but is not limited to the target detection model algorithm based on MonoCon, the monocular target detection model algorithm based on SMOKE, the target detection algorithm based on neural network, etc. For the subsequent calculation of the collision time, the origin of the world coordinate system is established with the front of the vehicle, and the three-dimensional center point coordinates of the target are converted to the world coordinate system.

[0032] In an optional implementation, the target detection algorithm is described by taking a target detection algorithm based on a neural network as an example. Detecting a target in the image data of the current frame using the target detection algorithm includes:

[0033] An animal sample data set is collected in advance, wherein the animal sample data set includes animal samples of various categories, samples of non-animal categories, and category labels corresponding to the samples. The animal samples of various categories include animal image samples of the same category in various postures. For example, animal samples of various categories include, but are not limited to, image data of various animal samples in various postures such as wild boars, elks, coyotes, foxes, sheep, bisons, horses, etc. Samples of non-animal categories include, but are not limited to, image data of various forms such as cars, buses, other vehicles, pedestrians, two-wheelers, three-wheelers, etc. The category label corresponding to the sample is the label corresponding to each sample, for example, the label corresponding to the fox is animal, and the label corresponding to the pedestrian is non-animal.

[0034] The animal sample data set is used as the input data of the target detection model, and the target detection model is trained until the target detection model reaches the training termination condition, thereby obtaining a trained target detection model. During the training process, the target detection model uses the labels in training as training targets, learns the characteristics of animal samples and non-animal samples in the animal sample data set, and can identify animal targets and non-animal targets after the training is completed.

[0035] In this embodiment, the target tracking algorithm is used to track the animal target and record the tracking data of the animal target. The tracking data includes the detection data of the animal target in each frame of the historical frame. The target tracking algorithm includes but is not limited to target tracking based on mean shift algorithm, target tracking based on Kalman filter, target tracking based on particle filter, etc.

[0036] S13. Calculate the collision time corresponding to the current animal target based on the detection data of the current animal target and the tracking data of the current animal target.

[0037] In this embodiment, by combining the tracking data of the current animal target in the historical frame and the detection data of the current animal target, the scale change of the current animal target can be determined, and the collision time of the current animal target in the current frame can be predicted according to the scale change.

[0038] S14. Determine the motion state of the current animal target based on the collision time corresponding to the current animal target.

[0039] In this embodiment, the motion state of the current animal target is determined by combining the collision time corresponding to the current animal target and the tracking data of the current animal target, wherein the motion state includes but is not limited to animal collision, animal crossing, animal moving away, etc.

[0040] S15. Execute an early warning based on the current movement status of the animal target.

[0041] In this embodiment, the number of current animal targets in the current frame is one or more. For any animal target, when tracking the animal target, the collision time between the current animal target and the vehicle equipment in the current frame can be predicted by using steps S11-S15. Therefore, when tracking each animal target, the collision time between the vehicle and each target can be predicted in real time, and the motion state of the current animal target can be analyzed according to the collision time, and different early warnings can be executed according to the different motion states of the current animal target.

[0042] In the above embodiment, image data of the vehicle equipment during driving is collected in real time, and the current animal target is identified from the image data of the current frame. Each animal target is tracked during the vehicle driving process. The collision time of the current animal target relative to the vehicle equipment is predicted based on the detection data and tracking data of the current animal target. Based on the collision time corresponding to the current animal target, the motion state of the current animal target is specifically analyzed. Different early warning alerts are executed according to the different motion states of the current animal target, thereby improving the accuracy of the animal collision warning and thus improving driving safety.

[0043] In some embodiments, step S13 specifically includes: Figure 3 The steps shown, Figure 3 This is a flow chart of calculating the collision time corresponding to the current animal target in a vehicle warning method based on animal collision in one embodiment, and the specific steps are as follows:

[0044] S131. Based on the detection data of the current animal target, divide the current animal target into a plurality of local blocks.

[0045] In this embodiment, since the animal movement is non-rigid movement and the posture will be deformed during the movement, a detection frame of a current animal target can be divided into multiple local blocks, each local block represents a feature point, and the corner point detection algorithm can be used to determine the corner point in the detection frame, and then the detection frame can be divided into multiple local blocks based on the detected corner points, for example, into 3*3 local blocks.

[0046] S132: Based on the tracking data of the current animal target, multiple local blocks are tracked, and the collision time corresponding to each local block is calculated.

[0047] In this embodiment, when tracking an animal target, each local block represents a feature point. For example, tracking is performed with the center point of the local block as the tracking point. During the tracking process, when the feature of the local block is detected, the neighborhood of the local block is calculated based on the tracking data of the local block, and the scale change value of the local block is calculated based on the neighborhood of the local block. Then, based on the scale change value of the local block, the collision time corresponding to the local block is calculated.

[0048] S133. Calculate the collision time corresponding to the current animal target based on the collision time corresponding to each local block.

[0049] In the above embodiment, the current animal target is divided into multiple local blocks for tracking, and the collision time of each local block is calculated respectively. Based on the collision time of each local block, the collision time of the current animal target as a whole is calculated, thereby reducing the impact of posture changes during animal movement on the collision time calculation, improving the accuracy of animal collision warning, and thus improving driving safety.

[0050] Optionally, before tracking the plurality of local blocks based on the tracking data of the current animal target, the method further includes:

[0051] Acquire the tracking data of the historical target in the preset frame before the current frame from the tracking data of the current animal target;

[0052] Based on the tracking data of the historical targets and the detection data of the current animal target, the historical targets associated with the current animal target are determined.

[0053] In this embodiment, in the process of target tracking, a multi-target association tracking method is adopted for tracking, and based on the detection frame of the current animal target in the current frame and the tracking data in the historical frame of the current frame, the similarity between the tracking frame of the historical frame and the detection frame in the current frame is calculated, and based on the calculated similarity, the tracking frame of the historical frame is associated with the detection frame in the current frame.

[0054] Optionally, the determining the historical target associated with the current animal target based on the tracking data of the historical target and the detection data of the current animal target includes:

[0055] The tracking frames in the previous preset frame are taken as a set X and the detection frames in the current frame are taken as a set Y, and a weight matrix is ​​obtained, wherein each column in the weight matrix represents a weight value corresponding to each tracking frame and each detection frame, and each row represents a weight value corresponding to each detection frame and each tracking frame;

[0056] Calculate the maximum value in each column of the weight matrix and store it in the tracking frame top mark set WX, and initialize the detection frame top mark set WY, where each tracking frame in WX is regarded as a vertex, the maximum value corresponding to each tracking frame is the top mark corresponding to each tracking frame, and each detection frame in WY is regarded as a vertex;

[0057] Traverse the tracking frames in set X in sequence. The currently traversed tracking frame is represented by u. For the currently traversed tracking frame, traverse the set Y to find whether there is a detection frame v that meets the preset condition, where the preset condition is WX[u]+WY[v]―MAP[u][v]=0;

[0058] If there is a detection box v that meets the preset conditions, return and continue to traverse the tracking boxes in the set X until all the tracking boxes in the set X are traversed;

[0059] If there is no detection box that meets the preset conditions, traverse the set Y to find the detection box v that satisfies minz = min{WX[u] + WY[v] - MAP[u][v]}, subtract minz from the top mark corresponding to the currently traversed tracking box, and add minz to the top mark corresponding to the detection box v, and return to continue traversing the set Y.

[0060] In this embodiment, for a currently traversed tracking frame u, if there is a detection frame v that meets the preset conditions, the currently traversed tracking frame u is associated with the detection frame v. Since the tracking frame is the tracking frame of the animal target in the historical frame, the historical target associated with the current animal target is found, where the current animal target and the historical target may or may not be the same target. The larger the corresponding weight value between the current animal target and the historical target, the greater the probability that the current animal target and the historical target are the same target.

[0061] Optionally, the tracking of the plurality of local blocks based on the tracking data of the current animal target includes:

[0062] Based on the historical targets associated with the current animal target, multi-target association tracking is performed on the plurality of local blocks.

[0063] In this embodiment, when the multi-target association tracking method is used to track the target, each local block is also tracked using the multi-target association tracking method, so that a more accurate local scale change value can be obtained later.

[0064] In the above embodiment, the current animal target is divided into multiple local blocks for tracking, and each local block is tracked using a multi-target association tracking method, and then the collision time of each local block is calculated separately. Based on the collision time of each local block, the collision time of the current animal target as a whole is calculated, thereby reducing the impact of posture changes during animal movement on the collision time calculation, improving the accuracy of animal collision warning, and thus improving driving safety.

[0065] In some embodiments, step S132 further includes: Figure 4 The flowchart steps shown, Figure 4This is a flow chart of calculating the collision time corresponding to the current animal target in the vehicle warning method based on animal collision in another embodiment, which is specifically as follows:

[0066] S1321. Track the local block and determine the tracking data of the local block.

[0067] In this embodiment, the current detection frame of a current animal target is divided into multiple local blocks, and any local block is tracked. The tracking data includes the detection data of the local block in each frame of the historical frame.

[0068] S1322: Calculate a neighborhood of the local block based on the tracking data of the local block, and calculate a local scale change value corresponding to the local block based on the neighborhood of the local block.

[0069] In this embodiment, during the tracking process, when the features of a local block are detected, the neighborhood of the local block is calculated based on the tracking data of the local block, the scale change value of the local block is calculated based on the neighborhood of the local block, and then the collision time corresponding to the local block is calculated based on the scale change value of the local block.

[0070] Optionally, a local scale change formula is used to calculate a local scale change value corresponding to the local block, wherein the local scale change formula is as follows:

[0071]

[0072] Wherein, one of the local blocks is used as a feature point. is the normalized local scale change of the ith feature point in the jth frame, where b is the number of frames for tracking the feature point. For point Neighborhood of is the coordinate corresponding to the i-th feature point in the neighborhood of the j-th frame, is the coordinate corresponding to the i-th feature point in the j-b-th frame, is the coordinate corresponding to the kth feature point in the neighborhood of the j-bth frame, is the coordinate corresponding to the i-th feature point in the j-th frame.

[0073] From the above formula, we can see that the jth frame is equivalent to the current frame, and the j-bth frame is equivalent to the historical frame before the current frame. For a local block of a current animal target, the b frame is tracked, and the local scale change value of a local block is calculated using the tracking data of the b frame.

[0074] S1323: Calculate the collision time corresponding to the local block based on the local scale change value corresponding to the local block.

[0075] The collision time calculation formula is used to calculate the collision time corresponding to the local block. The collision time calculation formula is:

[0076]

[0077] in, Indicates the collision time corresponding to the i-th feature point in the j-frame.

[0078] In the above embodiment, the current animal target is divided into multiple local blocks for tracking, and the local scale change value of each local block is calculated respectively, and the collision time corresponding to each local block is calculated based on the local scale change value. Based on the collision time of each local block, the collision time of the current animal target as a whole is calculated, thereby reducing the impact of posture changes during animal movement on the collision time calculation, improving the accuracy of animal collision warning, and thus improving driving safety.

[0079] In some embodiments, S133, based on the collision times corresponding to the local blocks, calculating the collision time corresponding to the current animal target includes:

[0080] Calculate the distance between each local block and the detection frame of the current animal target, take one local block as a sampling point, take the distance as the independent variable, and the local scale change value as the dependent variable, and fit the sampling points in the detection frame of the current animal target to obtain a fitting model;

[0081] Deleting sampling points whose distance from the fitting model is greater than a preset distance value to obtain a target local block;

[0082] The collision times corresponding to the local target blocks are averaged to obtain the collision time corresponding to the current animal target.

[0083] In this embodiment, since the posture of the animal will change during the movement, the animal target is divided into multiple local blocks for tracking during the tracking process. Due to the change in posture, some local blocks may have inaccurate calculated local scale change values ​​due to the influence of posture, so some noise values ​​need to be deleted. The distance between the center point of the local block and the center point of the detection frame of the current animal target can be calculated, and the calculated distance is used as the distance from the local block to the detection frame of the current animal target. For a local block, there is a distance and a local scale change value. A local block is used as a sampling point, the distance is used as the independent variable, and the local scale change value is used as the dependent variable. A plurality of sampling points formed by a plurality of local blocks are fitted to obtain a fitting model. The fitting model can be a fitting straight line. Then, the distance value from each sampling point to the fitting model is calculated, and the sampling points whose distance from the fitting model is greater than the preset distance value are deleted, that is, the local scale change values ​​corresponding to the local blocks that do not meet the requirements are deleted to obtain the target local block, and the collision time corresponding to the target local block is averaged to obtain the collision time corresponding to the current animal target, that is, t j Indicates the collision time corresponding to the current animal target in the jth frame.

[0084] In some embodiments, determining the motion state of the current animal target based on the collision time corresponding to the current animal target includes:

[0085] When the collision time corresponding to the current animal target is less than or equal to 0, determining that the current animal target is gradually moving away from the vehicle equipment;

[0086] When the collision time corresponding to the current animal target is greater than 0 and less than a preset collision time threshold, determining that the current animal target will collide with the vehicle equipment;

[0087] When the collision time corresponding to the current animal target is greater than a preset collision time threshold and the absolute value of the first horizontal coordinate is smaller than the absolute value of the second horizontal coordinate, it is determined that the current animal target has crossed the front area of ​​the vehicle equipment, wherein the first horizontal coordinate indicates the horizontal coordinate of the center point of the detection box of the current animal target in the current frame, and the second horizontal coordinate indicates the horizontal coordinate of the center point of the detection box of the current animal target in a preset historical frame before the current frame.

[0088] Specifically, Figure 5 As shown, Figure 5 The flowchart of determining the motion state of the current animal target in the vehicle early warning method based on animal collision in another embodiment includes the following steps:

[0089] S141, determining whether there is a current animal target in the current frame image.

[0090] If there is an animal target in the current frame image, that is, the current animal target, execute step S142; if there is no animal target in the current frame image, return to continue executing S141.

[0091] S142, determining whether the collision time corresponding to the current animal target is greater than 0.

[0092] If the collision time corresponding to the current animal target is greater than 0, execute S143.

[0093] If the collision time corresponding to the current animal target is less than or equal to 0, S144 is executed to determine that the current animal target is gradually moving away from the vehicle equipment.

[0094] S143, determining whether the collision time corresponding to the current animal target is less than a preset collision time threshold.

[0095] If the collision time corresponding to the current animal target is less than the preset collision time threshold, execute S145. If the collision time corresponding to the current animal target is greater than or equal to the preset collision time threshold, execute S146.

[0096] S145, determining that the current animal target will collide with vehicle equipment.

[0097] S146, determining whether the absolute value of the first horizontal coordinate is smaller than the absolute value of the second horizontal coordinate.

[0098] If the absolute value of the first horizontal coordinate is smaller than the absolute value of the second horizontal coordinate, S147 is executed. If the absolute value of the first horizontal coordinate is larger than the absolute value of the second horizontal coordinate, the process ends.

[0099] S147, determining that the current animal target crosses the front area of ​​the vehicle equipment.

[0100] In this embodiment, as the target moves away, the scale of the target in the current frame becomes smaller and smaller. Relative to the historical frame of the current frame, the scale change value is a negative value, and the calculated collision time is also a negative value. Therefore, when the collision time is less than 0, the target is moving away from the vehicle equipment.

[0101] In the above embodiment, the different motion states of the current animal target are analyzed through the collision time of the current animal target, so that different early warning alerts can be given according to different motion states, thereby improving the accuracy of animal collision warnings and thus improving driving safety.

[0102] In some embodiments, the preset collision time threshold includes collision time thresholds of multiple different sensitivity levels, wherein the higher the sensitivity, the larger the collision time threshold.

[0103] In this embodiment, a plurality of collision time thresholds of different sensitivity levels can be displayed on the display terminal 11, and the user can configure the preset collision time threshold on the interface, such as Figure 6 As shown, it is a schematic diagram of the collision time thresholds of different sensitivity levels in one embodiment. The sensitivity levels include high sensitivity, medium sensitivity and low sensitivity. The collision time threshold corresponding to high sensitivity is the largest, and the collision time threshold corresponding to low sensitivity is the smallest. The larger the collision time threshold, the earlier the alarm, and the smaller the collision time threshold, the later the alarm. It can be understood that the sensitivity level is not limited to Figure 6 The three gears shown in the figure can set three or more sensitivity gears according to needs. The higher the sensitivity, the larger the collision time threshold. When calculating the collision time based on the scale change value, if the collision time is greater than 0, the scale in the current frame changes greatly relative to the scale in the historical frame, indicating that the target is about to collide with the vehicle equipment. When the collision time is greater than 0, the smaller the calculated collision time, the closer the collision distance with the vehicle equipment. Therefore, users can set different sensitivities according to their own driving proficiency. When the driving proficiency is not high, you can set the collision time threshold corresponding to high sensitivity to make the alarm earlier. When the driving proficiency is very high, you can lower the sensitivity and correspondingly reduce the collision time threshold to make the alarm later.

[0104] In the above embodiment, by setting the collision time thresholds of different sensitivity levels, the needs of different users and different alarm requirements can be met, thereby improving driving safety.

[0105] In some embodiments, the performing of the early warning alert based on the current motion state of the animal target includes:

[0106] Determining an alarm type indicating a danger level in the area ahead of the vehicle device based on at least one of the following data, wherein the at least one data includes detection data of a current animal target, a motion state of the current animal target, and activity data of the current animal target at the location of the vehicle device;

[0107] According to the alarm type, different levels of early warning alerts are executed.

[0108] Optionally, the alarm type includes at least one of the following: animal collision, animal crossing, animal group, animal detection, and animal activity area, wherein the animal collision indicates that the current animal target will collide with the vehicle equipment, the animal crossing indicates that the current animal target has crossed the front area of ​​the vehicle equipment, the animal group indicates that the current animal targets are greater than a preset number, and the animal activity area indicates that the vehicle equipment has entered an animal infestation area.

[0109] In this embodiment, animal collision, animal crossing and animal detection have been described in one or more of the above embodiments, and will not be repeated here. For animal groups, if the detected animal targets are greater than the preset number, it is determined that there is an animal group in front of the vehicle equipment. For animal activity areas, the current location area of ​​the vehicle equipment can be obtained, and the historical data of the current location area can be obtained. If the historical data of the current location area indicates that the location is an animal infestation area, it means that the vehicle equipment has entered the animal activity area. For example, the vehicle equipment can obtain the historical data of the current location area from the connected digital map. If the historical data of the current location area obtained indicates that it is not an animal infestation area, but the vehicle equipment recognizes an animal group, the vehicle equipment can report the information of the animal group recognized in the current location area to update the historical data of the current location area, so that the vehicle can refer to the updated data after driving to the location area in the future.

[0110] like Figure 7 As shown, Figure 7 This is a schematic diagram of different alarm warnings in an embodiment. For different alarm types, different levels of alarm warning icons and sound prompts are given to the driver to reduce the risk of animal collision. Early warning warnings include but are not limited to one or more of the following combinations: different alarm types provide different alarm box styles, and different alarm types provide different voice content.

[0111] For example, the alarm alert method can be as follows:

[0112] When judging the animal collision warning, the priority is the highest. A red detection frame and a warning icon of the collision animal are displayed on the display terminal 11, and the alarm sound is a rapid beeping sound. The driver needs to brake as soon as possible.

[0113] If it is determined that an animal is crossing, a warning icon needs to be drawn on the display terminal 11, and a voice prompt is given to the driver, indicating that an animal is crossing and the current speed of the vehicle is what, and the driver should reduce the speed.

[0114] When a herd of animals is detected, a warning icon needs to be drawn on the display terminal 11, and a voice prompt is given to the driver, saying that there is a herd of animals and the current speed of the vehicle is so low. Please reduce the speed.

[0115] When an animal is detected, a warning icon needs to be drawn on the display terminal 11, and a voice prompt is given to the driver, "Animal detected, drive carefully."

[0116] If the GPS positioning information is in an animal activity area, a warning icon needs to be drawn on the display terminal 11, and a voice prompt is given to the driver to drive carefully when entering an animal activity area.

[0117] Optionally, executing different levels of early warning alerts according to the alarm type further includes:

[0118] When there are multiple types of alarms, early warning alerts are performed based on the alarm priorities.

[0119] In this embodiment, the priorities of various alarm types are set. Multiple alarm types may appear in front of the vehicle equipment. Here, alarms are issued based on the alarm priorities. In an optional implementation, the order of alarm priorities is: animal collision > animal crossing > animal group > animal detection > entering animal activity area. The higher the alarm priority, the higher the danger level of the alarm type. When multiple alarm types exist at the same time, the alarm type with a high alarm priority level is given priority.

[0120] In the above embodiment, based on the detection data, the movement state of the current animal target, and the activity data of the current animal target at the location of the vehicle equipment, multiple alarm types are divided so that different early warning alerts can be executed according to different levels of alarm types, and alarm types with higher levels of danger can be given priority according to the alarm priority, which can improve the accuracy of animal collision warnings and thus improve driving safety.

[0121] On the other hand, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the vehicle warning method based on animal collision described in any embodiment of the present application.

[0122] Among them, in the computer program product, an optional implementation form of the program module architecture of the computer program that implements each step of the target recognition method can be a vehicle warning device based on animal collision.

[0123] See also Figure 8 An embodiment of the present application provides a vehicle warning device based on animal collision, including: an acquisition module 81, used to acquire image data under the current frame acquired by the image acquisition device of the vehicle equipment; a detection module 82, used to perform target detection on the image data under the current frame using a target detection algorithm, obtain the detection data of the current animal target, and obtain the tracking data of the current animal target; a calculation module 83, used to calculate the collision time corresponding to the current animal target based on the detection data of the current animal target and the tracking data of the current animal target; the calculation module 83 is also used to determine the motion state of the current animal target based on the collision time corresponding to the current animal target; an early warning module 84, used to execute an early warning alert based on the motion state of the current animal target.

[0124] Optionally, the calculation module 83 is further used for:

[0125] Based on the detection data of the current animal target, dividing the current animal target into a plurality of local blocks;

[0126] Based on the tracking data of the current animal target, the plurality of local blocks are tracked, and the collision time corresponding to each of the local blocks is calculated;

[0127] Based on the collision times corresponding to the local blocks, the collision time corresponding to the current animal target is calculated.

[0128] Optionally, the calculation module 83 is further used for:

[0129] Acquire the tracking data of the historical target in the preset frame before the current frame from the tracking data of the current animal target;

[0130] Determining a historical target associated with the current animal target based on the tracking data of the historical target and the detection data of the current animal target;

[0131] Optionally, the calculation module 83 is also used for

[0132] Based on the historical targets associated with the current animal target, multi-target association tracking is performed on the plurality of local blocks.

[0133] Optionally, the calculation module 83 is further used for:

[0134] The tracking frames in the previous preset frame are taken as a set X and the detection frames in the current frame are taken as a set Y, and a weight matrix is ​​obtained, wherein each column in the weight matrix represents a weight value corresponding to each tracking frame and each detection frame, and each row represents a weight value corresponding to each detection frame and each tracking frame;

[0135] Calculate the maximum value in each column of the weight matrix and store it in the tracking frame top mark set WX, and initialize the detection frame top mark set WY, where each tracking frame in WX is regarded as a vertex, the maximum value corresponding to each tracking frame is the top mark corresponding to each tracking frame, and each detection frame in WY is regarded as a vertex;

[0136] Traverse the tracking frames in set X in sequence. The currently traversed tracking frame is represented by u. For the currently traversed tracking frame, traverse the set Y to find whether there is a detection frame v that meets the preset condition, where the preset condition is WX[u]+WY[v]―MAP[u][v]=0;

[0137] If there is a detection box v that meets the preset conditions, return and continue to traverse the tracking boxes in the set X until all the tracking boxes in the set X are traversed;

[0138] If there is no detection box that meets the preset conditions, traverse the set Y to find the detection box v that satisfies minz = min{WX[u] + WY[v] - MAP[u][v]}, subtract minz from the top mark corresponding to the currently traversed tracking box, and add minz to the top mark corresponding to the detection box v, and return to continue traversing the set Y.

[0139] Optionally, the calculation module 83 is further used for:

[0140] Tracking the local block to determine tracking data of the local block;

[0141] Based on the tracking data of the local block, a neighborhood of the local block is calculated, and based on the neighborhood of the local block, a local scale change value corresponding to the local block is calculated;

[0142] Based on the local scale change value corresponding to the local block, the collision time corresponding to the local block is calculated.

[0143] Optionally, the calculation module 83 is further used for:

[0144] The local scale change formula is used to calculate the local scale change value corresponding to the local block, where the local scale change formula is as follows:

[0145]

[0146] Wherein, one of the local blocks is used as a feature point. is the normalized local scale change of the ith feature point in the jth frame, where b is the number of frames for which the feature point has been tracked. For point Neighborhood of is the coordinate corresponding to the i-th feature point in the neighborhood of the j-th frame, is the coordinate corresponding to the i-th feature point in the j-b-th frame, is the coordinate corresponding to the kth feature point in the neighborhood of the j-bth frame, is the coordinate corresponding to the i-th feature point in the j-th frame;

[0147] Optionally, the calculation module 83 is further used for:

[0148] The collision time calculation formula is used to calculate the collision time corresponding to the local block. The collision time calculation formula is:

[0149]

[0150] in, Indicates the collision time corresponding to the i-th feature point in the j-frame.

[0151] Optionally, the calculation module 83 is further used for:

[0152] Calculate the distance between each local block and the detection frame of the current animal target, take one local block as a sampling point, take the distance as the independent variable, and the local scale change value as the dependent variable, and fit the sampling points in the detection frame of the current animal target to obtain a fitting model;

[0153] Deleting sampling points whose distance from the fitting model is greater than a preset distance value to obtain a target local block;

[0154] The collision times corresponding to the local target blocks are averaged to obtain the collision time corresponding to the current animal target.

[0155] Optionally, the calculation module 83 is further used for:

[0156] When the collision time corresponding to the current animal target is less than or equal to 0, determining that the current animal target is gradually moving away from the vehicle equipment;

[0157] When the collision time corresponding to the current animal target is greater than 0 and less than a preset collision time threshold, determining that the current animal target will collide with the vehicle equipment;

[0158] When the collision time corresponding to the current animal target is greater than a preset collision time threshold and the absolute value of the first horizontal coordinate is smaller than the absolute value of the second horizontal coordinate, it is determined that the current animal target has crossed the front area of ​​the vehicle equipment, wherein the first horizontal coordinate indicates the horizontal coordinate of the center point of the detection box of the current animal target in the current frame, and the second horizontal coordinate indicates the horizontal coordinate of the center point of the detection box of the current animal target in a preset historical frame before the current frame.

[0159] Optionally, the preset collision time threshold includes collision time thresholds of multiple different sensitivity levels, wherein the higher the sensitivity, the larger the collision time threshold.

[0160] Optionally, the early warning module 84 is further used for:

[0161] Determining an alarm type indicating a danger level in the area ahead of the vehicle device based on at least one of the following data, wherein the at least one data includes detection data of a current animal target, a motion state of the current animal target, and activity data of the current animal target at the location of the vehicle device;

[0162] According to the alarm type, different levels of early warning alerts are executed.

[0163] Optionally, the alarm type includes at least one of the following: animal collision, animal crossing, animal group, animal detection, and animal activity area, wherein the animal collision indicates that the current animal target will collide with the vehicle equipment, the animal crossing indicates that the current animal target has crossed the front area of ​​the vehicle equipment, the animal group indicates that the current animal targets are greater than a preset number, and the animal activity area indicates that the vehicle equipment has entered an animal infestation area.

[0164] Optionally, the early warning module 84 is further used for:

[0165] When there are multiple types of alarms, early warning alerts are performed based on the alarm priorities.

[0166] It can be understood by those skilled in the art that Figure 8 The structure of the vehicle warning device based on animal collision does not constitute a limitation on the vehicle warning device based on animal collision, and each module can be implemented in whole or in part by software, hardware, and a combination thereof. The above modules can be embedded in or independent of the controller in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the controller can call and execute the operations corresponding to the above modules. In other embodiments, the vehicle warning device based on animal collision may include more or fewer modules than those shown in the figure.

[0167] See also Fig. 9 In another aspect of the embodiment of the present application, a vehicle device 10 is provided, including a memory 3011 and a processor 3012. The memory 3011 stores a computer program. When the computer program is executed by the processor, the processor 3012 executes the steps of the vehicle warning method based on animal collision provided in any of the above embodiments of the present application. The vehicle device 10 is a device installed on any type of mobile body, such as a vehicle, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobile device, an airplane, a drone, a ship, or a robot, etc.

[0168] The processor 3012 is the control center, which uses various interfaces and lines to connect various parts of the entire computer device, and executes various functions of the computer device and processes data by running or executing software programs and / or modules stored in the memory 3011, and calling data stored in the memory 3011. Optionally, the processor 3012 may include one or more processing cores; preferably, the processor 3012 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user pages and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 3012.

[0169] The memory 3011 can be used to store software programs and modules. The processor 3012 executes various functional applications and data processing by running the software programs and modules stored in the memory 3011. The memory 3011 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 3011 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 3011 may also include a memory controller to provide the processor 3012 with access to the memory 3011.

[0170] On the other hand, an embodiment of the present application further provides a storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the vehicle warning method based on animal collision provided in any of the above embodiments of the present application.

[0171] Those skilled in the art can understand that all or part of the processes in the methods provided in the above embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0172] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A vehicle early warning method based on animal collision, applied to vehicle equipment, characterized in that: include: Acquire image data of a current frame acquired by an image acquisition device of the vehicle equipment; Using a target detection algorithm to perform target detection on the image data in the current frame, obtain detection data of the current animal target, and obtain tracking data of the current animal target; Calculating the collision time corresponding to the current animal target based on the detection data of the current animal target and the tracking data of the current animal target; Determining a motion state of the current animal target based on a collision time corresponding to the current animal target; Based on the current movement state of the animal target, a warning alert is executed.

2. The vehicle early warning method based on animal collision as claimed in claim 1, characterized in that: The calculating of the collision time corresponding to the current animal target based on the detection data of the current animal target and the tracking data of the current animal target comprises: Based on the detection data of the current animal target, dividing the current animal target into a plurality of local blocks; Based on the tracking data of the current animal target, the plurality of local blocks are tracked, and the collision time corresponding to each of the local blocks is calculated; Based on the collision times corresponding to the local blocks, the collision time corresponding to the current animal target is calculated.

3. The vehicle warning method based on animal collision as claimed in claim 2, characterized in that: Before tracking the plurality of local blocks based on the tracking data of the current animal target, the method further includes: Acquire the tracking data of the historical target in the preset frame before the current frame from the tracking data of the current animal target; Determining a historical target associated with the current animal target based on the tracking data of the historical target and the detection data of the current animal target; The tracking of the plurality of local blocks based on the tracking data of the current animal target includes: Based on the historical targets associated with the current animal target, multi-target association tracking is performed on the plurality of local blocks.

4. The vehicle warning method based on animal collision as claimed in claim 3, characterized in that: The step of determining the historical target associated with the current animal target based on the tracking data of the historical target and the detection data of the current animal target includes: The tracking frames in the previous preset frame are taken as a set X and the detection frames in the current frame are taken as a set Y, and a weight matrix is ​​obtained, wherein each column in the weight matrix represents a weight value corresponding to each tracking frame and each detection frame, and each row represents a weight value corresponding to each detection frame and each tracking frame; Calculate the maximum value in each column of the weight matrix and store it in the tracking frame top mark set WX, and initialize the detection frame top mark set WY, where each tracking frame in WX is regarded as a vertex, the maximum value corresponding to each tracking frame is the top mark corresponding to each tracking frame, and each detection frame in WY is regarded as a vertex; Traverse the tracking frames in set X in sequence. The currently traversed tracking frame is represented by u. For the currently traversed tracking frame, traverse the set Y to find whether there is a detection frame v that meets the preset condition, where the preset condition is WX[u]+WY[v]―MAP[u][v]=0; If there is a detection box v that meets the preset conditions, return and continue to traverse the tracking boxes in the set X until all the tracking boxes in the set X are traversed; If there is no detection box that meets the preset conditions, traverse the set Y to find the detection box v that satisfies minz = min{WX[u] + WY[v] - MAP[u][v]}, subtract minz from the top mark corresponding to the currently traversed tracking box, and add minz to the top mark corresponding to the detection box v, and return to continue traversing the set Y.

5. The vehicle warning method based on animal collision as claimed in claim 2, characterized in that: The step of tracking the plurality of local blocks based on the tracking data of the current animal target and calculating the collision time corresponding to each local block includes: Tracking the local block to determine tracking data of the local block; Based on the tracking data of the local block, a neighborhood of the local block is calculated, and based on the neighborhood of the local block, a local scale change value corresponding to the local block is calculated; Based on the local scale change value corresponding to the local block, the collision time corresponding to the local block is calculated.

6. The vehicle warning method based on animal collision as claimed in claim 5, characterized in that: The calculating the neighborhood of the local block based on the tracking data of the local block, and calculating the local scale change value corresponding to the local block based on the neighborhood of the local block, includes: The local scale change formula is used to calculate the local scale change value corresponding to the local block, where the local scale change formula is as follows: Wherein, one of the local blocks is used as a feature point. is the normalized local scale change of the ith feature point in the jth frame, where b is the number of frames for tracking the feature point. For point Neighborhood of is the coordinate corresponding to the i-th feature point in the neighborhood of the j-th frame, is the coordinate corresponding to the i-th feature point in the j-b-th frame, is the coordinate corresponding to the kth feature point in the neighborhood of the j-bth frame, is the coordinate corresponding to the i-th feature point in the j-th frame; The calculating the collision time corresponding to the local block based on the local scale change value corresponding to the local block includes: The collision time calculation formula is used to calculate the collision time corresponding to the local block. The collision time calculation formula is: in, Indicates the collision time corresponding to the i-th feature point in the j-frame.

7. The vehicle warning method based on animal collision as claimed in claim 5, characterized in that: The calculating the collision time corresponding to the current animal target based on the collision time corresponding to each of the local blocks comprises: Calculate the distance between each local block and the detection frame of the current animal target, take one local block as a sampling point, take the distance as the independent variable, and the local scale change value as the dependent variable, and fit the sampling points in the detection frame of the current animal target to obtain a fitting model; Deleting sampling points whose distance from the fitting model is greater than a preset distance value to obtain a target local block; The collision times corresponding to the local target blocks are averaged to obtain the collision time corresponding to the current animal target.

8. The vehicle warning method based on animal collision as claimed in claim 1, characterized in that: The determining the motion state of the current animal target based on the collision time corresponding to the current animal target includes: When the collision time corresponding to the current animal target is less than or equal to 0, determining that the current animal target is gradually moving away from the vehicle equipment; When the collision time corresponding to the current animal target is greater than 0 and less than a preset collision time threshold, determining that the current animal target will collide with the vehicle equipment; When the collision time corresponding to the current animal target is greater than a preset collision time threshold and the absolute value of the first horizontal coordinate is smaller than the absolute value of the second horizontal coordinate, it is determined that the current animal target has crossed the front area of ​​the vehicle equipment, wherein the first horizontal coordinate indicates the horizontal coordinate of the center point of the detection box of the current animal target in the current frame, and the second horizontal coordinate indicates the horizontal coordinate of the center point of the detection box of the current animal target in a preset historical frame before the current frame.

9. The vehicle warning method based on animal collision as claimed in claim 1, characterized in that: The preset collision time threshold includes collision time thresholds of multiple different sensitivity levels, wherein the higher the sensitivity, the greater the collision time threshold.

10. The vehicle warning method based on animal collision as claimed in claim 1, characterized in that: The performing of early warning based on the current motion state of the animal target includes: Determining an alarm type indicating a danger level in the area ahead of the vehicle device based on at least one of the following data, wherein the at least one data includes detection data of a current animal target, a motion state of the current animal target, and activity data of the current animal target at the location of the vehicle device; According to the alarm type, different levels of early warning alerts are executed.

11. The vehicle warning method based on animal collision as claimed in claim 10, characterized in that: The alarm type includes at least one of the following: animal collision, animal crossing, animal group, animal detection, and animal activity area, wherein the animal collision indicates that the current animal target will collide with the vehicle equipment, the animal crossing indicates that the current animal target crosses the front area of ​​the vehicle equipment, the animal group indicates that the current animal targets are greater than a preset number, and the animal activity area indicates that the vehicle equipment has entered an animal infestation area.

12. The vehicle warning method based on animal collision according to claim 10 or 11, characterized in that: The execution of different levels of early warning alerts according to the alarm type includes: When there are multiple types of alarms, early warning alerts are performed based on the alarm priorities.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the vehicle warning method based on animal collision as claimed in any one of claims 1 to 12 is implemented.

14. A vehicle device, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the vehicle warning method based on animal collision as claimed in any one of claims 1 to 12.

Images