Target collision warning method and device, electronic equipment and storage medium

Abstract

The application discloses a target collision warning method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a to-be-detected image collected by a vehicle camera, performing target detection on the to-be-detected image by using a preset target detection algorithm, and obtaining a target detection result; determining a sub-warning area where the target is located and a residence state of the target in the sub-warning area according to the target detection result and the positions of each sub-warning area in a warning area corresponding to the vehicle camera; determining whether the target meets a preset warning condition corresponding to the sub-warning area according to the residence state of the target in the sub-warning area; and triggering a target collision warning reminder in the case that the target meets the preset warning condition corresponding to the sub-warning area. The algorithm is simple, and collision warning can be performed in time and accurately without calculating the actual distance of the target. In addition, only the sub-warning area where the target falls needs to be concerned, so that the stability tolerance of the target detection frame is high, and the warning responsiveness is good.

Description

Technical Field

[0001] This application relates to the field of vehicle safety technology, and in particular to a target collision warning method, device, electronic device, and storage medium. Background Technology

[0002] Advanced Driver Assistance Systems (ADAS) are a hot research topic in the field of vehicle safety. Collision warning is one of the important functions of this system, which can effectively reduce the frequency of vehicle-pedestrian collisions caused by driver inattention or illegal crossing of vehicles.

[0003] Traditional target collision warning schemes based on monocular vision sensors have the following shortcomings: 1) Monocular imaging loses one-dimensional information, resulting in inaccurate ranging, which in turn leads to inaccurate estimated position and velocity; 2) The detection box based on target detection is unstable, which exacerbates the inaccuracy of positioning; 3) The false alarm rate is high.

[0004] The current common solution for target collision warning is to perform cross-intersection over union (CUI) matching on the 2D target detection results output by the target detection model to obtain the target time series. Then, the distance between the vehicle and the target is calculated by using intrinsic and extrinsic parameters. The actual distance and relative speed of the target are estimated by using Kalman filtering, the collision time is calculated, and finally, the result is compared with the alarm threshold to determine whether to trigger an alarm.

[0005] However, this algorithm requires high stability of the output target detection box, and monocular ranging itself is unstable and greatly affected by intrinsic and extrinsic parameters. Furthermore, Kalman filtering has a time delay; over-reliance on observations leads to frequent false alarms, while over-reliance on the system results in delayed alarms. Summary of the Invention

[0006] This application provides a target collision warning method, device, electronic device, and storage medium to improve the accuracy and timeliness of target collision warning.

[0007] The embodiments of this application adopt the following technical solutions:

[0008] In a first aspect, embodiments of this application provide a target collision warning method, wherein the method includes:

[0009] The system acquires the image to be detected captured by the vehicle camera, and performs target detection on the image using a preset target detection algorithm to obtain the target detection result.

[0010] Based on the target detection results and the location of each sub-warning area in the warning area corresponding to the vehicle camera, determine the sub-warning area where the target is located and the target's dwelling status in the sub-warning area;

[0011] Based on the target's residence status in the sub-early warning area, determine whether the target meets the preset early warning conditions corresponding to the sub-early warning area;

[0012] If the target meets the preset warning conditions corresponding to the sub-warning area, a target collision warning alert is triggered.

[0013] Optionally, determining the sub-warning area where the target is located and the target's dwelling state in the sub-warning area based on the target detection result and the position of each sub-warning area in the corresponding warning area of ​​the vehicle camera includes:

[0014] Determine the size of the warning area corresponding to the vehicle camera, and determine the location of the warning area based on the installation location of the vehicle camera;

[0015] The warning area is divided into multiple sub-warning areas using a preset area division strategy, and the locations of the multiple sub-warning areas are determined based on the location of the warning area.

[0016] Optionally, the location of the sub-warning area includes the boundary location of the sub-warning area, and the preset warning conditions corresponding to the sub-warning area are obtained in the following way:

[0017] Based on the relative position of the sub-warning area boundary and the vehicle camera's projection point on the ground, and the preset collision time, the dwell time corresponding to the sub-warning area is determined.

[0018] The number of dwell times corresponding to the sub-warning area is determined based on the dwell time and the frame rate of the vehicle camera.

[0019] The number of times a person stays in a sub-early warning area is used as the preset early warning condition for that sub-early warning area.

[0020] Optionally, determining the dwell time corresponding to the sub-warning area based on the relative position of the sub-warning area boundary and the vehicle camera's projection point on the ground, and a preset collision time, includes:

[0021] Determine the ordinate of the boundary position of the sub-warning area in the image coordinate system, and the ordinate of the vanishing point of the vehicle camera in the image coordinate system;

[0022] Obtain the focal length of the vehicle camera and the vertical distance from the vehicle camera to the ground;

[0023] Based on the ordinate of the sub-warning area boundary position in the image coordinate system, the ordinate of the vanishing point of the vehicle camera in the image coordinate system, the focal length of the vehicle camera, and the vertical distance from the vehicle camera to the ground, determine the distance from the sub-warning area boundary position to the projection point of the vehicle camera on the ground.

[0024] The dwell time corresponding to the sub-warning area is determined based on the distance from the boundary of the sub-warning area to the projection point of the vehicle camera on the ground, and the preset collision time.

[0025] Optionally, determining the dwell time corresponding to the sub-warning area based on the relative position of the sub-warning area boundary and the vehicle camera's projection point on the ground, and a preset collision time, includes:

[0026] Determine the distance from the upper boundary of the sub-warning area to the projection point of the vehicle camera on the ground, and the distance from the lower boundary of the sub-warning area to the projection point of the vehicle camera on the ground;

[0027] The dwell time corresponding to the sub-warning area is determined based on the distance from the upper boundary of the sub-warning area to the ground projection point of the vehicle camera, the distance from the lower boundary of the sub-warning area to the ground projection point of the vehicle camera, and a preset collision time.

[0028] Optionally, the image to be detected includes multiple consecutive frames of images to be detected, and the target detection result includes multiple target detection results corresponding to the multiple consecutive frames of images to be detected. Determining the sub-warning area where the target is located and the target's dwell state in the sub-warning area based on the target detection result and the position of each sub-warning area in the warning area corresponding to the vehicle camera includes:

[0029] Based on the multiple target detection results corresponding to the consecutive multiple frames of images to be detected and the position of each sub-warning area in the warning area corresponding to the vehicle camera, the sub-warning area where the target is located and the number of times it stays in the sub-warning area are determined.

[0030] The stationing status of the target in the sub-early warning area is determined based on the number of times the target stays in the sub-early warning area.

[0031] Optionally, the multi-frame image to be detected includes the current frame image and the previous frame image. The step of determining the sub-warning area where the target is located and the number of times it stays in the sub-warning area based on the multiple target detection results corresponding to the consecutive multi-frame image to be detected and the positions of each sub-warning area in the warning area corresponding to the vehicle camera includes:

[0032] The target detection result of the current frame image is matched with the target detection result of the previous frame image;

[0033] The target tracking queue is updated according to the matching results. The target tracking queue is used to store the target detection box of the same target in each frame of the image to be detected.

[0034] The sub-warning area where the target is located is determined based on the position of the lower boundary midpoint of the target detection box in the current frame image and the position of each sub-warning area in the warning area corresponding to the vehicle camera;

[0035] The number of times the target has stayed in the sub-early warning area is updated based on the sub-early warning area where the target is located and the target tracking queue.

[0036] Optionally, the dwell status includes the number of dwell times, and determining whether the target meets the preset warning conditions corresponding to the sub-warning area based on the target's dwell status in the sub-warning area includes:

[0037] Compare the number of times the target stays in the sub-early warning area with the number of times it stays in the corresponding sub-early warning area;

[0038] If the number of times the target stays in the sub-early warning area reaches the number of times it stays in the sub-early warning area, then the target is determined to meet the preset early warning conditions corresponding to the sub-early warning area.

[0039] Otherwise, it is determined that the target does not meet the preset warning conditions corresponding to the sub-warning area.

[0040] Secondly, embodiments of this application also provide a target collision warning device, wherein the device includes:

[0041] The target detection unit is used to acquire the image to be detected captured by the vehicle camera, and to perform target detection on the image to be detected using a preset target detection algorithm to obtain the target detection result;

[0042] The first determining unit is used to determine the sub-warning area where the target is located and the dwelling state of the target in the sub-warning area based on the target detection result and the position of each sub-warning area in the warning area corresponding to the vehicle camera;

[0043] The second determining unit is used to determine whether the target meets the preset warning conditions corresponding to the sub-warning area based on the target's residence status in the sub-warning area;

[0044] The warning unit is used to trigger a target collision warning alert when the target meets the preset warning conditions corresponding to the sub-warning area.

[0045] Thirdly, embodiments of this application also provide an electronic device, including:

[0046] Processor; and

[0047] A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform any of the methods described above.

[0048] Fourthly, embodiments of this application also provide a computer-readable storage medium that stores one or more programs, which, when executed by an electronic device including multiple applications, cause the electronic device to perform any of the methods described above.

[0049] The above-mentioned at least one technical solution adopted in the embodiments of this application can achieve the following beneficial effects: The target collision warning method of the embodiments of this application first acquires the image to be detected captured by the vehicle camera, and performs target detection on the image to be detected using a preset target detection algorithm to obtain the target detection result; then, based on the target detection result and the position of each sub-warning area in the warning area corresponding to the vehicle camera, the sub-warning area where the target is located and the target's dwelling state in the sub-warning area are determined; then, based on the target's dwelling state in the sub-warning area, it is determined whether the target meets the preset warning conditions corresponding to the sub-warning area; finally, if the target meets the preset warning conditions corresponding to the sub-warning area, a target collision warning reminder is triggered. The target collision warning method of the embodiments of this application is simple to implement and can provide timely and accurate collision alarms without calculating the actual distance of the target. In addition, since it only needs to focus on which sub-warning area the target falls in, it has a high tolerance for the stability of the target detection box and good warning response. Attached Figure Description

[0050] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0051] Figure 1 This is a flowchart illustrating a target collision warning method according to an embodiment of this application;

[0052] Figure 2 This is a schematic diagram illustrating the division effect of a preset warning area in an embodiment of this application;

[0053] Figure 3 This is a schematic diagram of the structure of a target collision warning device according to an embodiment of this application;

[0054] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0056] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0057] This application provides a target collision warning method, such as... Figure 1 The diagram shows a flowchart of a target collision warning method according to an embodiment of this application. The method includes at least the following steps S110 to S140:

[0058] Step S110: Obtain the image to be detected captured by the vehicle camera, and perform target detection on the image to be detected using a preset target detection algorithm to obtain the target detection result.

[0059] The target collision warning method of this application embodiment can be executed by the vehicle. When performing target collision warning, it is necessary to first acquire the image to be detected captured by the vehicle camera, and then use a pre-trained target detection model to perform target detection on the image to be detected to obtain the target detection result. Here, the "target" can be a pedestrian, an animal, or other obstacles that pose a collision hazard.

[0060] When training an object detection model, deep learning algorithms such as YOLO V3 and SSD (Single Shot MultiBox Detector) can be used to train a 2D object detection model. Then, the trained 2D object detection model is used to detect the position and category information of the object in the image. Generally, the output information of the 2D object detection model can be represented as (x1, y1, x2, y2, class, score), where x1, y1, x2, y2 represent the pixel position of the object in the image, (x1, y1) corresponds to the upper left corner of the detection box, (x2, y2) corresponds to the lower right corner of the detection box, class corresponds to the category of the object, and score is the confidence score of the object, which is generally between 0 and 1. The higher the confidence score, the more confident there is in the accuracy of the object.

[0061] In practical applications, targets of a corresponding category with a confidence level greater than a preset threshold can be selected from the 2D target detection results according to different collision warning scenarios and requirements. These targets will serve as inputs for subsequent warning processes. For example, if a pedestrian collision warning is issued, targets classified as pedestrians with a confidence level greater than 0.5 can be selected from the 2D target detection results as inputs for subsequent warning processes.

[0062] The above steps can be seen as a real-time process. The vehicle camera performs target detection every time it captures a frame of image to ensure the timeliness of collision warning.

[0063] Step S120: Based on the target detection results and the positions of each sub-warning area in the warning area corresponding to the vehicle camera, determine the sub-warning area where the target is located and the target's residence status in the sub-warning area.

[0064] This application embodiment predefines a warning area for the vehicle, including its size and location, wherein the location of the warning area is continuously updated as the vehicle moves. Furthermore, since collision warnings require high timeliness, this application embodiment further divides the predefining warning area into multiple sub-warning areas, thereby improving the sensitivity and timeliness of collision warning detection.

[0065] Specifically, based on the position of the target detection box and the position of each sub-warning area in the target detection results, the sub-warning area into which the target has currently fallen and the target's dwelling state in that sub-warning area can be determined. The dwelling state can be understood as the time the target stays in the sub-warning area or the number of times it appears, etc., to quantify the possibility of the target being collided in the sub-warning area.

[0066] Step S130: Based on the target's residence status in the sub-early warning area, determine whether the target meets the preset early warning conditions corresponding to the sub-early warning area.

[0067] In this embodiment, the sub-warning zones can be divided based on their relative distance from the vehicle. Different relative distances correspond to different warning requirements. For example, the closer a sub-warning zone is to the vehicle, the stricter the warning requirements will be; that is, different sub-warning zones have their own corresponding warning conditions.

[0068] Based on this, after obtaining the dwell status of the target in the sub-warning area, the embodiments of this application only need to determine whether the target of interest meets the preset warning conditions corresponding to the sub-warning area, without needing to calculate the actual distance between the target and the vehicle, which reduces the implementation difficulty of the algorithm, and has a high tolerance for the stability of the target detection results and good warning response.

[0069] Step S140: If the target meets the preset warning conditions corresponding to the sub-warning area, trigger a target collision warning reminder.

[0070] If the target of interest meets the preset warning conditions corresponding to the sub-warning area, it means that the target's presence in that sub-warning area has reached the collision probability threshold, and therefore a target collision warning should be triggered to allow the driver or vehicle to take timely measures to avoid danger. Conversely, if the target of interest does not meet the preset warning conditions corresponding to the sub-warning area, it can be considered that the target's presence in that sub-warning area is still safe, and therefore no warning is required.

[0071] The target collision warning method in this application is simple to implement and can issue timely and accurate collision warnings without calculating the actual distance to the target. Furthermore, since it only needs to focus on which sub-warning area the target falls into, it has a high tolerance for the stability of the target detection box and good warning response.

[0072] In one embodiment of this application, determining the sub-warning area where the target is located and the target's dwelling state in the sub-warning area based on the target detection result and the positions of each sub-warning area in the warning area corresponding to the vehicle camera includes: determining the size of the warning area corresponding to the vehicle camera, and determining the position of the warning area based on the installation position of the vehicle camera; dividing the warning area into multiple sub-warning areas using a preset area division strategy, and determining the positions of the multiple sub-warning areas based on the position of the warning area.

[0073] In determining the warning area corresponding to a vehicle, this application embodiment can flexibly set the warning area according to actual warning requirements and traffic regulations. For example, a rectangular area consisting of 80 meters in front of the vehicle and 30cm on the outer edges of the left and right sides of the vehicle can be used as the warning area. Assuming a vehicle coordinate system is constructed with the center of the rear axle of the vehicle as the origin, the vehicle's driving direction as the y-axis, and the rightward direction perpendicular to the vehicle's driving direction as the x-axis, and the vehicle width as w meters, the relative position of the camera installed at the upper left corner of the vehicle in the vehicle coordinate system is (x1, x2). Then, the coordinates of the four vertices of the alarm area in the vehicle coordinate system can be determined as (-0.3+x1, x2), (0.3+w+x1, x2), (0.3+w+x1, x2+80), and (-0.3+x1, x2+80).

[0074] Since the position of the target detection box in the target detection result is the pixel position in the image, the embodiments of this application can further calculate the pixel positions in the image corresponding to the four vertices of the alarm area based on the principle of similar triangles. Without considering camera distortion, the area formed by the four vertices in the image can be regarded as an isosceles trapezoidal area. Then, a certain area division strategy is used to divide the isosceles trapezoidal area. For example, it can be divided at equal intervals in the vertical direction or at unequal intervals. The general principle is that the pixel interval corresponds to the actual distance. As the actual distance is farther, the pixel interval is larger.

[0075] Furthermore, since the coordinates of the four vertices of the entire warning area can be determined, the positions of each sub-warning area can be further determined based on the divided intervals. This allows for the acquisition of multiple sub-warning areas with different relative distances to the vehicle and their locations. For example... Figure 2 As shown, a schematic diagram illustrating the division effect of a preset warning area in an embodiment of this application is provided.

[0076] In one embodiment of this application, the location of the sub-warning area includes the location of the area boundary of the sub-warning area, and the preset warning condition corresponding to the sub-warning area is obtained in the following way: the dwell time corresponding to the sub-warning area is determined according to the relative position relationship between the area boundary of the sub-warning area and the projection point of the vehicle camera on the ground, and the preset collision time; the number of dwell times corresponding to the sub-warning area is determined according to the dwell time and the acquisition frame rate of the vehicle camera; and the number of dwell times corresponding to the sub-warning area is used as the preset warning condition corresponding to the sub-warning area.

[0077] As mentioned earlier, different sub-warning zones have their corresponding warning conditions. In determining the warning conditions for each sub-warning zone in this embodiment, the boundary position of the sub-warning zone can be determined first, and then the relative position of the boundary position of the sub-warning zone with the projection point of the vehicle camera on the ground and the preset collision time TTC can be determined to calculate the dwell time T corresponding to the sub-warning zone. The preset collision time TTC refers to the time it takes for the vehicle to collide with the target, which can be determined according to current traffic regulations, for example, it is generally set to 2.7s. The dwell time represents the maximum time that the target is allowed to stay in the sub-warning zone.

[0078] Then, the dwell time T corresponding to the sub-warning area can be further calculated by multiplying the dwell time T of the sub-warning area by the frame rate of the vehicle camera. This dwell time is used to characterize the maximum number of times the target is allowed to appear in the sub-warning area. Finally, the dwell time can be used as a preset warning condition for the sub-warning area to measure whether the target's dwell state meets the corresponding warning condition.

[0079] It should be noted that although both "dwelling time" and "number of dwell times" can reflect the extent to which a target is allowed to remain in a sub-warning area, the embodiment of this application uses "number of dwell times" as the warning condition because the statistics of the number of dwell times are relatively simpler, while dwell time is a data that needs to be counted by timing and is easily affected by the image acquisition frequency. Of course, those skilled in the art can also flexibly adjust the setting of the warning condition according to actual needs.

[0080] In one embodiment of this application, determining the dwell time corresponding to the sub-warning area based on the relative position of the sub-warning area boundary position and the vehicle camera's projection point on the ground, and a preset collision time, includes: determining the distance from the upper boundary position of the sub-warning area to the vehicle camera's projection point on the ground, and the distance from the lower boundary position of the sub-warning area to the vehicle camera's projection point on the ground; and determining the dwell time corresponding to the sub-warning area based on the distances from the upper boundary position of the sub-warning area to the vehicle camera's projection point on the ground, the distances from the lower boundary position of the sub-warning area to the vehicle camera's projection point on the ground, and the preset collision time.

[0081] In this embodiment of the application, the dwell time corresponding to each sub-early warning area can be determined in the following way:

[0082]

[0083] Wherein, d2 is the actual distance from the upper boundary of the sub-warning area to the ground projection point of the vehicle camera, and d1 is the actual distance from the lower boundary of the sub-warning area to the ground projection point of the vehicle camera.

[0084] To facilitate understanding of the embodiments of this application, the derivation process of the above formula (1) is further elaborated here. Let the uniform speed of the vehicle be v, then we can obtain:

[0085] T=(d2-d1) / v, (2)

[0086] Using the midpoint between the upper and lower boundaries of the sub-warning area as a reference, we can obtain:

[0087] v=(d2+d1) / 2*TTC, (3)

[0088] Substituting formula (3) into formula (2) yields the above formula (1).

[0089] In one embodiment of this application, determining the dwell time corresponding to the sub-warning area based on the relative position of the sub-warning area's boundary position and the vehicle camera's projection point on the ground, and a preset collision time, includes: determining the ordinate of the sub-warning area's boundary position in the image coordinate system and the ordinate of the vehicle camera's vanishing point in the image coordinate system; obtaining the vehicle camera's focal length and the vehicle camera's vertical distance from the ground; determining the distance from the sub-warning area's boundary position to the vehicle camera's projection point on the ground based on the ordinate of the sub-warning area's boundary position in the image coordinate system, the vehicle camera's vanishing point in the image coordinate system, the vehicle camera's focal length, and the vehicle camera's vertical distance from the ground; and determining the dwell time corresponding to the sub-warning area based on the distance from the sub-warning area's boundary position to the vehicle camera's projection point on the ground, and a preset collision time.

[0090] To facilitate understanding of the above embodiments, this application further elaborates on the calculation method for the actual distance d from the boundary position of the sub-warning area to the projection point of the vehicle camera on the ground. Let Fcam be the focal length of the vehicle camera, and Hcam be the distance from the installation position of the vehicle camera to the horizontal line of the ground. Assuming that the vehicle camera is installed perpendicular to the ground, then based on the principle of similar triangles, we can obtain:

[0091]

[0092] Among them, y b Let y be the pixel value corresponding to the ordinate of the sub-warning area's boundary location in the image. h This represents the pixel value corresponding to the vertical coordinate of the camera vanishing point in the image.

[0093] It should be noted that the above formula (4) is applicable when the camera installation angle is consistent with the ground horizontal line or the angle deviation is negligible. When the angle deviation is relatively large, the pitch angle of the camera installation also needs to be considered.

[0094] Based on the above formula (4), it can be seen that there is a correspondence between the actual distance d from the boundary of the sub-warning area to the projection point of the vehicle camera on the ground and the pixel value corresponding to the vertical coordinate of the boundary of the sub-warning area in the image. Therefore, it can be further obtained that:

[0095]

[0096]

[0097] Substituting formulas (5) and (6) into formula (1), we get:

[0098]

[0099] It can be seen that, to calculate the dwell time T, only the pixel value y corresponding to the vertical coordinate of the upper boundary of the sub-warning area in the image is needed. b1 The pixel value y corresponding to the vertical coordinate of the upper boundary position of the region in the image. b2 And the pixel value y corresponding to the vertical coordinate of the camera vanishing point in the image. h That's it. For y h The calculation can be obtained from the intersection of two lane lines on the ground. For example, the lane line fitting equation of the two lane lines can be obtained based on the lane line recognition result, and then the coordinates of the intersection of the two lane lines can be solved based on the lane line fitting equation of the two lane lines.

[0100] In one embodiment of this application, the image to be detected includes multiple consecutive frames of images to be detected, and the target detection result includes multiple target detection results corresponding to the multiple consecutive frames of images to be detected. The step of determining the sub-warning area where the target is located and the dwelling state of the target in the sub-warning area based on the target detection results and the positions of each sub-warning area in the warning area corresponding to the vehicle camera includes: determining the sub-warning area where the target is located and the number of times it dwells in the sub-warning area based on the multiple target detection results corresponding to the multiple consecutive frames of images to be detected and the positions of each sub-warning area in the warning area corresponding to the vehicle camera; and determining the dwelling state of the target in the sub-warning area based on the number of times the target dwells in the sub-warning area.

[0101] In this embodiment, when determining the dwelling state of a target within a sub-warning area, the image to be detected, captured in real-time by the vehicle's camera, can be continuously detected. Each time a target detection result is obtained, the sub-warning area in which the target has fallen can be determined. Thus, the number of times the target falls into a sub-warning area can be statistically analyzed, and this count can be used as the target's dwelling state within that area. In other words, this embodiment is a process of detection and judgment occurring simultaneously. Once the target's dwelling state within a sub-warning area triggers the corresponding warning condition, an alarm can be issued.

[0102] In one embodiment of this application, the multi-frame image to be detected includes a current frame image and a previous frame image. The step of determining the sub-warning region where the target is located and the number of times it resides in the sub-warning region based on multiple target detection results corresponding to the consecutive multi-frame images to be detected and the positions of each sub-warning region in the warning area corresponding to the vehicle camera includes: matching the target detection results of the current frame image with the target detection results of the previous frame image; updating the target tracking queue based on the matching results, the target tracking queue being used to store target detection boxes of the same target in each frame of the image to be detected; determining the sub-warning region where the target is located based on the position of the lower boundary midpoint of the target detection box in the current frame image and the positions of each sub-warning region in the warning area corresponding to the vehicle camera; and updating the number of times the target resides in the sub-warning region based on the sub-warning region where the target is located and the target tracking queue.

[0103] In this application embodiment, determining the number of times a target resides in a sub-warning area is a cumulative calculation process. Specifically, for the current frame image captured by the vehicle camera, the target detection model can be used to detect the target detection result of the current frame image. Then, the target detection result of the previous frame image is obtained, and the two target detection results are matched. For example, the intersection-over-union (IoU) ratio of the target detection boxes can be used to determine the matching targets in two adjacent frames, which are considered to be the same target.

[0104] For each target detected in an image, a corresponding target tracking queue is maintained in advance. This target tracking queue is used to store the target detection results of the same target in each frame of the image to be detected, such as the target ID and the corresponding target detection box.

[0105] Therefore, based on the target tracking queue, the position of the target detection box of the same target in each frame of the image can be determined. Then, the position of the target detection box can be compared with the position of each sub-warning area in the warning area corresponding to the vehicle camera, thereby determining the sub-warning area that the target has currently fallen into, and updating the number of times the target has stayed in the corresponding sub-warning area.

[0106] In one embodiment of this application, the dwelling state includes the number of dwellings. Determining whether the target meets the preset warning conditions corresponding to the sub-warning area based on the dwelling state of the target in the sub-warning area includes: comparing the number of dwellings of the target in the sub-warning area with the number of dwellings corresponding to the sub-warning area; if the number of dwellings of the target in the sub-warning area reaches the number of dwellings corresponding to the sub-warning area, then it is determined that the target meets the preset warning conditions corresponding to the sub-warning area; otherwise, it is determined that the target does not meet the preset warning conditions corresponding to the sub-warning area.

[0107] As described in the previous embodiments, this application sets a corresponding number of dwell times for each sub-warning area. The number of dwell times for each sub-warning area is the maximum number of times a target is allowed to fall into that sub-warning area, which can be regarded as a safety threshold. Therefore, if the target's current number of dwell times in a sub-warning area has reached the maximum number of dwell times corresponding to that sub-warning area, it means that if the target continues to stay, it may face the risk of collision. That is, the target meets the preset warning conditions corresponding to the sub-warning area. At this time, a collision warning reminder needs to be triggered so that the driver or vehicle can take timely action.

[0108] Of course, in this embodiment of the application, a collision warning reminder can also be triggered only when the target reaches the preset alarm condition corresponding to the sub-warning area two or more times in a row, thereby reducing the false alarm rate.

[0109] This application also provides a target collision warning device 300, such as... Figure 3 The diagram shows a target collision warning device according to an embodiment of this application. The device 300 includes: a target detection unit 310, a first determination unit 320, a second determination unit 330, and a warning unit 340, wherein:

[0110] The target detection unit 310 is used to acquire the image to be detected captured by the vehicle camera, and to perform target detection on the image to be detected using a preset target detection algorithm to obtain the target detection result;

[0111] The first determining unit 320 is used to determine the sub-warning area where the target is located and the dwelling state of the target in the sub-warning area based on the target detection result and the position of each sub-warning area in the warning area corresponding to the vehicle camera;

[0112] The second determining unit 330 is used to determine whether the target meets the preset warning conditions corresponding to the sub-warning area based on the target's residence status in the sub-warning area;

[0113] The warning unit 340 is used to trigger a target collision warning reminder when the target meets the preset warning conditions corresponding to the sub-warning area.

[0114] In one embodiment of this application, the first determining unit 320 is specifically used to: determine the size of the warning area corresponding to the vehicle camera, and determine the position of the warning area according to the installation position of the vehicle camera; divide the warning area into multiple sub-warning areas using a preset area division strategy, and determine the positions of the multiple sub-warning areas according to the position of the warning area.

[0115] In one embodiment of this application, the location of the sub-warning area includes the location of the area boundary of the sub-warning area, and the preset warning condition corresponding to the sub-warning area is obtained in the following way: the dwell time corresponding to the sub-warning area is determined according to the relative position relationship between the area boundary of the sub-warning area and the projection point of the vehicle camera on the ground, and the preset collision time; the number of dwell times corresponding to the sub-warning area is determined according to the dwell time and the acquisition frame rate of the vehicle camera; and the number of dwell times corresponding to the sub-warning area is used as the preset warning condition corresponding to the sub-warning area.

[0116] In one embodiment of this application, the preset warning conditions corresponding to the sub-warning area are obtained as follows: The ordinate of the sub-warning area's boundary position in the image coordinate system and the ordinate of the vanishing point of the vehicle camera in the image coordinate system are determined; the focal length of the vehicle camera and the vertical distance from the vehicle camera to the ground are obtained; based on the ordinate of the sub-warning area's boundary position in the image coordinate system, the ordinate of the vehicle camera's vanishing point in the image coordinate system, the focal length of the vehicle camera, and the vertical distance from the vehicle camera to the ground, the distance from the sub-warning area's boundary position to the vehicle camera's projection point on the ground is determined; based on the distance from the sub-warning area's boundary position to the vehicle camera's projection point on the ground, and a preset collision time, the dwell time corresponding to the sub-warning area is determined.

[0117] In one embodiment of this application, the preset warning conditions corresponding to the sub-warning area are obtained as follows: the distance from the upper boundary of the sub-warning area to the ground projection point of the vehicle camera, and the distance from the lower boundary of the sub-warning area to the ground projection point of the vehicle camera are determined; based on the distance from the upper boundary of the sub-warning area to the ground projection point of the vehicle camera, the distance from the lower boundary of the sub-warning area to the ground projection point of the vehicle camera, and a preset collision time, the dwell time corresponding to the sub-warning area is determined.

[0118] In one embodiment of this application, the image to be detected includes multiple consecutive frames of images to be detected, and the target detection result includes multiple target detection results corresponding to the multiple consecutive frames of images to be detected. The first determining unit 320 is specifically used to: determine the sub-warning area where the target is located and the number of times it stays in the sub-warning area based on the multiple target detection results corresponding to the multiple consecutive frames of images to be detected and the position of each sub-warning area in the warning area corresponding to the vehicle camera; and determine the dwelling state of the target in the sub-warning area based on the number of times the target stays in the sub-warning area.

[0119] In one embodiment of this application, the multiple frames of images to be detected include a current frame image and a previous frame image. The first determining unit 320 is specifically configured to: match the target detection result of the current frame image with the target detection result of the previous frame image; update the target tracking queue according to the matching result, the target tracking queue being used to store the target detection boxes of the same target in each frame of images to be detected; determine the sub-warning area where the target is located based on the position of the lower boundary midpoint of the target detection box of the current frame image and the position of each sub-warning area in the warning area corresponding to the vehicle camera; and update the number of times the target stays in the sub-warning area according to the sub-warning area where the target is located and the target tracking queue.

[0120] In one embodiment of this application, the dwelling state includes the number of dwellings, and the second determining unit 330 is specifically used to: compare the number of dwellings of the target in the sub-early warning area with the number of dwellings corresponding to the sub-early warning area; if the number of dwellings of the target in the sub-early warning area reaches the number of dwellings corresponding to the sub-early warning area, then determine that the target meets the preset early warning condition corresponding to the sub-early warning area; otherwise, determine that the target does not meet the preset early warning condition corresponding to the sub-early warning area.

[0121] It is understood that the above-mentioned target collision warning device can realize each step of the target collision warning method provided in the foregoing embodiments. The relevant explanations of the target collision warning method are applicable to the target collision warning device, and will not be repeated here.

[0122] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 4At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0123] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0124] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0125] The processor reads the corresponding computer program from non-volatile memory into main memory and then runs it, forming a target collision warning device at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0126] The system acquires the image to be detected captured by the vehicle camera, and performs target detection on the image using a preset target detection algorithm to obtain the target detection result.

[0127] Based on the target detection results and the location of each sub-warning area in the warning area corresponding to the vehicle camera, determine the sub-warning area where the target is located and the target's dwelling status in the sub-warning area;

[0128] Based on the target's residence status in the sub-early warning area, determine whether the target meets the preset early warning conditions corresponding to the sub-early warning area;

[0129] If the target meets the preset warning conditions corresponding to the sub-warning area, a target collision warning alert is triggered.

[0130] The above is as stated in this application. Figure 1 The method executed by the target collision warning device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0131] The electronic device can also perform Figure 1 The method for implementing a target collision warning device, and the realization of the target collision warning device in... Figure 1 The functions of the embodiments shown are not described again in this application.

[0132] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by an electronic device including multiple applications, enable the electronic device to perform... Figure 1 The method executed by the target collision warning device in the illustrated embodiment is specifically used to perform:

[0133] The system acquires the image to be detected captured by the vehicle camera, and performs target detection on the image using a preset target detection algorithm to obtain the target detection result.

[0134] Based on the target detection results and the location of each sub-warning area in the warning area corresponding to the vehicle camera, determine the sub-warning area where the target is located and the target's dwelling status in the sub-warning area;

[0135] Based on the target's residence status in the sub-early warning area, determine whether the target meets the preset early warning conditions corresponding to the sub-early warning area;

[0136] If the target meets the preset warning conditions corresponding to the sub-warning area, a target collision warning alert is triggered.

[0137] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0138] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0139] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0140] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0141] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0142] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0143] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0144] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0145] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0146] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A target collision warning method, wherein, The method comprises: acquiring a to-be-detected image collected by a vehicle camera, and performing target detection on the to-be-detected image by using a preset target detection algorithm to obtain a target detection result; determining a sub-warning region where a target is located and a residence state of the target in the sub-warning region according to the target detection result and positions of each sub-warning region in a warning region corresponding to the vehicle camera; determining whether the target meets a preset warning condition corresponding to the sub-warning region according to the residence state of the target in the sub-warning region; in a case where the target meets the preset warning condition corresponding to the sub-warning region, triggering a target collision warning reminder; the position of the sub-warning region comprises a region boundary position of the sub-warning region, and the preset warning condition corresponding to the sub-warning region is obtained by the following manner: determining a residence time corresponding to the sub-warning region according to a relative position relationship between the region boundary position of the sub-warning region and a ground projection point of the vehicle camera and a preset collision time; determining a residence frequency corresponding to the sub-warning region according to the residence time and a collection frame rate of the vehicle camera; taking the residence frequency corresponding to the sub-warning region as the preset warning condition corresponding to the sub-warning region.

2. The method of claim 1, wherein, The determining of the sub-warning region where the target is located and the residence state of the target in the sub-warning region according to the target detection result and the positions of each sub-warning region in the warning region corresponding to the vehicle camera comprises: determining a size of the warning region corresponding to the vehicle camera, and determining a position of the warning region according to an installation position of the vehicle camera; performing region division on the warning region by using a preset region division strategy to obtain a plurality of sub-warning regions, and determining positions of the plurality of sub-warning regions according to the position of the warning region.

3. The method of claim 1, wherein, The determining of the residence time corresponding to the sub-warning region according to the relative position relationship between the region boundary position of the sub-warning region and the ground projection point of the vehicle camera and the preset collision time comprises: determining a vertical coordinate of the region boundary position of the sub-warning region in an image coordinate system and a vertical coordinate of a vanishing point of the vehicle camera in the image coordinate system; acquiring a focal length of the vehicle camera and a vertical distance from the vehicle camera to the ground; determining a distance from the region boundary position of the sub-warning region to the ground projection point of the vehicle camera according to the vertical coordinate of the region boundary position of the sub-warning region in the image coordinate system and the vertical coordinate of the vanishing point of the vehicle camera in the image coordinate system and the focal length of the vehicle camera and the vertical distance from the vehicle camera to the ground; determining the residence time corresponding to the sub-warning region according to the distance from the region boundary position of the sub-warning region to the ground projection point of the vehicle and the preset collision time.

4. The method of claim 1, wherein, The determining of the residence time corresponding to the sub-warning region according to the relative position relationship of the region boundary position of the sub-warning region and the ground projection point of the vehicle camera and a preset collision time comprises: determining distances from a region upper boundary position of the sub-warning area to a ground projection point of the vehicle camera and from a region lower boundary position of the sub-warning area to the ground projection point of the vehicle camera; determining a residence time corresponding to the sub-warning area according to the distances from the region upper boundary position of the sub-warning area to the ground projection point of the vehicle camera and from the region lower boundary position of the sub-warning area to the ground projection point of the vehicle camera, and a preset collision time.

5. The method of claim 1, wherein, The to-be-detected images include continuous multiple to-be-detected images, and the target detection results include multiple target detection results corresponding to the continuous multiple to-be-detected images. The determining the sub-warning area where the target is located and the residence state of the target in the sub-warning area according to the target detection results and the positions of each sub-warning area in the warning area corresponding to the vehicle camera includes: determining the sub-warning area where the target is located and a residence frequency of the target in the sub-warning area according to the multiple target detection results corresponding to the continuous multiple to-be-detected images and the positions of each sub-warning area in the warning area corresponding to the vehicle camera; determining the residence state of the target in the sub-warning area according to the residence frequency of the target in the sub-warning area.

6. The method of claim 5, wherein, The multiple to-be-detected images include a current frame image and a previous frame image. The determining the sub-warning area where the target is located and the residence frequency of the target in the sub-warning area according to the multiple target detection results corresponding to the continuous multiple to-be-detected images and the positions of each sub-warning area in the warning area corresponding to the vehicle camera includes: matching the target detection result of the current frame image with a target detection result of the previous frame image; updating a target tracking queue according to a matching result, the target tracking queue being used to store target detection boxes of a same target in each to-be-detected image; determining the sub-warning area where the target is located according to a lower boundary midpoint position of the target detection box of the current frame image and the positions of each sub-warning area in the warning area corresponding to the vehicle camera; updating the residence frequency of the target in the sub-warning area according to the sub-warning area where the target is located and the target tracking queue.

7. The method of claim 1, wherein, The residence state includes a residence frequency. The determining whether the target meets a preset warning condition corresponding to the sub-warning area according to the residence state of the target in the sub-warning area includes: comparing the residence frequency of the target in the sub-warning area with a residence frequency corresponding to the sub-warning area; if the residence frequency of the target in the sub-warning area reaches the residence frequency corresponding to the sub-warning area, it is determined that the target meets the preset warning condition corresponding to the sub-warning area; otherwise, it is determined that the target does not meet the preset warning condition corresponding to the sub-warning area.

8. A target collision warning device, wherein, The apparatus includes: a target detection unit, configured to acquire a to-be-detected image collected by a vehicle camera, and perform target detection on the to-be-detected image by using a preset target detection algorithm to obtain a target detection result; The first determination unit is configured to determine a sub-warning region where the target is located and a residence state of the target in the sub-warning region according to the target detection result and positions of each sub-warning region in a pre-warning region corresponding to the vehicle camera. The second determination unit is configured to determine whether the target meets a preset pre-warning condition corresponding to the sub-warning region according to the residence state of the target in the sub-warning region. The pre-warning unit is configured to trigger a target collision pre-warning reminder if the target meets the preset pre-warning condition corresponding to the sub-warning region. The position of the sub-warning region includes a region boundary position of the sub-warning region, and the preset pre-warning condition corresponding to the sub-warning region is obtained by the following way: According to a relative position relationship between the region boundary position of the sub-warning region and a ground projection point of the vehicle camera and a preset collision time, a residence time corresponding to the sub-warning region is determined. According to the residence time and a collection frame rate of the vehicle camera, a residence number corresponding to the sub-warning region is determined. The residence number corresponding to the sub-warning region is taken as the preset pre-warning condition corresponding to the sub-warning region. 9.An electronic device comprising: a processor; and a memory arranged to store computer-executable instructions that, when executed, cause the processor to perform the method of any one of claims 1-7. 10.A computer-readable storage medium storing one or more programs, the one or more programs, when executed by an electronic device including multiple applications, cause the electronic device to perform the method of any one of claims 1-7. ​

Images