Methods, devices, terminals, and storage media for selecting vehicle target collision points

Abstract

This invention provides a method, apparatus, terminal, and storage medium for selecting a vehicle target collision point. The method includes: performing target detection and semantic segmentation on an image; segmenting the detected target bounding box into the target portion as the foreground; obtaining the intersection points of the foreground within the target bounding box and the edge lines of the target bounding box; connecting the intersection points of every two adjacent edge lines to obtain a target line segment, wherein the target line segment includes at least one line segment; and determining the point closest to the vehicle based on the target line segment as the vehicle's target collision point. This invention can improve the accuracy of collision point selection.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving technology, and in particular to a method, device, terminal and storage medium for selecting a vehicle target collision point. Background Technology

[0002] The development of intelligent vehicles is of great significance in reducing traffic accidents and minimizing their injuries. One key technical challenge that intelligent vehicles need to address in complex traffic environments is collision warning and active collision avoidance. Precise selection of the collision point is a crucial factor in solving this technical challenge.

[0003] Existing technologies can determine the collision point using data collected by radar. However, in some applications, such as blind spot monitoring, cameras, such as fisheye lenses with a wide field of view, are often used for blind spot monitoring instead of radar.

[0004] When existing technologies determine collision points using cameras, the accuracy of collision point selection is low, resulting in inaccurate collision warnings and active collision avoidance. Summary of the Invention

[0005] In view of this, the present invention provides a method, device, terminal and storage medium for selecting vehicle target collision points, which can solve the problem of low accuracy when determining collision points through cameras.

[0006] In a first aspect, embodiments of the present invention provide a method for selecting a vehicle target collision point, including:

[0007] The image is subjected to object detection and semantic segmentation, and the portion of the detected object box is segmented into the object and used as the foreground.

[0008] Obtain the intersection point between the foreground within the target bounding box and the edge line of the target bounding box;

[0009] Connecting the intersection points of every two adjacent edges yields a target line segment, which includes at least one line segment.

[0010] Based on the target line segment, determine the point closest to the vehicle as the target collision point of the vehicle.

[0011] In one possible implementation, determining the point closest to the vehicle as the target collision point based on the target line segment includes:

[0012] For any one of the target line segments, divide the line segment into multiple sub-segments;

[0013] Select a point on each sub-segment;

[0014] Based on the mapping relationship between the world coordinate system and the pixel coordinate system, and the coordinate values ​​of each selected point in the pixel coordinate system, determine the coordinate values ​​of each point in the world coordinate system.

[0015] Based on the coordinates of each point in the world coordinate system, the point closest to the vehicle is determined as the first reference point;

[0016] The target region is determined in the pixel coordinate system with the first reference point as the center;

[0017] According to preset rules, a point in the target area that is closest to the vehicle is determined as the target collision point.

[0018] In one possible implementation, the method further includes:

[0019] The target collision point is used as the target tracking point to track the target;

[0020] If the tracking point of the current frame image undergoes a sudden change, the tracking point of the current frame image is updated based on the tracking point of the previous frame image and the tracking point of the current frame image. The sudden change in the tracking point means that the distance between the tracking point of the current frame image and the tracking point of the previous frame image in the world coordinate system exceeds a preset threshold.

[0021] In one possible implementation, updating the tracking points of the current frame image based on the tracking points of the previous frame image and the tracking points of the current frame image if a sudden change occurs in the tracking points of the current frame image includes:

[0022] Based on the coordinates of the tracking point in the pixel coordinate system and the position of the target box in the pixel coordinate system, the relative position of the tracking point in the target box is determined.

[0023] Obtain the pixel coordinates of the tracking point in the current frame image;

[0024] Based on the relative position of the tracking point in the target box of the previous frame image and the target box of the current frame image, determine the pixel coordinate value of the tracking point in the previous frame image in the current frame image;

[0025] Based on the mapping relationship between the pixel coordinate system and the world coordinate system, determine the first world coordinate value corresponding to the pixel coordinate value of the tracking point in the current frame image, and the second world coordinate value corresponding to the pixel coordinate value of the tracking point in the previous frame image in the current frame image.

[0026] If the distance between the first world coordinate value and the second world coordinate value exceeds a preset threshold, then the tracking points of the previous frame image and the tracking points of the current frame image are subjected to smoothing filtering to obtain the updated tracking points of the current frame image.

[0027] In one possible implementation, the method further includes:

[0028] If the tracking result of tracking the target in the current frame image does not include a target bounding box containing the target, then the tracking point obtained from the previous frame image is used as the second reference point.

[0029] Based on the coordinates of the second reference point in the world coordinate system and the mapping relationship between the pixel coordinate system and the world coordinate system, the coordinates of the second reference point in the pixel coordinate system are determined.

[0030] Based on the coordinates of the second reference point in the pixel coordinate system and the relative position of the target box corresponding to the second reference point in the previous frame image, the target box corresponding to the target in the current frame image is determined, and the second reference point is used as the tracking point of the vehicle in the current frame image.

[0031] Secondly, embodiments of the present invention provide a vehicle target collision point selection device, comprising: a first acquisition module, a second acquisition module, and a determination module;

[0032] The first acquisition module is used to perform object detection and semantic segmentation on the image, and to segment the detected target box into the part of the target as the foreground;

[0033] The second acquisition module is used to acquire the intersection points of the foreground within the target box and the edge line of the target box. The intersection points of every two adjacent edge lines are connected to obtain a target line segment, and the target line segment includes at least one line segment.

[0034] The determining module is used to determine the point closest to the vehicle based on the target line segment, as the target collision point of the vehicle.

[0035] In one possible implementation, the determining module is configured to:

[0036] For any one of the target line segments, divide the line segment into multiple sub-segments;

[0037] Select a point on each sub-segment;

[0038] Based on the mapping relationship between the world coordinate system and the pixel coordinate system, and the coordinate values ​​of each selected point in the pixel coordinate system, determine the coordinate values ​​of each point in the world coordinate system.

[0039] Based on the coordinates of each point in the world coordinate system, the point closest to the vehicle is determined as the first reference point;

[0040] The target region is determined in the pixel coordinate system with the first reference point as the center;

[0041] According to preset rules, a point in the target area that is closest to the vehicle is determined as the target collision point.

[0042] In one possible implementation, the determining module is further configured to:

[0043] The target collision point is used as the target tracking point to track the target;

[0044] If the tracking point of the current frame image undergoes a sudden change, the tracking point of the current frame image is updated based on the tracking point of the previous frame image and the tracking point of the current frame image. The sudden change in the tracking point means that the distance between the tracking point of the current frame image and the tracking point of the previous frame image in the world coordinate system exceeds a preset threshold.

[0045] Thirdly, embodiments of the present invention provide a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method as described in the first aspect or any possible implementation thereof.

[0046] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method as described in the first aspect or any possible implementation thereof.

[0047] The beneficial effects of the embodiments of the present invention compared with the prior art are as follows:

[0048] This invention performs target detection and semantic segmentation on images captured by vehicle-mounted cameras. Based on semantic segmentation, the background within the target bounding box is removed, and the point closest to the vehicle within the target bounding box is determined as the target collision point based on the foreground. This reduces the influence of the background on the target collision point selection process and improves the accuracy of target collision point selection. Based on the intersection of the foreground within the target bounding box and the edge line, the intersection of every two adjacent edge lines is connected to obtain the target line segment. The point closest to the vehicle is determined as the target collision point through the target line segment, which reduces the computational load of obtaining the target collision point. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating the implementation of a method for selecting a vehicle target collision point according to an embodiment of the present invention.

[0051] Figure 2 This is a schematic diagram of a target detection result provided in an embodiment of the present invention;

[0052] Figure 3 This is a schematic diagram of a method for selecting a vehicle target collision point provided in an embodiment of the present invention;

[0053] Figure 4 This is a flowchart illustrating the implementation of another method for selecting a vehicle target collision point provided in an embodiment of the present invention.

[0054] Figure 5 This is a flowchart illustrating the implementation of a tracking point update process provided in an embodiment of the present invention;

[0055] Figure 6 This is a schematic diagram illustrating the relative position of a tracking point within a target bounding box, provided by an embodiment of the present invention.

[0056] Figure 7 This is a schematic diagram of the structure of a vehicle target collision point selection device provided in an embodiment of the present invention;

[0057] Figure 8 This is a schematic diagram of the terminal provided in an embodiment of the present invention. Detailed Implementation

[0058] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0059] To make the objectives, technical solutions, and advantages of the present invention clearer, specific embodiments will be described below in conjunction with the accompanying drawings.

[0060] See Figure 1This document illustrates a flowchart of the vehicle target collision point selection method provided in an embodiment of the present invention. The method provided in this embodiment is applicable to both passenger vehicles and commercial vehicles. Details are as follows:

[0061] In step 101, target detection and semantic segmentation are performed on the image, and the portion of the detected target box that is segmented into the target portion is used as the foreground.

[0062] The method provided in this embodiment of the invention is applicable to any scenario where target tracking and target collision point determination are performed based on images acquired by a vehicle-mounted camera. This embodiment of the invention does not limit the installation location or type of the vehicle-mounted camera.

[0063] Vehicle-mounted cameras can be either ordinary cameras or fisheye lenses. In some application scenarios, such as when there is a large blind spot on the right side of a commercial vehicle, fisheye lenses are suitable for monitoring such blind spots due to their advantages, including a wide field of view, broad coverage, resistance to lens dirt, and lower cost. They are typically installed on the side of the vehicle.

[0064] Since the targets detected during the image acquisition process from the vehicle-mounted camera are mostly vehicles, and vehicle targets occupy a certain space, they cannot be simply considered as point masses. When other vehicles approach the vehicle, those other vehicles become the targets referred to in this step. Of course, the targets referred to in this embodiment of the invention can also be other targets besides vehicles, such as pedestrians or people using other means of transportation. In this embodiment of the invention, since vehicles are the primary targets, this embodiment of the invention is described using vehicles as targets; however, this embodiment of the invention does not limit the type of target.

[0065] The camera installed on the vehicle continuously captures images. For each frame, target detection is performed on the image to obtain the target detection result, which includes a bounding box containing the detected target.

[0066] Optionally, a target detection model can be pre-trained based on deep learning. After obtaining a frame of image, the frame of image is input into the target detection model to obtain the target detection result. Other target detection algorithms can also be used to perform target detection on the image, and this embodiment of the invention does not limit this.

[0067] Figure 2 This is a schematic diagram of a target detection result provided in an embodiment of the present invention, as shown below. Figure 2 As shown, ABCD is the image, the vehicles in the image are the targets, and EFGH are the target bounding boxes.

[0068] For the same frame of image, semantic segmentation is also required.

[0069] Semantic segmentation is an important branch of image processing and machine vision. Unlike classification tasks, semantic segmentation requires determining the category of each pixel in an image for precise segmentation.

[0070] Optionally, a semantic segmentation model can be pre-trained based on deep learning. After obtaining a frame of image, the frame can be input into the semantic segmentation model to obtain the semantic segmentation result of the frame. Semantic segmentation can also be performed using other semantic segmentation algorithms, which are not limited in this embodiment of the invention.

[0071] It should be noted that for the same frame of image, object detection can be performed first, followed by semantic segmentation, or semantic segmentation can be performed first, followed by object detection, or both can be performed simultaneously.

[0072] In this embodiment of the invention, taking road driving as an example and a vehicle as the target, the background is typically a road, lawn, sky, etc. Through this step, after removing the background of the target bounding box, the effective pixels of the target bounding box only include the pixels of the retained foreground, i.e., the pixels corresponding to the vehicle. Because this step removes the background, interference from the background is avoided in the subsequent determination of the target collision point, thus improving the accuracy of the target collision point.

[0073] In step 102, the intersection point of the foreground within the target box and the edge line of the target box is obtained. The intersection points of every two adjacent edge lines are connected to obtain the target line segment, which includes at least one line segment.

[0074] Optionally, a pixel coordinate system and a world coordinate system can be pre-defined. Images captured by a camera or webcam are typically two-dimensional digital images, stored as a pixel matrix, with each element representing the grayscale value of that pixel. The imaging plane coordinate system describes the position coordinates of an image point on the image plane, while the pixel coordinate system describes the point's position within the pixel matrix. The pixel coordinate system is established on the image plane, with its origin o typically located at the top left corner of the image plane. The world coordinate system is used to uniformly describe the position of bitmaps in the real three-dimensional world, including the camera's position, the target's position, etc.

[0075] In this embodiment of the invention, to simplify the calculation, the optional rule for setting the world coordinate system is: obtain a perpendicular line passing through the camera's mounting point and perpendicular to the ground, and take the intersection of the perpendicular line and the ground as the origin of the world coordinate system. Using the perpendicular line as the world coordinate system axis, The positive axis points upwards, in the world coordinate system. plane and The axis is perpendicular.

[0076] Based on this, the ground height is 0, and a point in the real world is in the world coordinate system. The value of the axis is the actual height of that point.

[0077] In one alternative implementation, a point is in the world coordinate system. The absolute value of the axis is used to represent the lateral distance from the vehicle, and the lateral distance is used to represent the vertical distance from the target to the side of the vehicle body. For example, in this embodiment of the invention, the camera is a fisheye lens, applied to the vehicle's BSD (Blind-Spot Detection) system, and mounted on the side of the vehicle. The target is then positioned in the world coordinate system. The absolute value of the axis is the distance from the side of the vehicle body. In this embodiment of the invention, a point in the world coordinate system... The absolute value of the axis value is used to represent the longitudinal distance from the vehicle. The longitudinal distance is used to represent the vertical distance of the target from the front of the vehicle or the parking space. For example, in this embodiment of the invention, the fisheye lens is mounted at the front of the vehicle, and the target is in the world coordinate system. The absolute value of the axle value is the distance from the front of the vehicle.

[0078] In another alternative implementation, a point is in the world coordinate system. The absolute value of the axis is used to represent the longitudinal distance from the vehicle; a point in the world coordinate system... The absolute value of the axis is used to represent the lateral distance from the vehicle. This embodiment of the invention does not limit this.

[0079] By using the camera's intrinsic and extrinsic parameters, a unique mapping relationship between the pixel coordinate system and the world coordinate system can be determined.

[0080] Since points in the foreground of the target bounding box correspond to points on the target, the point closest to the vehicle in the foreground is obtained as the target collision point of the vehicle.

[0081] Optionally, a camera can be mounted on the side of the vehicle, with the target in a world coordinate system. Taking the distance from the side of the vehicle body as an example, where the absolute value of the axis value is the distance, we obtain the coordinates of each point in the foreground in the world coordinate system based on the mapping relationship between the pixel coordinate system and the world coordinate system. Obtain the absolute value of the axis. The point with the smallest absolute value of the axis value is taken as the target collision point.

[0082] Alternatively, the coordinates of each pixel in the foreground in the world coordinate system can be obtained sequentially. The point with the smallest absolute value of the axis value is taken as the target collision point.

[0083] This method involves a large amount of computation. To reduce the computational load, alternatively, the foreground can be sequentially divided into multiple continuous but non-overlapping sub-regions. A point is selected within each sub-region, resulting in multiple selected points. The coordinates of each selected point in the world coordinate system are then calculated sequentially to obtain the desired result. The point with the smallest absolute value of the axis value is taken as the target collision point.

[0084] In another alternative implementation, to further reduce computational load, the intersection points of the foreground within the target bounding box and the edge lines of the target bounding box are obtained; the intersection points of every two adjacent edge lines are connected to obtain a target line segment, which includes at least one line segment; based on the target line segment, the point closest to the vehicle is determined as the target collision point of the vehicle.

[0085] Figure 3 This is a schematic diagram of a method for selecting a vehicle target collision point provided by an embodiment of the present invention; combined with Figure 3 Because the background within the target bounding box is removed, and only the foreground within the target bounding box is retained, the accuracy of identifying the intersection point between the foreground within the target bounding box and the target bounding box edge is higher. In this embodiment of the invention, the target bounding box edge is used to represent the line segment corresponding to one edge of the target bounding box. For example... Figure 3 As shown, the boundaries of the target bounding box are EF, FH, GH, and EG, and the intersections of the foreground within the target bounding box with the boundaries are points J, K, and L. Optionally, since the points corresponding to the upper boundary line are usually far from the actual distance of the vehicle during target tracking, they are generally not considered collision points. Therefore, the connection between the intersection of the upper boundary line and the intersection of the adjacent boundary lines can be omitted. In this case, the target line segments obtained are JK and KL.

[0086] In an embodiment of the present invention, Figure 3 In this example, the target box is a rectangle, which is only one example. The target box can also be other shapes, and the embodiments of the present invention do not limit this.

[0087] In step 103, based on the target line segment, the point closest to the vehicle is determined as the target collision point of the vehicle.

[0088] When the target volume is large, the points on the target line segment obtained by the method provided in this embodiment of the invention are definitely points on the target, and are points relatively close to the vehicle.

[0089] Optionally, obtain the coordinates of all pixels on the target line segment in the world coordinate system, so that the target is in the world coordinate system. Taking the absolute value of the axis as the distance from the side of the vehicle body as an example, we obtain... The point with the smallest absolute value of the axis value is taken as the target collision point.

[0090] Another optional implementation, to further reduce computational load, involves dividing any segment of the target line segment into multiple sub-segments; selecting a point on each sub-segment; determining the coordinates of each point in the world coordinate system based on the mapping relationship between the world coordinate system and the pixel coordinate system, and the coordinates of each selected point in the pixel coordinate system; determining the point closest to the vehicle as the first reference point based on the coordinates of each point in the world coordinate system; defining the target area in the pixel coordinate system with the first reference point as the center; and determining the point closest to the vehicle in the target area as the target collision point according to a preset rule.

[0091] Combination Figure 3 The target line segments include line segments JK and KL. JK and KL are each evenly divided into multiple sub-segments, specifically 5 sub-segments each. One point is selected from each sub-segment, either arbitrarily or at its midpoint. This results in 10 selected points. The coordinates of these 10 points in the world coordinate system are then calculated to obtain the desired coordinates. The point with the smallest absolute value of the axis value is taken as the first reference point. At this point, the first reference point and the points in its vicinity can be considered as the points closest to the vehicle.

[0092] Optionally, a target region is determined in a pixel coordinate system with the first reference point as the center; a point closest to the vehicle is determined in the target region according to a preset rule as the target collision point. The target region can be circular, rectangular, or other shapes, and this embodiment of the invention does not limit this.

[0093] After obtaining the target region, since it is relatively small compared to the entire foreground region, the world coordinates of each pixel in the target region can be calculated sequentially to obtain the point closest to the vehicle as the target collision point. Alternatively, the target region can be further divided into multiple continuous but non-overlapping sub-regions. A point is selected from each sub-region, and the coordinates of the selected point in the world coordinate system are calculated to obtain the point closest to the vehicle as the target collision point.

[0094] The method provided by this invention allows for the acquisition of a target collision point in each frame of the image during the tracking of the same target. This target collision point can be applied to vehicle warning and braking strategies. For example, when the distance between the vehicle and the target collision point is less than a preset value, the vehicle automatically decelerates or performs warning deceleration.

[0095] This invention performs target detection and semantic segmentation on images captured by an in-vehicle camera. Based on semantic segmentation, the background within the target bounding box is removed, and the point closest to the vehicle within the target bounding box is determined as the target collision point based on the foreground. This reduces the influence of the background on the target collision point selection process and improves the accuracy of target collision point selection. Based on the intersection of the foreground within the target bounding box and the edge line, the intersection points of every two adjacent edge lines are connected to obtain the target line segment. The point closest to the vehicle is determined as the target collision point through the target line segment, which reduces the computational load of obtaining the target collision point.

[0096] Figure 4 A flowchart illustrating another method for selecting a vehicle target collision point provided by an embodiment of the present invention is shown, and is described in detail below:

[0097] In step 401, target detection and semantic segmentation are performed on the image, and the portion of the detected target box that is segmented into the target portion is used as the foreground.

[0098] In step 402, the intersection point of the foreground within the target box and the edge line of the target box is obtained. The intersection points of every two adjacent edge lines are connected to obtain the target line segment, which includes at least one line segment.

[0099] In step 403, based on the target line segment, the point closest to the vehicle is determined as the target collision point of the vehicle.

[0100] The specific implementation of steps 401 to 403 can be found in steps 101 to 103, and will not be repeated here in this embodiment of the invention.

[0101] In step 404, the target collision point is used as the target tracking point to track the target.

[0102] In this embodiment of the invention, when the target is a large object, such as a vehicle, the position of the target vehicle's mass point cannot be determined because the vehicle's posture is not fixed and the observation surface of the vehicle's camera changes. After obtaining the target collision point through steps 401 to 404, the target collision point can be used as the target's mass point for tracking, that is, as the target's coordinate point in the world coordinate system for tracking. This tracking point is used as a dimension for target matching and target tracking.

[0103] In step 405, if the tracking points of the current frame image change abruptly, the tracking points of the current frame image are updated based on the tracking points of the previous frame image and the tracking points of the current frame image.

[0104] A sudden change in tracking point refers to the distance between the tracking point in the current frame and the tracking point in the previous frame in the world coordinate system exceeding a preset threshold.

[0105] It should be noted that when the target is a vehicle and is parallel to the vehicle, the distance between each point on its side and the vehicle is basically equal. The target collision points calculated in steps 101 to 104 or steps 401 to 404 between adjacent frames are prone to sudden changes. When the target collision points are applied to fields such as automatic braking, braking warning, and collision warning, sudden changes in the target collision points will not interfere with the above fields. Since the target collision points actually calculated in each frame represent the real target collision points of the current frame, when the target collision points are applied to the above fields, the target collision points actually calculated in each frame, that is, the target collision points before the update, are used as the basis for warning, braking, etc.

[0106] However, since the target tracking position cannot be artificially influenced during target tracking, the target collision point, i.e., the tracking point, cannot undergo abrupt changes during target tracking. To achieve tracking continuity and avoid the impact of abrupt changes in the target collision point on target tracking, one optional implementation method is... Figure 5 This invention provides a flowchart of a tracking point update process, combined with... Figure 5 ,include:

[0107] In step 4051, the relative position of the tracking point in the target box is determined based on the coordinates of the tracking point in the pixel coordinate system and the position of the target box in the pixel coordinate system.

[0108] Figure 6 This is a schematic diagram illustrating the relative position of a tracking point within a target bounding box, provided by an embodiment of the present invention. Figure 5 , ouv Using pixel coordinates, and through the methods provided in steps 401 to 404, a target collision point, i.e., a tracking point, can be determined for each frame of the image. For example, combining... Figure 5 The tracking point acquired in the previous frame is A1, and the tracking point acquired in the current frame is B. Taking the tracking point A1 from the previous frame as an example, when the target box is rectangular, the relative position of A1 within the target box can be determined as follows: the four vertices of the target box in the previous frame are s1s2s3s4. The vertical distance L1 from A1 to the left edge s1s3 and the vertical distance L2 from the top edge s1s2 of the target box are obtained. The length of the left edge s1s3 is taken as the height of the target box, and the length of the top edge s1s3 is taken as the width of the target box. The ratio of the vertical distance of A1 from the left edge of the target box to the width of the target box is marked as the first relative position parameter. The ratio of the vertical distance of A1 from the top edge of the target box to the height of the target box is marked as the second relative position parameter. The first and second relative position parameters together constitute the relative position of the tracking point A1 within the target box in the previous frame.

[0109] Other methods can also be used to record the relative position of the tracking point in the target box, but this embodiment of the invention does not limit this.

[0110] In step 4052, the pixel coordinates of the tracking points of the current frame image are obtained.

[0111] In step 4053, the pixel coordinates of the tracking point in the previous frame image in the current frame image are determined based on the relative position of the tracking point in the target box in the previous frame image and the target box in the current frame image.

[0112] In step 4054, based on the mapping relationship between the pixel coordinate system and the world coordinate system, the first world coordinate value corresponding to the pixel coordinate value of the tracking point in the current frame image and the second world coordinate value corresponding to the pixel coordinate value of the tracking point in the previous frame image in the current frame image are determined.

[0113] Among them, the pixel coordinate system, the world coordinate system, and the mapping relationship between the pixel coordinate system and the world coordinate system are predetermined.

[0114] In step 4055, if the distance value corresponding to the first world coordinate value and the second world coordinate value exceeds a preset threshold, then the tracking points of the previous frame image and the tracking points of the current frame image are subjected to smoothing filtering to obtain the updated tracking points of the current frame image.

[0115] If the distance value exceeds the preset threshold, it indicates that the position of the target collision point has changed abruptly. Since the embodiment of the present invention tracks the target based on the target collision point as the tracking point, in order to avoid the interference caused by the sudden change in the position of the target collision point to the target tracking, it is necessary to perform balanced filtering on the tracking points in the two frames of images.

[0116] Combination Figure 6The process of updating tracking points is explained. A1 is the tracking point obtained from the previous frame image, and B is the tracking point obtained from the current frame image. Target box 1 represents the target box of the previous frame image, and target box 2 represents the target box of the current frame image. Based on the relative position of A1 in target box 1 and the corresponding point A2 of A1 in target box 2, the coordinates of A2 in the world coordinate system are calculated as the second world coordinates, and the coordinates of B in the world coordinate system are calculated as the first world coordinates. The distance between the first world coordinates and the second world coordinates is calculated. If the distance exceeds a preset threshold, it means that A2 and B are far apart in the real world, indicating that the position of the collision point has changed abruptly. Based on the principle that the position of the collision point should not change abruptly, A2 and B are processed by smoothing filtering. Optionally, an impact response filter can be set to reduce the impact of abrupt collision position changes on target tracking. Alternatively, the midpoint C of the line connecting the two coordinate points of A2 and B in the world coordinate system can be obtained, and the coordinate value of C in the pixel coordinate system can be calculated as the collision point of the current frame image. If target tracking is lost in the next frame image, that is, no target box containing the target is obtained, the target box is reconstructed based on point C.

[0117] If the distance between the first world coordinate value and the second world coordinate value is less than a preset threshold, it means that A2 and B are relatively close in the real world. Therefore, the above tracking point update process is not required, and point B is marked as the tracking point of the current frame image.

[0118] Furthermore, during target tracking, there may be instances where the tracked target is lost. That is, the target bounding box in the previous frame may fail to appear in the next frame due to algorithmic or environmental factors, such as excessive relative speed between the vehicle and the target vehicle, or occlusion. To improve the robustness of target tracking, in this embodiment of the invention, based on the principle that tracking points do not change abruptly between adjacent frames, the tracking points acquired in the previous frame are used as reference points for reconstructing the target bounding box in the current frame.

[0119] Optionally, if the tracking result of tracking the target in the current frame image does not include a bounding box containing the target, then the tracking point obtained from the previous frame image is used as the second reference point. Based on the coordinates of the second reference point in the world coordinate system and the mapping relationship between the pixel coordinate system and the world coordinate system, the coordinates of the second reference point in the pixel coordinate system are determined. Based on the coordinates of the second reference point in the pixel coordinate system and the relative position of the target bounding box corresponding to the second reference point in the previous frame image, the bounding box corresponding to the target in the current frame image is determined, and the second reference point is used as the tracking point of the vehicle in the current frame image. In this embodiment of the invention, based on the coordinates of the second reference point (i.e., the tracking point in the previous frame image) in the world coordinate system of the previous frame image, the coordinates of the second reference point in the world coordinate system of the current frame image can be obtained based on an association algorithm. Based on the mapping relationship between the pixel coordinate system and the world coordinate system, the coordinates of the second reference point in the pixel coordinate system of the current frame image can be obtained.

[0120] Given the pixel coordinates of the second reference point and its relative position to the bounding box in the previous frame, the bounding box corresponding to the target in the current frame can be reconstructed. Figure 6 As shown, the target bounding box of the previous frame image is determined, the target collision point is A1, the relative position of A1 and the target bounding box is determined, and the coordinate point of A1 in the current frame image is determined as A2. Based on the relative position of A1 and the target bounding box, a target bounding box can be reconstructed based on the determined A2. The method provided by this embodiment of the invention applies the target collision point to target tracking, improving the stability and robustness of target tracking.

[0121] This invention performs target detection and semantic segmentation on images captured by an in-vehicle camera. Based on semantic segmentation, it removes the background within the target bounding box and determines the closest point to the vehicle within the target bounding box as the target collision point based on the foreground. This reduces the influence of the background on the target collision point selection process and improves the accuracy of target collision point selection. Furthermore, embodiments of this invention apply the target collision point to target tracking, improving the stability and robustness of target tracking.

[0122] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0123] The following are device embodiments of the present invention. For details not described in detail, please refer to the corresponding method embodiments described above.

[0124] Figure 7 A schematic diagram of the vehicle target collision point selection device provided in an embodiment of the present invention is shown. For ease of explanation, only the parts related to the embodiment of the present invention are shown, and are described in detail below:

[0125] like Figure 7 As shown, the vehicle target collision point selection device 7 includes:

[0126] First acquisition module 71, second acquisition module 72, determination module 73;

[0127] The first acquisition module 71 is used to perform object detection and semantic segmentation on the image, and to segment the part of the detected target box into the target as the foreground;

[0128] The second acquisition module 72 is used to acquire the intersection point of the foreground within the target box and the edge line of the target box. The intersection points of every two adjacent edge lines are connected to obtain the target line segment. The target line segment includes at least one line segment.

[0129] The determination module 73 is used to determine the point closest to the vehicle based on the target line segment, which is then used as the target collision point of the vehicle.

[0130] This invention performs target detection and semantic segmentation on images captured by vehicle-mounted cameras. Based on semantic segmentation, the background within the target bounding box is removed, and the point closest to the vehicle within the target bounding box is determined as the target collision point based on the foreground. This reduces the influence of the background on the target collision point selection process and improves the accuracy of target collision point selection. Based on the intersection of the foreground within the target bounding box and the edge line, the intersection of every two adjacent edge lines is connected to obtain the target line segment. The point closest to the vehicle is determined as the target collision point through the target line segment, which reduces the computational load of obtaining the target collision point.

[0131] In one possible implementation, module 73 is defined for:

[0132] For any segment in the target line segment, divide the line segment into multiple sub-segments;

[0133] Select a point on each sub-segment;

[0134] Based on the mapping relationship between the world coordinate system and the pixel coordinate system, and the coordinate values ​​of each selected point in the pixel coordinate system, determine the coordinate values ​​of each point in the world coordinate system.

[0135] Based on the coordinates of each point in the world coordinate system, the point closest to the vehicle is determined as the first reference point;

[0136] Determine the target area in the pixel coordinate system with the first reference point as the center;

[0137] According to preset rules, a point in the target area that is closest to the vehicle is determined as the target collision point.

[0138] In one possible implementation, module 73 is defined for:

[0139] The target collision point is used as the target tracking point to track the target;

[0140] If a sudden change occurs in the tracking point of the current frame image, the tracking point of the current frame image is updated based on the tracking point of the previous frame image and the tracking point of the current frame image. A sudden change in the tracking point means that the distance between the tracking point of the current frame image and the tracking point of the previous frame image in the world coordinate system exceeds a preset threshold.

[0141] In one possible implementation, the determining module 73 is also used for:

[0142] The relative position of the tracking point within the target box is determined based on the coordinates of the tracking point in the pixel coordinate system and the position of the target box in the pixel coordinate system.

[0143] Obtain the pixel coordinates of the tracking point in the current frame image;

[0144] Based on the relative position of the tracking point in the target box of the previous frame image and the target box of the current frame image, determine the pixel coordinate value of the tracking point in the previous frame image in the current frame image;

[0145] Based on the mapping relationship between the pixel coordinate system and the world coordinate system, determine the first world coordinate value corresponding to the pixel coordinate value of the tracking point in the current frame image, and the second world coordinate value corresponding to the pixel coordinate value of the tracking point in the previous frame image in the current frame image.

[0146] If the distance between the first world coordinate value and the second world coordinate value exceeds a preset threshold, then the tracking points of the previous frame image and the tracking points of the current frame image are subjected to smoothing filtering to obtain the updated tracking points of the current frame image.

[0147] In one alternative implementation, the determining module 73 is further configured to:

[0148] If the tracking result of tracking the target in the current frame does not include the target bounding box containing the target, then the tracking point obtained from the previous frame is used as the second reference point.

[0149] Based on the coordinates of the second reference point in the world coordinate system and the mapping relationship between the pixel coordinate system and the world coordinate system, determine the coordinates of the second reference point in the pixel coordinate system.

[0150] Based on the coordinates of the second reference point in the pixel coordinate system and the relative position of the target box corresponding to the second reference point in the previous frame image, the target box corresponding to the target in the current frame image is determined, and the second reference point is used as the tracking point of the vehicle in the current frame image.

[0151] The vehicle target collision point selection device provided in this embodiment can be used to execute the above-described vehicle target collision point selection method embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0152] Figure 8 This is a schematic diagram of a terminal provided in an embodiment of the present invention. For example... Figure 8 As shown, the terminal 8 in this embodiment includes: a processor 80, a memory 81, and a computer program 82 stored in the memory 81 and executable on the processor 80. When the processor 80 executes the computer program 82, it implements the steps in the various vehicle target collision point selection method embodiments described above, for example... Figure 1 Steps 101 to 103 are shown. Alternatively, when the processor 80 executes the computer program 82, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 7 The functions of modules 71 to 73 are shown.

[0153] For example, the computer program 82 can be divided into one or more modules / units, which are stored in the memory 81 and executed by the processor 80 to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 82 in the terminal 8.

[0154] The terminal 8 can be a vehicle-mounted terminal, or a computing device such as a desktop computer, laptop, handheld computer, or cloud server. The terminal 8 may include, but is not limited to, a processor 80 and a memory 81. Those skilled in the art will understand that... Figure 8 This is merely an example of terminal 8 and does not constitute a limitation on terminal 8. It may include more or fewer components than shown, or combine certain components, or different components. For example, the terminal may also include input / output devices, network access devices, buses, etc.

[0155] The processor 80 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0156] The memory 81 can be an internal storage unit of the terminal 8, such as a hard disk or memory of the terminal 8. The memory 81 can also be an external storage device of the terminal 8, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the terminal 8. Furthermore, the memory 81 can include both internal storage units and external storage devices of the terminal 8. The memory 81 is used to store the computer program and other programs and data required by the terminal. The memory 81 can also be used to temporarily store data that has been output or will be output.

[0157] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0158] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0159] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0160] In the embodiments provided by this invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0161] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0162] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0163] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above embodiments of the vehicle target collision point selection method. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately added to or subtracted from the content as required by the legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium may not include electrical carrier signals and telecommunication signals.

[0164] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for selecting a vehicle target collision point, characterized in that, include: The image is subjected to object detection and semantic segmentation, and the portion of the detected object box is segmented into the object and used as the foreground. Obtain the intersection point of the foreground within the target box and the edge line of the target box. Connect the intersection points of every two adjacent edge lines to obtain the target line segment. The target line segment includes at least one line segment. Based on the target line segment, determine the point closest to the vehicle as the target collision point of the vehicle; The method also includes: The target collision point is used as the target tracking point to track the target; If the tracking point of the current frame image changes abruptly, the tracking point of the current frame image is updated based on the tracking point of the previous frame image and the tracking point of the current frame image. The change in the tracking point means that the distance between the tracking point of the current frame image and the tracking point of the previous frame image in the world coordinate system exceeds a preset threshold. If a sudden change occurs in the tracking points of the current frame image, updating the tracking points of the current frame image based on the tracking points of the previous frame image and the current frame image includes: Based on the coordinates of the tracking point in the pixel coordinate system and the position of the target box in the pixel coordinate system, the relative position of the tracking point in the target box is determined. Obtain the pixel coordinates of the tracking point in the current frame image; Based on the relative position of the tracking point in the target box of the previous frame image and the target box of the current frame image, determine the pixel coordinate value of the tracking point in the previous frame image in the current frame image; Based on the mapping relationship between the pixel coordinate system and the world coordinate system, determine the first world coordinate value corresponding to the pixel coordinate value of the tracking point in the current frame image, and the second world coordinate value corresponding to the pixel coordinate value of the tracking point in the previous frame image in the current frame image. If the distance between the first world coordinate value and the second world coordinate value exceeds a preset threshold, then the tracking points of the previous frame image and the tracking points of the current frame image are subjected to smoothing filtering to obtain the updated tracking points of the current frame image.

2. The method according to claim 1, characterized in that, The step of determining the point closest to the vehicle as the target collision point based on the target line segment includes: For any one of the target line segments, divide the line segment into multiple sub-segments; Select a point on each sub-segment; Based on the mapping relationship between the world coordinate system and the pixel coordinate system, and the coordinate values ​​of each selected point in the pixel coordinate system, determine the coordinate values ​​of each point in the world coordinate system. Based on the coordinates of each point in the world coordinate system, the point closest to the vehicle is determined as the first reference point; The target region is determined in the pixel coordinate system with the first reference point as the center; According to preset rules, a point in the target area that is closest to the vehicle is determined as the target collision point.

3. The method according to claim 1, characterized in that, The method also includes: If the tracking result of tracking the target in the current frame image does not include a target bounding box containing the target, then the tracking point obtained from the previous frame image is used as the second reference point. Based on the coordinates of the second reference point in the world coordinate system and the mapping relationship between the pixel coordinate system and the world coordinate system, the coordinates of the second reference point in the pixel coordinate system are determined. Based on the coordinates of the second reference point in the pixel coordinate system and the relative position of the target box corresponding to the second reference point in the previous frame image, the target box corresponding to the target in the current frame image is determined, and the second reference point is used as the tracking point of the vehicle in the current frame image.

4. A device for selecting a vehicle target collision point, characterized in that, include: The module consists of a first acquisition module, a second acquisition module, and a determination module. The first acquisition module is used to perform object detection and semantic segmentation on the image, and to segment the detected target box into the part of the target as the foreground; The second acquisition module is used to acquire the intersection points of the foreground within the target box and the edge line of the target box. The intersection points of every two adjacent edge lines are connected to obtain a target line segment, and the target line segment includes at least one line segment. The determining module is used to determine the point closest to the vehicle based on the target line segment, as the target collision point of the vehicle; The determining module is also used for: The target collision point is used as the target tracking point to track the target; If the tracking point of the current frame image changes abruptly, the tracking point of the current frame image is updated based on the tracking point of the previous frame image and the tracking point of the current frame image. The change in the tracking point means that the distance between the tracking point of the current frame image and the tracking point of the previous frame image in the world coordinate system exceeds a preset threshold. The determining module is also used for: Based on the coordinates of the tracking point in the pixel coordinate system and the position of the target box in the pixel coordinate system, the relative position of the tracking point in the target box is determined. Obtain the pixel coordinates of the tracking point in the current frame image; Based on the relative position of the tracking point in the target box of the previous frame image and the target box of the current frame image, determine the pixel coordinate value of the tracking point in the previous frame image in the current frame image; Based on the mapping relationship between the pixel coordinate system and the world coordinate system, determine the first world coordinate value corresponding to the pixel coordinate value of the tracking point in the current frame image, and the second world coordinate value corresponding to the pixel coordinate value of the tracking point in the previous frame image in the current frame image. If the distance between the first world coordinate value and the second world coordinate value exceeds a preset threshold, then the tracking points of the previous frame image and the tracking points of the current frame image are subjected to smoothing filtering to obtain the updated tracking points of the current frame image.

5. The apparatus according to claim 4, characterized in that, The determining module is used for: For any one of the target line segments, divide the line segment into multiple sub-segments; Select a point on each sub-segment; Based on the mapping relationship between the world coordinate system and the pixel coordinate system, and the coordinate values ​​of each selected point in the pixel coordinate system, determine the coordinate values ​​of each point in the world coordinate system. Based on the coordinates of each point in the world coordinate system, the point closest to the vehicle is determined as the first reference point; The target region is determined in the pixel coordinate system with the first reference point as the center; According to preset rules, a point in the target area that is closest to the vehicle is determined as the target collision point.

6. A terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 3 above.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 3 above.

Images