Method and apparatus for detecting stationary object, and device, storage medium and vehicle

Abstract

The present application relates to the technical field of object detection. Disclosed are a method and apparatus for detecting a stationary object, and a device, a storage medium and a vehicle. The method comprises: acquiring the current image frame collected by a camera, and identifying a camera observation object from the current image frame; associating the camera observation object with a current object detection box, and performing state prediction and update on the associated object detection box on the basis of a relative motion parameter of the camera observation object, wherein the current object detection box comprises a detection box created on the basis of historical observed objects; determining whether the state of the updated object detection box has converged; and when the state of the updated object detection box has converged, determining whether the updated object detection box is stationary. In the present application, an observation object is associated with an object detection box and a state is updated, such that whether the object detection box is stationary can be accurately determined by using the object detection box with a converged state.

Description

Methods, devices, equipment, storage media, and vehicles for detecting stationary targets.

[0001] This application claims priority to Chinese Patent Application No. 202410972892.5, filed on July 18, 2024, entitled "Method, Apparatus, Device, Storage Medium and Vehicle for Detecting Stationary Targets", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of target detection technology, specifically to methods, apparatus, equipment, storage media, and vehicles for detecting stationary targets. Background Technology

[0003] Intelligent driving vehicles are equipped with various sensors such as cameras and radar. Based on the data collected by these sensors, they can detect objects around the vehicle and avoid them. Among these, stationary objects have a significant impact on vehicle movement, so the vehicle needs to be able to detect stationary objects around it accurately.

[0004] When detecting stationary targets, especially those at long distances (e.g., over 100m), camera velocity and range measurement accuracy is low, while radar is prone to false detections of stationary targets. Some solutions fuse camera image features and radar features, but data fusion requires a large amount of computation and has low processing efficiency.

[0005] Summary of the Invention

[0006] In view of this, this application provides a method, apparatus, device, storage medium, and vehicle for detecting stationary targets, in order to solve the problem of inaccurate detection of stationary targets.

[0007] Firstly, this application provides a method for detecting stationary targets, comprising:

[0008] Acquire the current frame image captured by the camera and identify the camera-observed target in the current frame image;

[0009] The camera-observed target is associated with the current target detection box, and the state of the associated target detection box is predicted and updated based on the relative motion parameters of the camera-observed target; the current target detection box includes detection boxes created based on observed historical targets;

[0010] Determine whether the updated target detection box has converged.

[0011] If the updated target detection box state converges, determine whether the updated target detection box is stationary.

[0012] In some optional implementations, associating the camera-observed target with the current target detection box includes:

[0013] Associate the first camera observation target with the current target detection box; the first camera observation target is the camera observation target corresponding to the most critical moving object;

[0014] Next, the second camera observation target is associated with a target detection box that is not currently associated with the first camera observation target; the second camera observation target is a camera observation target other than the first camera observation target.

[0015] Create new target detection boxes for second-camera observation targets that could not be associated with target detection boxes.

[0016] In some optional implementations, associating the camera-observed target with the current target detection bounding box further includes:

[0017] If the first camera observation target cannot be associated with a target detection box, after creating a new target detection box for the second camera observation target that cannot be associated with a target detection box, the first camera observation target is reassociated with the target detection box that is not currently associated with the camera observation target;

[0018] If the target observed by the first camera fails to be associated with a target detection box again, a new target detection box is created for the target observed by the first camera.

[0019] In some optional implementations, determining whether the updated target detection box has converged includes:

[0020] Determine the first association count of the updated target detection box; the first association count is the number of times the updated target detection box is associated with the camera observation target corresponding to the most critical moving object;

[0021] If the first association count is greater than the first threshold, the updated target detection box state is determined to have converged.

[0022] And / or,

[0023] Determine the second association count of the updated target detection box; the second association count is the number of times the updated target detection box is associated with the camera-observed target;

[0024] If the second association count is greater than the second threshold, determine the covariance matrix between the relative velocity and relative depth corresponding to the updated target detection box;

[0025] If the covariance matrix satisfies the convergence condition, the updated target detection box state is determined to be converged.

[0026] In some alternative implementations, the covariance matrix is:

[0027] Where depth represents the relative depth observation, speed represents the relative velocity observation, and P(depth,speed) represents the covariance matrix between relative velocity and relative depth. dd P represents the variance of the relative depth observations. ss P represents the variance of the relative velocity observations. ds P represents the covariance between the relative depth observation and the relative velocity observation. sd This represents the covariance between the relative velocity observation and the relative depth observation;

[0028] The convergence conditions include:

[0029] Where depth' is the relative depth of the target detection box, and k1 and k2 are preset threshold parameters.

[0030] In some optional implementations, determining whether the updated target detection box is stationary when the updated target detection box state converges includes:

[0031] If the relative velocity of the updated target detection box is less than the current velocity threshold when the updated target detection box state converges, then the updated target detection box is determined to be stationary.

[0032] or,

[0033] If the updated target detection box state converges, the cumulative probability of the normal distribution corresponding to the self-velocity is determined by using the negative value of its own velocity as a variable and the relative velocity of the updated target detection box as the expectation.

[0034] If the cumulative probability of the normal distribution is greater than a preset probability value, the updated target detection box is determined to be stationary.

[0035] In some optional implementations, the cumulative probability of the normal distribution is:

[0036] Where x represents the negative value of its own velocity, Δv represents the preset velocity difference, μ represents the expected relative velocity, and σ represents the standard deviation of the relative velocity.

[0037] In some alternative implementations, the method further includes:

[0038] Identify the radar targets currently being observed by the radar and filter out the stationary radar targets;

[0039] The stationary radar-observed target is associated with the current target detection box, and the state of the associated target detection box is predicted and updated based on the relative motion parameters of the stationary radar-observed target.

[0040] In some optional implementations, determining whether the updated target detection box has converged includes:

[0041] The third association count of the updated target detection box is determined; the third association count is the number of times the updated target detection box is associated with a stationary radar observation target; if the third association count is greater than a third threshold, the state of the updated target detection box is determined to be converged.

[0042] The step of determining whether the updated target detection box is stationary when the updated target detection box state converges includes:

[0043] If the third association count of the updated target detection box with a stationary radar-observed target is greater than a third threshold, the updated target detection box is determined to be stationary.

[0044] In some alternative implementations, the method further includes:

[0045] Delete the target detection boxes that meet the failure criteria;

[0046] The failure conditions include: the target detection box is not associated for a period of time exceeding a first preset time; or, the target type of the target detection box is inconsistent with the target type of the associated observed target, and the inconsistency lasts for a period of time exceeding a second preset time.

[0047] Secondly, this application provides a detection device for stationary targets, comprising:

[0048] The target recognition module is used to acquire the current frame image captured by the camera and identify the camera-observed target in the current frame image;

[0049] The association update module is used to associate the camera-observed target with the current target detection box, and to predict and update the state of the associated target detection box according to the relative motion parameters of the camera-observed target; the current target detection box includes detection boxes created based on observed historical targets;

[0050] The convergence judgment module is used to determine whether the updated target detection box has converged.

[0051] The detection module is used to determine whether the updated target detection box is stationary when the updated target detection box state converges.

[0052] Thirdly, this application provides a computer device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the method for detecting a stationary target as described in the first aspect or any corresponding embodiment.

[0053] Fourthly, this application provides a computer-readable storage medium storing computer instructions for causing a computer to execute the method for detecting a stationary target according to the first aspect or any corresponding embodiment described above.

[0054] Fifthly, this application provides a vehicle including a vehicle controller, the vehicle controller being used to perform the method for detecting stationary targets according to the first aspect or any corresponding embodiment described above.

[0055] This application associates and updates the state of the observed target with the target detection box after multiple rounds of processing. This allows the target detection box state to converge. Even if the observed value of the target has a large observation error, the converged target detection box can accurately represent the true state of the corresponding observed target, reducing the impact of observation errors. This enables more accurate determination of whether the target detection box is stationary, and it also has good detection performance for distant stationary targets, exhibiting high accuracy and recall. When applied to autonomous vehicles, it can accurately identify stationary objects such as obstacles in the lane, improving the safety of autonomous vehicles. Camera-based visual detection and recognition of stationary targets is low-cost, does not require data fusion, has low computational load, and high recognition efficiency. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the specific embodiments or related technologies of this application, the drawings used in the description of the specific embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0057] Figure 1 is a flowchart illustrating a method for detecting a stationary target according to an embodiment of this application;

[0058] Figure 2 is a flowchart illustrating another method for detecting a stationary target according to an embodiment of this application;

[0059] Figure 3 is a schematic diagram of a process of observing a target with an associated camera according to an embodiment of this application;

[0060] Figure 4 is a flowchart illustrating a method for detecting another stationary target according to an embodiment of this application;

[0061] Figure 5 is a structural block diagram of a stationary target detection device according to an embodiment of this application;

[0062] Figure 6 is a schematic diagram of the hardware structure of a computer device according to an embodiment of this application. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0064] During autonomous or assisted driving, vehicles need to use sensors such as cameras and radar to detect objects around the vehicle to alert the driver or automatically avoid them. Because stationary objects have a significant impact on safe driving, the ability to accurately detect stationary objects is particularly important.

[0065] For example, during autonomous driving, if vehicle A is in front of another vehicle B, and vehicle B temporarily stops in the middle of the lane due to a malfunction, traffic accident, or other reason, vehicle A's cameras and radar need to detect vehicle B and determine that it is currently stationary (i.e., a stationary target). This allows vehicle A to automatically avoid vehicle B and prevent a collision. However, if vehicle A mistakenly believes that vehicle B is not stationary—for example, mistakenly believing that vehicle B is also moving forward—vehicle A may fail to avoid vehicle B in time due to this misjudgment, thus posing a collision risk.

[0066] Furthermore, sensors such as cameras and radar generally have significant errors when detecting the state of distant objects, making it difficult to accurately determine information such as the relative speed and position of distant objects. If stationary objects in the distance could be accurately identified, vehicle A could avoid them in advance, even when traveling at high speeds on highways or other road conditions, ensuring the safety of the vehicle.

[0067] According to an embodiment of this application, a method for detecting stationary targets is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0068] This embodiment provides a method for detecting stationary targets, which can be applied to local terminals such as vehicle controllers, servers located on the network side, or distributed systems located on the network side, depending on the actual situation. Figure 1 is a flowchart of the method for detecting stationary targets according to an embodiment of this application. As shown in Figure 1, the process includes the following steps.

[0069] Step S101: Acquire the current frame image captured by the camera and identify the camera observation target in the current frame image.

[0070] In this embodiment, when detecting stationary targets, a camera is used to acquire images in real time. The image captured by the camera at the current moment is called the current frame image. The current frame image includes targets around the camera, which are observed by the camera and are referred to as camera-observed targets.

[0071] For example, the camera is a vehicle-mounted camera that can acquire images of the environment around the vehicle in real time, generate a corresponding current frame image, and identify targets such as vehicles and pedestrians in the current frame image by performing target detection. These targets are all observed by the camera. Some of these targets may be stationary targets, such as vehicles or pedestrians stopped in the lane, or roadblocks or fallen rocks in the lane.

[0072] Step S102: Associate the camera-observed target with the current target detection box, and predict and update the state of the associated target detection box based on the relative motion parameters of the camera-observed target; the current target detection box includes detection boxes created based on observed historical targets.

[0073] In this embodiment, when performing target detection, corresponding detection boxes are created for the observed targets. These detection boxes can serve as target detection boxes. Specifically, corresponding target detection boxes can be created for the initially observed targets. For example, based on the first frame image captured by the camera, the observed targets can be identified, and an initial target detection box can be created for each observed target. Subsequently, these initial target detection boxes are used as a basis to associate with the camera-observed targets.

[0074] Specifically, after determining the current camera observation targets, these targets can be associated with the current target detection boxes to determine which target detection box is associated with each camera observation target. For example, the IOU (Intersection over Union) or size ratio between the camera observation targets and target detection boxes can be calculated to determine which target detection box the camera observation target can be associated with. Here, the current target detection box is the target detection box that exists at the current moment, and this current target detection box specifically includes the detection boxes corresponding to previously observed historical targets.

[0075] Furthermore, each camera-observed target possesses certain relative motion parameters. If a camera-observed target 1 is associated with a target detection box A, the state of target detection box A can be predicted and updated based on the relative motion parameters of camera-observed target 1, thereby updating the state of target detection box A. For example, these relative motion parameters may specifically include relative velocity (i.e., the relative velocity between itself and the camera-observed target) and relative depth (i.e., the relative depth between itself and the camera-observed target, also known as relative distance). Kalman filtering can be used to predict and update the state of target detection box A, thereby updating the relative position and relative velocity of target detection box A. The Kalman filtering formula is as follows:

[0076] The Kalman filter time update equation is:

[0077] The Kalman filter state update equation is:

[0078] Each parameter is explained below:

[0079] and Let represent the posterior state estimates at time k-1 and time k, respectively, i.e., the updated results.

[0080] Let k represent the prior state estimate at time k, which is the result of the prediction equation.

[0081] P k-1 and P k These represent the posterior estimated covariances at time k-1 and time k, respectively, and are one of the results of filtering.

[0082] This represents the prior estimate of the covariance at time k, which is an intermediate calculation result of the filtering.

[0083] H is the transformation matrix from state variables to measurements (observations).

[0084] zk This represents the observed input value.

[0085] K k This represents the Kalman filter gain matrix.

[0086] A represents the state transition matrix.

[0087] A T and H T These represent the transposes of matrices A and H, respectively.

[0088] Q represents the process noise covariance matrix.

[0089] R represents the measurement noise covariance.

[0090] I represents the identity matrix.

[0091] For example, after creating initial target detection boxes based on the first frame image, since the initial target detection boxes correspond one-to-one with the observed targets, the state of the initial target detection boxes can be predicted and updated based on the relative motion parameters of these observed targets. After identifying the camera-observed targets in the second frame image, these updated target detection boxes can be used as the current target detection boxes and associated with the camera-observed targets in the second frame image, i.e., the second frame image is the current frame image, and step S102 is executed. Similarly, after identifying the camera-observed targets in the third frame image, the existing target detection boxes can also be determined, and these target detection boxes can be used as the current target detection boxes and associated with the camera-observed targets in the third frame image, i.e., the third frame image is the current frame image, and step S102 is executed. This process is repeated to achieve real-time association and updating of the target detection boxes.

[0092] Step S103: Determine whether the updated target detection box has converged.

[0093] In this embodiment, whenever the current frame image is acquired, the target detection box can be updated based on the currently identified camera observation target. After multiple rounds of updates, the state of the target detection box can be stabilized, that is, the state converges.

[0094] Taking Kalman filtering state prediction of target detection boxes as an example, the more times the target detection boxes are updated, the more stable the state of the target detection boxes becomes. Therefore, we can judge whether the state has converged based on the number of times the target detection boxes are updated.

[0095] Step S104: If the updated target detection box state converges, determine whether the updated target detection box is stationary.

[0096] If the target detection box state converges, it means that the state (position, velocity, acceleration, etc.) of the target detection box has become stable. At this point, even if the camera's observation of the target has a large error—for example, if the camera's velocity observation of the target has a large error—updating the target detection box with this erroneous velocity observation can still ensure that the velocity of the target detection box is relatively close to the true velocity observed by the camera. In other words, a converged target detection box has a stable state and is less affected by errors in the observations.

[0097] Therefore, target detection boxes based on state convergence can accurately represent the state of the corresponding observed target, reducing the impact of observation errors. This is particularly beneficial for long-distance target detection, mitigating measurement inaccuracies caused by greater distance and larger errors, and enabling accurate determination of whether the target detection box is stationary. If the target detection box is stationary, the associated observed target is also stationary. In this case, a stationary target detection box can be sent to downstream modules for planning and control. For example, if the updated target detection box is stationary, the vehicle's driving path can be controlled based on the stationary target detection box, such as controlling the vehicle's speed, acceleration, and orientation.

[0098] The relative velocity of the updated target detection box can be used to determine whether the updated target detection box is stationary. For example, if the relative velocity of the updated target detection box is negative compared to the vehicle's velocity, satisfying the stationary condition, then the updated target detection box can be determined to be stationary.

[0099] The downstream module receives a converged and static target detection box and predicts the target vehicle's behavior and trajectory. Based on this behavior and trajectory information, the module performs deceleration and obstacle avoidance control on the vehicle itself to ensure its safety.

[0100] The stationary target detection method provided in this application associates and updates the state of the observed target with the target detection box. After multiple rounds of processing, the state of the target detection box converges. Even if the observed value of the target has a large observation error, the converged target detection box can accurately represent the true state of the corresponding observed target, reducing the impact of observation error. This allows for more accurate determination of whether the target detection box is stationary, and it also has good detection performance for stationary targets at a distance, with high accuracy and recall. When applied to autonomous vehicles, it can accurately identify stationary objects such as obstacles in the lane, improving the safety of autonomous vehicles. Camera-based visual detection and recognition of stationary targets is low-cost, does not require data fusion, has low computational load, and high recognition efficiency.

[0101] This embodiment provides a method for detecting stationary targets, which can be applied to local terminals such as vehicle controllers, servers located on the network side, or distributed systems located on the network side. Figure 2 is a flowchart of the method for detecting stationary targets according to an embodiment of this application. As shown in Figure 2, the process includes the following steps.

[0102] Step S201: Acquire the current frame image captured by the camera and identify the camera observation target in the current frame image.

[0103] For details, please refer to step S101 of the embodiment shown in Figure 1, which will not be repeated here.

[0104] Step S202: Associate the camera-observed target with the current target detection box, and predict and update the state of the associated target detection box based on the relative motion parameters of the camera-observed target; the target detection box includes detection boxes created based on observed historical targets.

[0105] For details, please refer to step S102 of the embodiment shown in Figure 1, which will not be repeated here.

[0106] In some alternative implementations, the above step S202, "associating the camera-observed target with the current target detection box," includes the following steps A1 to A3.

[0107] Step A1: Associate the first camera observation target with the current target detection box; the first camera observation target is the camera observation target corresponding to the most critical moving object.

[0108] Step A2, then, associate the second camera observation target with the target detection box that is not currently associated with the first camera observation target; the second camera observation target is other camera observation targets besides the first camera observation target.

[0109] Step A3: Create a new target detection box for the second camera observation target that could not be associated with a target detection box.

[0110] In autonomous driving scenarios, there exists a most critical dynamic object (MCDO) that significantly impacts vehicle movement. Since the first object in a lane is considered the most dangerous, it is generally regarded as the MCDO. After identifying multiple camera observation targets in the current frame image, it can be determined which camera observation target is the MCDO. For ease of description, the camera observation target corresponding to the MCDO is referred to as the first camera observation target, and all other camera observation targets besides the first are referred to as the second camera observation targets; that is, the second camera observation targets are non-MCDO observation targets.

[0111] When performing bounding box association, the first camera observation target related to MCDO is first associated. This can increase the probability of the first camera observation target being associated and avoid the risk of association failure due to conflict with other camera observation targets.

[0112] After associating the targets observed by the first camera, the remaining targets observed by the second camera can be associating. Specifically, the targets observed by the second camera are associated with target detection boxes that are not currently associated with targets observed by the first camera.

[0113] If a target observed by the first camera is successfully associated with a target detection box, then a target observed by the second camera cannot be associated with that successfully associated target detection box. If the first camera fails to associate a target, then the second camera can attempt to associate it with any target detection box.

[0114] Furthermore, if there are second-camera observed targets that cannot be associated with target detection boxes, new target detection boxes are created for these second-camera observed targets; if all second-camera observed targets are successfully associated, no additional processing is required for the second-camera observed targets.

[0115] In this embodiment, the camera observation targets corresponding to the MCDO are first associated, which allows these targets to be prioritized for association and updating, improving the recall rate of MCDO targets and thus enhancing the safety of autonomous vehicles. Creating new target detection boxes for second camera observation targets that could not be associated with target detection boxes improves the detection rate of stationary targets. Furthermore, creating new target detection boxes based on camera observation targets eliminates the need for radar, avoiding the problem of numerous false target detection boxes being created due to radar ghosting.

[0116] Optionally, the above step S202, "associating the camera-observed target with the current target detection box", may also include the following steps A4 to A5.

[0117] Step A4: If the target observed by the first camera cannot be associated with a target detection box, after creating a new target detection box for the second camera that cannot be associated with a target detection box, the first camera observation target is reassociated with the target detection box that is not currently associated with the camera observation target.

[0118] Step A5: If the target observed by the first camera fails to be associated with a target detection box again, create a new target detection box for the target observed by the first camera.

[0119] In this embodiment, if the target observed by the first camera cannot be associated with a target detection box, a new target detection box can be created for the first camera-observed target, similar to step A3 described above. Specifically, to improve the detection recall rate of the MCDO, the target observed by the first camera is associated again.

[0120] Specifically, if the first camera observation target fails to be associated with a target detection box in step A1 above, then after step A3 ("creating a new target detection box for the second camera observation target that failed to be associated with a target detection box"), the first camera observation target is associated again, that is, the first camera observation target is reassociated with the target detection box that is not currently associated with a camera observation target. If the first camera observation target is successfully associated, the associated target detection box can be state predicted and updated; if the first camera observation target fails to be associated again, that is, it fails to be associated with a target detection box again, then a new target detection box is also created for the first camera observation target.

[0121] Optionally, the method further includes step A6.

[0122] Step A6: Delete the target detection boxes that meet the failure conditions.

[0123] The failure conditions include: the target detection box is not associated for a period of time exceeding a first preset time; or, the target type of the target detection box is inconsistent with the target type of the associated observed target, and the inconsistency lasts for a period of time exceeding a second preset time.

[0124] In this embodiment of the application, if a target detection box is not associated for a period of time exceeding a first preset time (e.g., 300ms, 400ms, etc.), it indicates that the target detection box has failed to be associated for a long time and the target detection box can be deleted.

[0125] Alternatively, if a target detection box is associated with an observed target, but the target type of the detection box is inconsistent with the target type of the associated observed target for an extended period (i.e., the inconsistency lasts longer than a second preset time (e.g., 200ms, 300ms), then the association is considered invalid, and the target detection box can be deleted. For example, if the target type of a target detection box is "vehicle," but the target type of the associated observed target is "pedestrian," and the target types are inconsistent for an extended period, then the target detection box should be deleted.

[0126] Figure 3 illustrates a process of associating camera-observed targets. As shown in Figure 3, if there are currently seven target detection boxes, namely target detection boxes A to G; and six camera-observed targets are identified in the current frame image, namely camera-observed targets 1 to 6.

[0127] First, MCDO association is performed. As shown in Figure 3, if the camera-observed target 1 corresponds to an MCDO, then the camera-observed target 1 is associated with the target detection box. If the IOU between the camera-observed target 1 and the target detection box A is greater than a threshold, then the two can be associated. Subsequently, the state prediction and update of the target detection box A are performed based on the relative motion parameters of the camera-observed target 1.

[0128] Next, non-MCDO association is performed. As shown in Figure 3, camera observation targets 2 to 6 are all second camera observation targets, and these camera observation targets can be associated with the remaining target detection boxes. As shown in Figure 3, camera observation targets 2, 3, 4, and 5 are associated with target detection boxes B, C, D, and E respectively, while the association of camera observation target 6 fails.

[0129] Then, new target detection boxes can be created for the camera-observed targets that failed to be associated. As shown in Figure 3, a new target detection box H can be created based on the position, velocity, acceleration, and other parameters of the camera-observed target 6.

[0130] Furthermore, if a target detection box meets the failure criteria, the failed target detection box is deleted. As shown in Figure 3, if target detection box G has not been associated for a long time, it can be deleted. When the next frame of the image is acquired, the camera-observed targets extracted from it are used to associate with target detection boxes A, B, C, D, E, F, and H.

[0131] It should be noted that in this embodiment, the target detection box can also be associated with the radar detection result. If a target detection box is not associated with the camera's observed target, but is successfully associated with the radar's detection result, then the target detection box is an associated target detection box. The target detection box will only be deleted if it has not been associated with the camera and radar's observed target for a long time.

[0132] In this embodiment, performing a second round of association updates on the first camera observation target corresponding to the MCDO can improve the detection recall rate of the MCDO and effectively prevent the MCDO observation target from being missed; timely deletion of invalid target detection boxes can improve the accuracy of target detection.

[0133] Step S203: Determine the radar observation targets currently identified by the radar and filter out the stationary radar observation targets.

[0134] Because camera detection is greatly affected by lighting conditions, it may miss detections. Therefore, this embodiment also incorporates radar for target detection.

[0135] Specifically, based on radar data collected by the radar, targets observed by the radar can be identified, i.e., radar observation targets. These radar observation targets may include radar point cloud targets, and tracking these radar point cloud targets can generate corresponding radar track targets, which can also serve as radar observation targets.

[0136] For each radar observation target, the probability that it belongs to an obstacle can be determined. For radar observation targets with a probability exceeding a threshold, they can be preliminarily considered to be stationary, and these radar observation targets will be referred to as stationary radar observation targets.

[0137] Specifically, this radar can be a millimeter-wave radar, eliminating the need for expensive lidar and thus reducing the hardware cost of autonomous vehicles. Furthermore, camera detection and radar detection can operate in parallel without interfering with each other, and data fusion is not required, resulting in low computational load and high processing efficiency.

[0138] Step S204: Associate the stationary radar-observed target with the current target detection box, and predict and update the state of the associated target detection box based on the relative motion parameters of the stationary radar-observed target.

[0139] In this embodiment, similar to identifying camera-observed targets, after determining the stationary radar-observed targets, these stationary radar-observed targets are associated with the current target detection boxes. Furthermore, the associated target detection boxes can be state-predicted and updated, which will not be elaborated here.

[0140] Since radar generally does not distinguish between target types, some of the static radar observation targets selected may be useless targets, such as roadside fences and trees. However, the target detection box is created based on the target observed by the camera. As a vision-based target detection box, it is unrelated to useless targets such as fences and trees. By associating the static radar observation target with the current target detection box, useless targets identified by the radar can also be excluded.

[0141] It is understandable that camera target detection and radar target detection are independent. Therefore, for a target detection box, if it is associated with a camera-detected target, a state prediction and update can be performed based on the relative motion parameters of the camera-detected target. If the target detection box is also associated with a stationary radar-observed target, a state prediction and update can be performed again based on the relative motion parameters of the stationary radar-detected target.

[0142] Step S205: Determine whether the updated target detection box has converged.

[0143] For details, please refer to step S103 of the embodiment shown in Figure 1, which will not be repeated here.

[0144] In some optional implementations, for more important MCDO observation targets, i.e., the first camera observation targets, the convergence of the target detection boxes associated with the MCDO observation targets can be determined separately. Specifically, the above step S205, "determining whether the updated target detection boxes have converged," can specifically include steps B1 to B2.

[0145] Step B1: Determine the first association count of the updated target detection box; the first association count is the number of times the updated target detection box is associated with the camera observation target corresponding to the most critical moving object.

[0146] Step B2: If the first association count is greater than the first threshold, determine that the updated target detection box state has converged.

[0147] In this embodiment, as described above, the camera observation targets include a first camera observation target corresponding to the most critical moving object, and a second camera observation target other than the first camera observation target. For any target detection box, if it is associated with the first camera observation target, the number of times the target detection box is associated with the camera observation target corresponding to the most critical moving object can be counted, thereby obtaining the first association count of the target detection box. Specifically, for each frame of image where an MCDO observation target is associated with the target detection box, the first association count of the target detection box is incremented by one.

[0148] To ensure the accuracy of the first association count, the number of times the MCDO observed targets are continuously associated can be counted. That is, the first association count is the number of times the updated target detection box is continuously associated with the camera observation targets corresponding to the most critical driving objects.

[0149] If the first association count is greater than the first threshold, it means that the target detection box has been predicted and updated based on the relative motion parameters of the target observed by MCDO at multiple times. The state of the target detection box at this time can accurately represent the state of the target observed by MCDO. Therefore, it can be determined that the state of the target detection box has converged.

[0150] Alternatively, for any target detection box, the number of times the target was observed by the associated camera can be counted, and convergence can be judged based on this number. Specifically, step S205, "determining whether the updated target detection box has converged," can include steps B3 to B5.

[0151] Step B3: Determine the second association count of the updated target detection box; the second association count is the number of times the updated target detection box is associated with the camera's observed target.

[0152] Step B4: If the second association count is greater than the second threshold, determine the covariance matrix between the relative velocity and relative depth corresponding to the updated target detection box.

[0153] Step B5: If the covariance matrix satisfies the convergence condition, determine that the updated target detection box state has converged.

[0154] In this embodiment, for at least some target detection boxes, the number of times they are associated with camera-observed targets can be counted, and this number can be used as the second association count of the target detection box. Specifically, to avoid false positives, this second association count can be the number of times the updated target detection box is associated with the same camera-observed target; for example, different camera-observed targets can be assigned IDs, and the second association count is the number of times the updated target detection box is associated with camera-observed targets corresponding to the same ID.

[0155] If the second association count is greater than the second threshold, it indicates that the target detection box has likely stabilized. Furthermore, to ensure the accuracy of the convergence judgment, this embodiment also determines the covariance matrix between the relative velocity and relative depth corresponding to the target detection box, and further judges the convergence of the target detection box state based on this covariance matrix.

[0156] The convergence condition of the covariance matrix is ​​preset. If the second association count of the target detection box is greater than a second threshold and the covariance matrix satisfies the convergence condition, the state of the target detection box can be determined to be converged. For example, convergence can be determined based on the rank of the covariance matrix or based on the residual.

[0157] For the target detection boxes associated with the MCDO observation target, convergence judgment can be performed again based on steps B3 to B5 above to improve the reliability of the judgment results.

[0158] Optionally, the covariance matrix of the target detection box is:

[0159] Where depth represents the relative depth observation, speed represents the relative velocity observation, and P(depth,speed) represents the covariance matrix between relative velocity and relative depth. dd P represents the variance of the relative depth observations. ss P represents the variance of the relative velocity observations. ds P represents the covariance between the relative depth observation and the relative velocity observation. sd This represents the covariance between the relative velocity observation and the relative depth observation.

[0160] In this embodiment, for the current frame image captured by the camera, the relative depth and relative velocity of the target observed by the camera can be determined, i.e., the relative depth observation and the relative velocity observation. For example, the current frame image can be input into a velocity network, and the relative depth and relative velocity of the target observed by the camera can be determined based on the output of the velocity network. Based on the relative depth observation and relative velocity observation corresponding to the frame image, the covariance matrix between relative velocity and relative depth can be calculated. Where P... ds P sd The physical meanings of the two are different, but their numerical values ​​are generally the same.

[0161] In this embodiment, the convergence condition is specifically set as follows:

[0162] Where depth' is the relative depth of the target detection box, that is, the relative depth predicted by the target detection box. k1 and k2 are both preset threshold parameters, which are pre-set hyperparameters.

[0163] After determining the covariance matrix of the target detection box, it is easy and quick to determine whether the above formula holds: if P dd -P ds *P sd / P ss <(k1*depth') 2 And P ss -P sd *P ds / P dd <k2 2 If the covariance matrix satisfies the convergence condition, then the target detection box state is converged.

[0164] In this embodiment, by counting the first association count of the target detection boxes, the convergence of the target detection boxes associated with the MCDO observed targets can be quickly determined, facilitating a rapid assessment of whether the corresponding target detection boxes are stationary. By pre-setting the convergence condition of the covariance matrix, and based on the second association count of the target detection boxes and the covariance matrix, convergence can be determined relatively accurately, effectively reducing the possibility of misjudgment.

[0165] Optionally, the convergence of the corresponding target detection box can also be determined based on the number of associations with the target observed by the stationary radar. Specifically, step S205, "determining whether the updated target detection box has converged," may also include step B6.

[0166] Step B6: Determine the third association count of the updated target detection box; the third association count is the number of times the updated target detection box is associated with the stationary radar observation target; if the third association count is greater than the third threshold, determine that the updated target detection box has converged.

[0167] In this embodiment, similar to the process of judging the convergence of the target detection box based on the first association number, after determining the third association number of the target detection box with the stationary radar observation target, it can be determined whether the third association number is greater than the third threshold. If the third association number is greater than the third threshold, it can be determined that the updated target detection box state has converged.

[0168] The first threshold, the second threshold, and the third threshold can be the same or different, depending on the actual situation.

[0169] Step S206: If the updated target detection box state converges, determine whether the updated target detection box is stationary.

[0170] For details, please refer to step S104 of the embodiment shown in Figure 1, which will not be repeated here.

[0171] In some alternative implementations, the above step S206 "determining whether the updated target detection box is stationary when the updated target detection box state converges" includes the following step C1.

[0172] Step C1: If the relative velocity of the updated target detection box is less than the current velocity threshold when the updated target detection box state converges, then the updated target detection box is determined to be stationary.

[0173] In this embodiment, if the observed target is stationary, its relative velocity to itself should be the negative of its own velocity; for example, if the vehicle's own speed is 30 m / s, then its relative velocity to the stationary target should be -30 m / s. When the target detection box converges, the smaller its relative velocity, the closer it is to the negative of its own velocity, and the more likely the target detection box is to be stationary.

[0174] Specifically, a corresponding speed threshold can be set based on its own speed. This speed threshold can be slightly larger than the negative value of its own speed. If the relative speed of the target detection box is less than the current speed threshold, it can be considered that the relative speed of the target detection box is basically the same as the negative value of its own speed, and the target detection box can be easily and quickly determined to be stationary.

[0175] For example, if it is determined from steps B1 to B2 that the target detection box associated with the MCDO observation target has converged, then step C1 can be used to further determine whether the target detection box associated with the MCDO observation target is stationary.

[0176] Alternatively, step S206 above, "determining whether the updated target detection box is stationary when the updated target detection box state converges," may also include steps C2 to C3.

[0177] Step C2: If the updated target detection box state converges, determine the cumulative probability of the normal distribution corresponding to the target detection box with the negative value of its own velocity as the variable and the relative velocity of the updated target detection box as the expectation.

[0178] In this embodiment, for a target detection box whose state has converged, the relative velocity of the target detection box can be determined, and its own velocity can also be determined. This own velocity can be, for example, the speed of an autonomous vehicle. Furthermore, using the negative value of its own velocity as a variable and the updated relative velocity of the target detection box as the expectation, a probability density function of a normal distribution cumulative probability is constructed, thereby determining the normal distribution cumulative probability corresponding to its own velocity.

[0179] Optionally, the cumulative probability F(x; μ,σ) of this normal distribution is:

[0180] Where x represents the negative value of its own velocity, Δv represents the preset velocity difference, μ represents the expected relative velocity, and σ represents the standard deviation of the relative velocity.

[0181] In this embodiment, a speed difference Δv is preset, which can be dynamically adjusted based on actual conditions. After detecting its own speed v, the negative value of its own speed v can be used as the variable x, i.e., x = -v; and after determining the relative speed speed' of the target detection box, this relative speed speed' can be used as the expectation μ. Furthermore, the relative speed standard deviation σ can be calculated based on the relative speed observations of multiple frames. It can be understood that the variance σ2 of the relative speed is also the variance P in the covariance matrix. ss .

[0182] Step C3: If the cumulative probability of the normal distribution is greater than the preset probability value, determine that the updated target detection box is stationary.

[0183] In this embodiment, a preset probability value is used to determine whether the target is stationary, i.e., a preset probability value. This preset probability value can be specifically determined based on the standard deviation σ of the relative velocity. If the cumulative normal distribution F(x; μ, σ) corresponding to the negative value x of its own velocity is greater than the preset probability value (e.g., 0.6 to 0.8), it can be said that x is close to the relative velocity μ, so it can be determined that the target detection box is stationary.

[0184] Judging whether a target is stationary based on the cumulative probability of a normal distribution can improve the detection accuracy of stationary targets and prevent some moving targets from being judged as stationary, thus causing autonomous vehicles to decelerate abnormally.

[0185] Optionally, for radar-observed targets, it can be simply determined whether the associated target detection box is stationary. Specifically, the step "determining whether the updated target detection box is stationary when the updated target detection box state converges" can include step C4.

[0186] Step C4: If the third association count of the updated target detection box with the stationary radar observation target is greater than the third threshold, then the updated target detection box is determined to be stationary.

[0187] In this embodiment, if a target detection box is associated with a stationary radar observation target, and the third association number of the associated stationary radar observation target is greater than the third threshold, since the radar can generally detect stationary targets relatively accurately, the target detection box can be directly determined to be stationary when the target detection box state converges.

[0188] Figure 4 shows a detailed flowchart of a method for detecting stationary targets, combining camera and radar detection. As shown in Figure 4, the method includes the following steps.

[0189] Step S401, Initialize the target detection box.

[0190] For example, an initial target detection box is created for the observed target acquired by the camera, and Kalman filtering state prediction is performed to obtain an updated target detection box.

[0191] Step S402: Extract the target observed by the camera.

[0192] For example, the current frame image captured by the camera can be acquired, and the camera-observed targets in the current frame image can be identified, specifically including MCDO-observed targets and non-MCDO-observed targets.

[0193] Step S403: First round of association and update of MCDO observation targets.

[0194] Step S404: Association and update of non-MCDO observation targets.

[0195] Step S405: Create target detection boxes for the remaining camera-observed targets.

[0196] Step S406: First round of association and update of unassociated MCDO observation targets.

[0197] Step S407: Delete invalid target detection boxes.

[0198] The processes of steps S403 to S407 are detailed in the descriptions of steps A1 to A6 above, and will not be repeated here.

[0199] Step S408: Determine whether the first association count N1 is greater than the first threshold Th1. If N1>Th1, it means that the target detection box state has converged, and proceed to the next step S409.

[0200] If the MCDO observation target is successfully associated after step S403, then step S408 can be executed.

[0201] In step S409, if the relative velocity of the target detection box is less than the current velocity threshold, the target detection box remains stationary. Then, proceed to step S415.

[0202] For details, please refer to the relevant description of step C1, which will not be repeated here.

[0203] Step S410: Determine whether the second association count N2 is greater than the second threshold Th2. If N2>Th2, then continue to execute the subsequent step S411.

[0204] Step S411: Determine whether the covariance matrix P satisfies the convergence condition. If the covariance matrix P satisfies the convergence condition, it indicates that the target detection box state has converged, and proceed to the next step S412.

[0205] Step S412: Determine the cumulative probability of the normal distribution corresponding to its own velocity. If the cumulative probability of the normal distribution is greater than the preset probability value, determine that the target in the target detection box is stationary. Then, execute the subsequent step S415.

[0206] For details, please refer to the relevant descriptions in steps C2 to C3, which will not be repeated here.

[0207] Step S413: Extract the target observed by the stationary radar.

[0208] Step S414: Determine whether the third association count N3 is greater than the third threshold Th3. If N3>Th3, then directly determine that the target detection box is stationary and execute the subsequent step S415.

[0209] Step S415: Publish the static target detection bounding box.

[0210] For example, a static target detection box can be used downstream of the abdomen to perform planning control based on the static target detection box.

[0211] The method for detecting stationary targets provided in this application uses a camera to create a target detection box and combines the camera and radar to update the target detection box to determine whether the target is stationary. It has a high precision and recall rate for detecting stationary targets and can effectively avoid the problem of missed detection of stationary vehicle targets caused by missed detection by long-range cameras and radar tracks.

[0212] This embodiment also provides a stationary target detection device, which is used to implement the above embodiments and optional implementations; details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0213] This embodiment provides a detection device for stationary targets, as shown in Figure 5, including:

[0214] The target recognition module 501 is used to acquire the current frame image captured by the camera and identify the camera-observed target in the current frame image;

[0215] The association update module 502 is used to associate the camera-observed target with the current target detection box, and to predict and update the state of the associated target detection box according to the relative motion parameters of the camera-observed target; the current target detection box includes detection boxes created based on observed historical targets;

[0216] The convergence judgment module 503 is used to determine whether the updated target detection box has converged.

[0217] The detection module 504 is used to determine whether the updated target detection box is stationary when the updated target detection box state converges.

[0218] In some optional implementations, the association update module 502 associates the camera-observed target with the current target detection box, including:

[0219] Associate the first camera observation target with the current target detection box; the first camera observation target is the camera observation target corresponding to the most critical moving object;

[0220] Next, the second camera observation target is associated with a target detection box that is not currently associated with the first camera observation target; the second camera observation target is a camera observation target other than the first camera observation target.

[0221] Create new target detection boxes for second-camera observation targets that could not be associated with target detection boxes.

[0222] In some optional implementations, the association update module 502 associates the camera-observed target with the current target detection box, and further includes:

[0223] If the first camera observation target cannot be associated with a target detection box, after creating a new target detection box for the second camera observation target that cannot be associated with a target detection box, the first camera observation target is reassociated with the target detection box that is not currently associated with the camera observation target;

[0224] If the target observed by the first camera fails to be associated with a target detection box again, a new target detection box is created for the target observed by the first camera.

[0225] In some optional implementations, the convergence determination module 503 determines whether the updated target detection box has converged, including:

[0226] Determine the first association count of the updated target detection box; the first association count is the number of times the updated target detection box is associated with the camera observation target corresponding to the most critical moving object;

[0227] If the first association count is greater than the first threshold, the updated target detection box state is determined to have converged.

[0228] And / or,

[0229] Determine the second association count of the updated target detection box; the second association count is the number of times the updated target detection box is associated with the camera-observed target;

[0230] If the second association count is greater than the second threshold, determine the covariance matrix between the relative velocity and relative depth corresponding to the updated target detection box;

[0231] If the covariance matrix satisfies the convergence condition, the updated target detection box state is determined to be converged.

[0232] In some alternative implementations, the covariance matrix is:

[0233] Where depth represents the relative depth observation, speed represents the relative velocity observation, and P(depth,speed) represents the covariance matrix between relative velocity and relative depth. dd P represents the variance of the relative depth observations. ss P represents the variance of the relative velocity observations. ds P represents the covariance between the relative depth observation and the relative velocity observation. sdThis represents the covariance between the relative velocity observation and the relative depth observation;

[0234] The convergence conditions include:

[0235] Where depth' is the relative depth of the target detection box, and k1 and k2 are preset threshold parameters.

[0236] In some optional implementations, the detection module 504 determines whether the updated target detection box is stationary when the updated target detection box state converges, including:

[0237] If the relative velocity of the updated target detection box is less than the current velocity threshold when the updated target detection box state converges, then the updated target detection box is determined to be stationary.

[0238] or,

[0239] If the updated target detection box state converges, the cumulative probability of the normal distribution corresponding to the self-velocity is determined by using the negative value of its own velocity as a variable and the relative velocity of the updated target detection box as the expectation.

[0240] If the cumulative probability of the normal distribution is greater than a preset probability value, the updated target detection box is determined to be stationary.

[0241] In some optional implementations, the cumulative probability of the normal distribution is:

[0242] Where x represents the negative value of its own velocity, Δv represents the preset velocity difference, μ represents the expected relative velocity, and σ represents the standard deviation of the relative velocity.

[0243] In some optional implementations, the target identification module 501 is further configured to determine the radar observation target currently identified by the radar and filter out stationary radar observation targets;

[0244] The association update module 502 is further configured to associate the stationary radar observed target with the current target detection box, and to predict and update the state of the associated target detection box according to the relative motion parameters of the stationary radar observed target.

[0245] In some optional implementations, the convergence determination module 503 determines whether the updated target detection box has converged, including:

[0246] The third association count of the updated target detection box is determined; the third association count is the number of times the updated target detection box is associated with a stationary radar observation target; if the third association count is greater than a third threshold, the state of the updated target detection box is determined to be converged.

[0247] When the updated target detection box state converges, the detection module 504 determines whether the updated target detection box is stationary, including:

[0248] If the third association count of the updated target detection box with a stationary radar-observed target is greater than a third threshold, the updated target detection box is determined to be stationary.

[0249] In some optional implementations, the association update module 502 is further configured to:

[0250] Delete the target detection boxes that meet the failure criteria;

[0251] The failure conditions include: the target detection box is not associated for a period of time exceeding a first preset time; or, the target type of the target detection box is inconsistent with the target type of the associated observed target, and the inconsistency lasts for a period of time exceeding a second preset time.

[0252] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0253] In this embodiment, the stationary target detection device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, including a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0254] This application also provides a vehicle, which includes a vehicle controller that can implement the stationary target detection method provided in any of the above embodiments.

[0255] This application also provides a computer device having the stationary target detection apparatus shown in FIG5 above. For example, the computer device can be a vehicle controller.

[0256] Please refer to Figure 6, which is a schematic diagram of the structure of a computer device provided in an optional embodiment of this application. As shown in Figure 6, the computer device includes: one or more processors 10, memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components communicate with each other using different buses and can be installed on a common motherboard or otherwise as needed. The processor can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, multiple processors and / or multiple buses can be used with multiple memories, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 6 uses one processor 10 as an example.

[0257] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may also include hardware chips. These hardware chips may be application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or combinations thereof. The programmable logic devices may be complex programmable logic devices (CLPs), field-programmable gate arrays (FPGAs), general-purpose array logic (GDAs), or any combination thereof.

[0258] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0259] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0260] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0261] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0262] This application also provides a computer-readable storage medium. The methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; optionally, the storage medium may also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0263] A portion of this application can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to this application through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0264] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method of detecting a stationary target, characterized by, The method comprises: acquiring a current frame image collected by a camera, and identifying a camera observation target in the current frame image; associating the camera observation target with a current target detection box, and performing state prediction and updating on the associated target detection box according to relative motion parameters of the camera observation target; the current target detection box comprises a detection box created based on an observed historical target; determining whether the updated target detection box is state-converged; in the case that the updated target detection box is state-converged, determining whether the updated target detection box is stationary.

2. The method of claim 1, wherein, The association of the camera observation target with the current target detection box comprises: associating a first camera observation target with the current target detection box; the first camera observation target is a camera observation target corresponding to a most critical moving object; then, associating a second camera observation target with a target detection box that is not associated with the first camera observation target; the second camera observation target is a camera observation target other than the first camera observation target; creating a new target detection box for the second camera observation target that is unable to associate a target detection box.

3. The method of claim 2, wherein, The association of the camera observation target with the current target detection box further comprises: in the case that the first camera observation target is unable to associate a target detection box, after the creation of the new target detection box for the second camera observation target that is unable to associate a target detection box, re-associating the first camera observation target with a target detection box that is not associated with the camera observation target; in the case that the first camera observation target is again unable to associate a target detection box, creating a new target detection box for the first camera observation target.

4. The method of claim 1, wherein, The determination of whether the updated target detection box is state-converged comprises: determining a first association frequency of the updated target detection box; the first association frequency is the number of times that the updated target detection box associates the camera observation target corresponding to the most critical moving object; in the case that the first association frequency is greater than a first threshold value, determining that the updated target detection box is state-converged; and / or, determining a second association frequency of the updated target detection box; the second association frequency is the number of times that the updated target detection box associates the camera observation target; in the case that the second association frequency is greater than a second threshold value, determining a covariance matrix between a relative speed and a relative depth corresponding to the updated target detection box; in the case that the covariance matrix satisfies a convergence condition, determining that the updated target detection box is state-converged.

5. The method of claim 4, wherein, The covariance matrix is: where depth represents the relative depth observation, speed represents the relative speed observation, P dd represents the variance of the relative depth observation, P ss represents the variance of the relative speed observation, P ds represents the covariance between the relative depth observation and the relative speed observation, P sd represents the covariance between the relative speed observation and the relative depth observation; The convergence condition includes: Wherein, depth' is the relative depth of the target detection box, and k1 and k2 are both preset threshold parameters.

6. The method of claim 1, wherein, The determination of whether the updated target detection box is stationary in the case that the updated target detection box is state-converged comprises: in the case that the updated target detection box is state-converged, if the relative speed of the updated target detection box is less than a current speed threshold value, it is determined that the updated target detection box is stationary; or, In a case where the updated target detection box state converges, taking a negative value of the ego speed as a variable and a relative speed of the updated target detection box as an expectation, a normal distribution cumulative probability corresponding to the ego speed is determined; In a case where the normal distribution cumulative probability is greater than a preset probability value, it is determined that the updated target detection box is stationary.

7. The method of claim 6, wherein, The normal distribution cumulative probability is: Wherein, x represents the negative value of the ego speed, Δv represents the preset speed difference value, μ represents the relative speed expectation, and σ represents the relative speed standard deviation.

8. The method of claim 1, wherein, Further comprising: A radar currently identified radar observation target is determined, and a stationary radar observation target is screened out; The stationary radar observation target is associated with a current target detection box, and a state of the associated target detection box is predicted and updated according to relative motion parameters of the stationary radar observation target.

9. The method of claim 8, wherein, The judgment of whether the updated target detection box state converges comprises: A third association frequency of the updated target detection box is determined; the third association frequency is a frequency of the updated target detection box associated with the stationary radar observation target; in a case where the third association frequency is greater than a third threshold value, it is determined that the updated target detection box state converges; In a case where the third association frequency of the updated target detection box associated with the stationary radar observation target is greater than the third threshold value, it is determined that the updated target detection box is stationary. Further comprising:

10. The method of claim 1, wherein, A target detection box meeting a failure condition is deleted; The failure condition comprises: a time length that a target detection box is not associated exceeds a first preset time length; or a target type of a target detection box is inconsistent with a target type of an associated observation target, and a time length of inconsistency exceeds a second preset time length. The device comprises:

11. An apparatus for detecting a stationary target, characterized by A target identification module is configured to acquire a current frame image collected by a camera and identify a camera observation target in the current frame image; An association update module is configured to associate the camera observation target with a current target detection box and predict and update a state of the associated target detection box according to relative motion parameters of the camera observation target; the current target detection box comprises a detection box created based on an observed historical target; A convergence judgment module is configured to judge whether an updated target detection box state converges; A detection module is configured to determine whether the updated target detection box is stationary in a case where the updated target detection box state converges. Comprise:

12. A computer device, comprising: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the stationary target detection method in any one of claims 1 to 10. The computer readable storage medium stores computer instructions, and the computer instructions are used to make the computer execute the stationary target detection method in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The vehicle comprises a vehicle controller configured to perform the stationary target detection method in any one of claims 1 to 10.

14. A vehicle characterized by comprising: ​

Images