Label avoidance determination method, electronic equipment, device and storage medium

Abstract

The invention discloses a label avoidance determination method and device, electronic equipment and a storage medium, and the electronic equipment comprises a processor which is configured to determine multi-frame image data; determining first state information of the vehicle at the image data acquisition moment and second state information of an obstacle in the image data at the acquisition moment; determining multiple frames of target images based on the first state information, the second state information and the multiple frames of image data; based on the first state information, the second state information and the multiple frames of target images, an avoidance label of the obstacle is determined, and the avoidance label is used for indicating the avoidance operation of the vehicle aiming at the obstacle. According to the technical scheme, the multiple frames of target images are screened out from the multiple frames of image data, and then the avoidance labels of the obstacles in the multiple frames of target images are determined based on the state information of the vehicle and the obstacles at the collection moment of the multiple frames of target images; therefore, automatic labeling of the avoidance label can be accurately and simply realized by using the posterior information in a manner of first performing preliminary screening and then performing accurate labeling.

Description

Technical Field

[0001] This disclosure relates to the field of intelligent driving technology, and in particular to a method for determining avoidance tags, as well as electronic devices, apparatuses, and storage media. Background Technology

[0002] As intelligent driving technologies mature, vehicles need to accurately perceive their surroundings to ensure driving safety. However, accurately determining the interaction between the vehicle and obstacles in complex traffic scenarios and formulating appropriate avoidance strategies remains a major challenge. Avoidance strategies include bypassing or following obstacles, and if bypassing an obstacle, determining whether to go around it from the left or right.

[0003] Currently, intelligent driving systems typically employ end-to-end models, which can directly output the commands needed for obstacle avoidance and vehicle planning and control based on environmental information. However, when training end-to-end models, related technologies often rely on manual annotation to obtain the necessary obstacle avoidance labels. This method is not only time-consuming and labor-intensive but also easily affected by subjective factors, making it difficult to guarantee the consistency and accuracy of the annotation results. Summary of the Invention

[0004] To address the aforementioned technical problems, this disclosure provides a method for determining avoidance tags, as well as an electronic device, apparatus, and storage medium.

[0005] A first aspect of this disclosure provides an electronic device including at least one processor configured to: determine multiple frames of image data; determine first state information of a vehicle at the time of image data acquisition, and second state information of an obstacle in the image data at the time of acquisition; determine multiple target images based on the first state information, the second state information, and the multiple frames of image data; and determine obstacle avoidance tags based on the first state information, the second state information, and the multiple frames of target images, the obstacle avoidance tags being used to instruct the vehicle to perform obstacle avoidance operations.

[0006] A second aspect of this disclosure provides a method for determining an obstacle avoidance tag, comprising: determining multiple frames of image data; determining first state information of a vehicle at the time of image data acquisition, and second state information of an obstacle in the image data at the time of acquisition; determining multiple target images based on the first state information, the second state information, and the multiple frames of image data; and determining an obstacle avoidance tag based on the first state information, the second state information, and the multiple target images, wherein the obstacle avoidance tag is used to instruct the vehicle to perform an obstacle avoidance operation.

[0007] A third aspect of this disclosure provides an apparatus for determining an obstacle avoidance tag, comprising: a data determining module for determining multiple frames of image data; a state determining module for determining first state information of a vehicle at the time of image data acquisition, and second state information of an obstacle in the image data at the time of acquisition; an image determining module for determining multiple frames of target images based on the first state information, the second state information, and the multiple frames of image data; and a tag determining module for determining an obstacle avoidance tag based on the first state information, the second state information, and the multiple frames of target images, wherein the obstacle avoidance tag is used to instruct the vehicle to perform an obstacle avoidance operation.

[0008] A fourth aspect of this disclosure provides a computer-readable storage medium storing a computer program for performing the method for determining avoidance tags provided in the second aspect embodiment above.

[0009] A fifth aspect of this disclosure provides an electronic device comprising: a processor; a memory for storing processor-executable instructions; and a processor for reading executable instructions from the memory and executing the instructions to implement the tag-determination method provided in the second aspect of the disclosure.

[0010] A sixth aspect of this disclosure provides a computer program product that, when instructions in the computer program product are executed by a processor, performs the method for determining avoidance tags provided in the second aspect of the present invention.

[0011] The electronic device provided in this disclosure includes at least one processor, configured to first filter out multiple target images from multiple frames of image data, and then determine obstacle avoidance labels in the multiple target images based on the state information of the vehicle and the obstacle at the time of acquisition of the multiple target images. In this way, this disclosure can not only realize the automatic labeling of avoidance labels, but also, compared with the related technology that determines whether the vehicle should go around or follow the obstacle based on some limited vector information (such as the lane position of the obstacle, whether there are other vehicles in front of or next to the obstacle, etc.), this disclosure can use posterior information to accurately and quickly obtain obstacle avoidance labels by first screening and then accurately labeling. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of an application scenario provided for an exemplary embodiment of this disclosure.

[0013] Figure 2A A schematic diagram of the structure of an electronic device provided for an exemplary embodiment of this disclosure.

[0014] Figure 2B A schematic diagram of the structure of an electronic device provided for another exemplary embodiment of this disclosure.

[0015] Figure 3A This is a schematic diagram of a vehicle and an obstacle provided for an exemplary embodiment of this disclosure.

[0016] Figure 3B A schematic diagram of a vehicle and an obstacle provided for another exemplary embodiment of this disclosure.

[0017] Figure 4 This is a flowchart illustrating a method for determining an avoidance tag, provided as an exemplary embodiment of the present disclosure.

[0018] Figure 5 A flowchart illustrating a method for determining an avoidance tag, provided as another exemplary embodiment of this disclosure.

[0019] Figure 6 A flowchart illustrating a method for determining an avoidance tag, which is yet another exemplary embodiment of this disclosure.

[0020] Figure 7 A flowchart illustrating a method for determining an avoidance tag, which is yet another exemplary embodiment of this disclosure.

[0021] Figure 8 A flowchart illustrating a method for determining an avoidance tag, which is yet another exemplary embodiment of this disclosure.

[0022] Figure 9 A schematic diagram of the structure of an obstacle avoidance tag determination device provided for an exemplary embodiment of this disclosure.

[0023] Figure 10 A schematic diagram of the structure of an obstacle avoidance tag determination device provided for another exemplary embodiment of this disclosure.

[0024] Figure 11 A schematic diagram of the structure of an obstacle avoidance tag determination device provided as another exemplary embodiment of this disclosure. Detailed Implementation

[0025] To explain this disclosure, exemplary embodiments of the disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the disclosure, and not all of them. It should be understood that the disclosure is not limited to exemplary embodiments.

[0026] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0027] Application Overview

[0028] The method for determining avoidance labels provided in this disclosure can be applied to ground truth labeling scenarios during model training, as well as any other feasible scenarios. For example, it can be applied to labeling the sample data required for training an end-to-end intelligent driving model: when training an end-to-end intelligent driving model, the method for determining avoidance labels provided in this disclosure can be used to first perform avoidance labeling on multi-frame image data collected by a visual sensor to obtain labeling results, and then use these labeling results as training samples to train the end-to-end intelligent driving model until the preset training completion conditions are met, resulting in a trained end-to-end intelligent driving model that can be used for vehicle planning and control.

[0029] Figure 1 This is a schematic diagram illustrating an application scenario provided for an exemplary embodiment of this disclosure. For example... Figure 1 As shown, the vehicle 10 is equipped with at least one vision sensor 11 (e.g., a camera). During the driving of the vehicle 10, the vision sensor 11 acquires images of the surrounding environment of the vehicle at a specific frequency, obtaining multiple frames of image data.

[0030] Figure 1 For illustrative purposes only, one vision sensor 11 or multiple vision sensors 11 may be installed on the vehicle 10. This disclosure embodiment... Figure 1 The number and location of the vision sensors 11 are not limited. In practical use, the vision sensors 11 can be the vehicle's forward-facing wide-angle / narrow-angle camera, or the vehicle's side-view / surround-view / panoramic-view camera, etc.

[0031] Furthermore, the obstacle avoidance labels in the multi-frame image data collected by the vision sensor 11 can be processed based on the obstacle avoidance label determination method provided in the embodiments of this disclosure to obtain obstacle avoidance labels in the multi-frame image data. In this way, the image data with obstacle avoidance labels can be used as training sample data for the end-to-end intelligent driving model.

[0032] In related technologies, manual annotation is usually used to obtain the avoidance labels needed for model training. This method is not only time-consuming and labor-intensive, but also easily affected by subjective factors, making it difficult to guarantee the consistency and accuracy of the annotation results, which in turn affects model training.

[0033] To address the aforementioned issues, embodiments of this disclosure provide an electronic device comprising at least one processor configured to first filter out multiple target images from multiple frames of image data, and then determine obstacle avoidance labels in the multiple target images based on the state information of the vehicle and obstacles at the time of acquisition of the multiple target images. Thus, this disclosure not only enables automatic labeling of avoidance labels, but also, compared to related technologies that rely on limited vector information (such as the lane position of the obstacle, whether there are other vehicles in front of or beside the obstacle) to determine whether the vehicle should detour or follow the obstacle, this disclosure can accurately and quickly obtain obstacle avoidance labels using posterior information through a preliminary screening followed by precise labeling.

[0034] Exemplary electronic devices

[0035] Figure 2A This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. The electronic device can be a local terminal device or a cloud server, etc., and this disclosure does not impose any limitations on it. Figure 2A As shown, the electronic device 100 includes at least one processor 101, which is configured to: Determine the image data for multiple frames.

[0036] For example, the multi-frame image data in this disclosure embodiment may be Figure 1 The visual sensor 11 collects image data of the surrounding environment, for example, the image data of the vehicle's surrounding environment collected by the forward-facing camera while the vehicle is in motion.

[0037] In some examples, a large amount of expert driving data can be collected and used as the aforementioned multi-frame image data. Expert driving data refers to a comprehensive, high-precision, and high-quality collection of driving behavior and environmental data gathered by onboard perception, positioning, and control devices from professional drivers operating vehicles in various real-world road scenarios and conditions. For instance, a large amount of expert driving data can be collected and segmented into multi-frame image data, each segment lasting 120 seconds, stored in the cloud for later retrieval. Furthermore, obstacle avoidance annotations can be performed on these 120-second segments of multi-frame image data to obtain annotation results, which can then be used as training samples to train the end-to-end intelligent driving model.

[0038] Determine the first state information of the vehicle at the moment of image data acquisition, and the second state information of the obstacles in the image data at the moment of acquisition.

[0039] For example, embodiments of this disclosure can first determine the acquisition time of multiple frames of image data, and then determine the first state information of the vehicle at that acquisition time, and the second state information of obstacles in the multiple frames of image data at that acquisition time. For example, the multiple frames of image data include Image data collected in real time 1. Image data collected at all times 2. Image data collected at different times 3, Image data collected at different times 4, and Image data 5 was collected at various times, and obstacle A was included in each frame of the image data. Therefore, it is necessary to determine... Time to The first state information of the vehicle and the second state information of obstacle A at any given time.

[0040] Taking an obstacle as an example, in some examples, the first state information of the vehicle and the second state information of the obstacle include, but are not limited to, the vehicle's speed information, the vehicle's position information, and the vehicle's orientation information. The position information can be the coordinates of the vehicle or the obstacle in a preset coordinate system. This preset coordinate system can be the vehicle's VCS (Vehicle Coordinate System), which is a coordinate system centered on the vehicle, with its origin at a specific reference point on the vehicle (such as the center of the rear axle); the X-axis points in the vehicle's forward direction (positive), i.e., the vehicle's longitudinal direction; the Y-axis points to the left of the vehicle (positive), i.e., the vehicle's lateral direction; and the Z-axis is perpendicular to the ground and upwards (positive). Of course, the preset coordinate system in this embodiment can also be other predefined coordinate systems, and this embodiment does not limit this.

[0041] In some examples, the vehicle's first state information and the obstacle's second state information can be determined using the vehicle's GNSS (Global Navigation Satellite System), wheel speed sensors, visual sensors, and radar sensors. For instance, the vehicle's first state information can be determined using GPS data obtained from GNSS, IMU (Inertial Measurement Unit) data, wheel speed signals obtained from wheel speed sensors, image data obtained from visual sensors, and point cloud information obtained from radar sensors. Similarly, the obstacle's second state information can be determined using image data obtained from visual sensors, point cloud information obtained from lidar sensors, and obstacle speed and relative distance obtained from millimeter-wave radar.

[0042] Of course, the first state information of the vehicle and the second state information of the obstacle can also be determined solely from the image data collected by the vision sensor, and this embodiment of the present disclosure does not impose any limitations on this. The image data collected by the vision sensor can be image data collected by vision sensors from different perspectives, or it can be image data collected by a single-view vision sensor.

[0043] In other words, those skilled in the art can choose an appropriate method to determine the first state information of the vehicle at the acquisition time of the multi-frame image data and the second state information of the obstacle at the acquisition time, based on the actual application situation. The embodiments of this disclosure do not limit the method of determining the first state information and the second state information.

[0044] It should be noted that, in this embodiment of the disclosure, obstacles refer to objects or entities that are in motion, or that are currently stationary but may begin to move in the future. These types of obstacles can also be called dynamic objects. For example, other vehicles besides a car, pedestrians, cyclists, animals, etc.

[0045] Based on the first state information, the second state information, and multiple frame image data, multiple target images are determined.

[0046] For example, embodiments of this disclosure can determine multiple target images from multiple frames of image data based on first and second state information, and then use these multiple target images as images to be labeled. For instance, the first and second state information corresponding to each frame of image data can be determined, and then the spatial positional relationship and speed between the vehicle and obstacles at the time of acquisition of each frame of image data can be determined based on the first and second state information. Then, based on the spatial relative relationship and speed information, multiple target images can be determined from the multiple frames of image data. In this way, multiple target images to be labeled can be roughly selected from the multiple frames of image data.

[0047] Based on the first state information, the second state information, and multiple frames of target images, obstacle avoidance tags are determined. These avoidance tags are used to indicate the vehicle's obstacle avoidance operation.

[0048] For example, after selecting multiple target images to be labeled, obstacle avoidance labels can be determined for each target image based on the first state information of the vehicle and the second state information of the obstacle at the time of acquisition of each target image. These obstacle avoidance labels are used to indicate the vehicle's obstacle avoidance operation. The label content of the obstacle avoidance label includes, but is not limited to: detour (e.g., left or right detour), follow, ignore, etc. Detour includes active detour and passive detour, and follow includes active following and passive following. The "ignore" label means that the obstacle does not participate in path planning and obstacle avoidance decision-making, i.e., no obstacle avoidance action is triggered. Furthermore, embodiments of this disclosure can use multiple target images with obstacle avoidance labels as training sample data for an end-to-end intelligent driving model. For example, a large number of target images with "active detour" and "active follow" labels, and a small number of target images with "passive detour," "passive follow," and "ignore" labels can be selected as training sample data.

[0049] In some examples, obstacle avoidance labels are determined based on first state information, second state information, and multiple target images. This includes: determining the acquisition time of the multiple target images; and determining the vehicle's first sub-state information and the obstacle's second sub-state information based on the first state information, second state information, and the acquisition time of the multiple target images. The obstacle avoidance labels are then determined based on the first sub-state information, second sub-state information, and multiple target images. Here, the first sub-state information refers to the vehicle's speed, position, and orientation at the acquisition time of the multiple target images; similarly, the second sub-state information refers to the obstacle's speed, position, and orientation at the acquisition time of the multiple target images.

[0050] The electronic device provided in this disclosure includes at least one processor, configured to first filter out multiple target images from multiple frames of image data, and then determine the obstacle avoidance label in the multiple target images based on the state information of the vehicle and the obstacle at the time of acquisition of the multiple target images. In this way, this disclosure can not only realize the automatic labeling of avoidance labels, but also, compared with the related technology that determines whether the vehicle should go around or follow the obstacle based on some limited vector information (such as the lane position of the obstacle, whether there are other vehicles in front of or next to the obstacle, etc.), this disclosure can use posterior information to accurately and quickly obtain the obstacle avoidance label by first screening and then accurately labeling.

[0051] In some embodiments of this disclosure, such as Figure 2A As shown, the processor 101 executes the process of determining multiple target images based on the first state information, the second state information, and multiple frame image data, specifically configured as follows: Based on the first state information and the second state information, the target frame interval is determined. The target frame interval is the frame interval in which there is a possibility of interaction between the vehicle and the obstacle.

[0052] For example, a target frame interval refers to a continuous subsequence extracted from a sequence of multiple consecutive image frames. Embodiments of this disclosure can determine a target frame interval in multi-frame image data based on the vehicle's first state information and the obstacle's second state information. This target frame interval is the frame interval where interaction between the vehicle and the obstacle is possible. Specifically, by using information such as the position, speed, and orientation of the vehicle and the obstacle at the acquisition time of each frame of image data, it can be determined whether there is a potential interaction possibility between the vehicle and the obstacle at the acquisition time of each frame of image data, thereby identifying the target frame interval with interaction potential.

[0053] In some examples, interactive keyframes can be determined from multi-frame image data based on first and second state information; then, the target frame range can be determined based on these interactive keyframes. For example, the multi-frame image data includes... Image data collected in real time 1. Image data collected at all times 2. Image data collected at different times, and Image data collected at each moment: 4, ..., and so on, until... Image data acquired at various times 11. Further, based on... Time to The first state information of the vehicle and the second state information of the obstacle at each time point are used to determine the interaction keyframes. The frame data at each moment. Furthermore, the time interval [ The corresponding frame interval is determined as the target frame interval. and It can be a natural number greater than or equal to 1, and its unit is seconds. and They can be the same or different.

[0054] Based on the target frame interval and multi-frame image data, determine the multi-frame target images.

[0055] For example, if the frame rate of the visual sensor that acquires multiple frames of image data is 30 FPS (frames per second), that is, it acquires 30 frames of images per second, and the target frame interval is a time interval [ The corresponding frame interval; then, the multi-frame target image is the image data at time T in the multi-frame image data, and the image data 30 frames before time T. Image data of the frame, and 30 after time T. Image data of a frame.

[0056] In some examples, the target frame interval determined from multiple frames of image data based on the first and second state information may be one or more. Furthermore, when there are multiple target frame intervals, a single target frame interval can be determined from among them. Based on this determined target frame interval and the multiple frames of image data, multiple target images can be determined. This saves computational resources and improves computational efficiency. For example, a single target frame interval can be determined from among multiple target frame intervals based on their varying importance to determine the multiple target images.

[0057] It should be noted that some target frame intervals can be selected from multiple target frame intervals, or all target frame intervals can be selected to determine multiple target images. This disclosure does not limit this.

[0058] The electronic device provided in this disclosure includes at least one processor configured to determine a target frame interval based on first state information and second state information. The target frame interval is a frame interval in which there is a possibility of interaction between the vehicle and an obstacle. Based on the target frame interval and multiple frame image data, multiple target images are determined. In this way, the target frame interval in which there is a possibility of interaction between the vehicle and the obstacle can be determined first based on the state of the vehicle and the obstacle, and then multiple target images can be determined based on the target frame interval, thereby conveniently, quickly and accurately determining the target image to be labeled.

[0059] In some embodiments of this disclosure, such as Figure 2A As shown, the processor 101 executes a process based on the first state information and the second state information to determine the target frame interval, specifically configured as follows:

[0060] Based on the first state information and the second state information, interactive keyframes are determined from multiple frames of image data.

[0061] For example, it can be determined whether a frame of image data is an interactive keyframe based on the first state information of the vehicle and the second state information of the obstacle in each frame of image data. The multi-frame image data may include one interactive keyframe or multiple interactive keyframes.

[0062] For example, if the vehicle and the obstacle are longitudinally aligned at a time T, meaning the obstacle's X-axis coordinate in the vehicle's VCS coordinate system is 0, then the image data corresponding to time T is used as an interactive keyframe. Similarly, if the vehicle and the obstacle are both stationary, and the obstacle is the vehicle's CIPV (Closest In-Path Vehicle) for a period of [T0-T1], then the image data corresponding to time T0 and time T1 are used as interactive keyframes. Furthermore, if the vehicle and the obstacle are laterally aligned for the last time at time T, meaning the obstacle's Y-axis coordinate in the vehicle's VCS coordinate system is 0, then the image data corresponding to time T is used as an interactive keyframe. Finally, if the vehicle's longitudinal distance never exceeds that of the obstacle in multiple frames of image data, then the time T with the minimum longitudinal distance between the vehicle and the obstacle is determined, and the image data corresponding to time T is used as an interactive keyframe.

[0063] Based on the interactive keyframes, determine the index information corresponding to the interactive keyframes.

[0064] For example, in this embodiment of the present disclosure, the index information corresponding to the interactive keyframe refers to the identifier that uniquely identifies, locates, and searches for the interactive keyframe in multi-frame image data. For instance, the index information of the interactive keyframe may be the timestamp information corresponding to the interactive keyframe, which is used to record the absolute or relative time when the interactive keyframe was acquired. Alternatively, the index information of the interactive keyframe may be the frame number corresponding to the interactive keyframe.

[0065] The target frame range is determined based on the index information.

[0066] The target frame interval can be understood as the frame interval where there is a possibility of interaction between the vehicle and obstacles. For example, after determining the interaction keyframes, a certain time range can be extended forward and backward, centered on the index information corresponding to the interaction keyframes, to determine the start and end times of the target frame interval. For instance, assuming the index information corresponding to the interaction keyframe is time T, then T minus the first time offset can be used as the start time of the target frame interval, and T plus the second time offset can be used as the end time of the target frame interval. Here, the first time offset (i.e., a natural number)... ) and the offset at the second time step (i.e., the natural number) The value of ) can be set according to the actual application scenario, and this embodiment does not impose any restrictions on it. Furthermore, the first time offset corresponding to the interaction keyframe in different scenarios can be the same or different; similarly, the second time offset corresponding to the interaction keyframe in different scenarios can be the same or different, and this embodiment does not impose any restrictions on it.

[0067] The electronic device provided in this disclosure includes at least one processor configured to determine interactive keyframes from multiple frames of image data based on first state information and second state information; determine index information corresponding to the interactive keyframes based on the interactive keyframes; and determine a target frame range based on the index information. In this way, the target frame range can be accurately and quickly determined through the determined interactive keyframes.

[0068] In some embodiments of this disclosure, such as Figure 2A As shown, the processor 101 performs the task of determining interactive keyframes from multi-frame image data based on first and second state information, specifically configured as follows: Based on the first state information and the second state information, the first image is determined from the multi-frame image data; wherein, at the time of acquisition of the first image, the longitudinal coordinates of the vehicle and the obstacle in the preset coordinate system are the same.

[0069] For example, a first image can be determined from multiple frames of image data based on the vehicle's position information in the first state information and the obstacle's position information in the second state information. At the time the first image is acquired, the vehicle and the obstacle have the same longitudinal coordinate value in a preset coordinate system. For example, this preset coordinate system can be the vehicle's VCS coordinate system. When the obstacle's X-axis coordinate in the vehicle's VCS coordinate system is 0, it is considered that the longitudinal alignment condition (i.e., the longitudinal coordinate values ​​are the same) is met at that moment, and the image data corresponding to that moment is determined as the aforementioned first image. If the origin of the vehicle's VCS coordinate system is the center of the vehicle's rear axle, and the obstacle is another vehicle, it can be determined whether the center of the other vehicle's rear axle has a X-axis coordinate value of 0 in the vehicle's VCS coordinate system.

[0070] In some examples, such as Figure 3A As shown, Figure 3A The longitudinal coordinates of vehicle 31 and other vehicles 32 in the vehicle's VCS coordinate system are the same (that is, the X-axis coordinates of both the vehicle and the obstacle are 0 in the vehicle's VCS coordinate system).

[0071] Based on the first state information, the second state information, and the first image, the first speed of the vehicle and the second speed of the obstacle are determined.

[0072] For example, the acquisition time of the first image can be determined first, and then the first speed of the vehicle can be determined from the first state information based on the acquisition time; similarly, the second speed of the obstacle can be determined from the second state information based on the acquisition time.

[0073] In response to a first speed being greater than or equal to a second speed, the first image is determined as an interactive keyframe.

[0074] For example, if the speed of the vehicle is greater than the speed of the obstacle at the time the first image is acquired, the first image is determined as an interactive keyframe, and there is a possibility that the vehicle will actively bypass the obstacle within the target frame interval corresponding to the interactive keyframe.

[0075] The electronic device provided in this disclosure includes at least one processor configured to determine a first image from multiple frames of image data based on first state information and second state information; wherein, at the time the first image is acquired, the longitudinal coordinates of the vehicle and the obstacle are the same in a preset coordinate system; based on the first state information, the second state information, and the first image, a first speed of the vehicle and a second speed of the obstacle are determined; in response to the first speed being greater than or equal to the second speed, the first image is determined as an interactive keyframe. Thus, it is possible to determine interactive keyframes where the vehicle may actively avoid the obstacle.

[0076] In some embodiments of this disclosure, such as Figure 2A As shown, the processor 101 performs the task of determining interactive keyframes from multi-frame image data based on first and second state information, and is further configured to: In response to the failure to determine the interaction keyframe, a second image is determined from the multi-frame image data based on the first state information and the second state information; wherein, at the time of acquisition of the second image, the obstacle is the nearest vehicle in the same direction in the lane where the vehicle is located, and the vehicle and the obstacle are both stationary.

[0077] For example, if the interaction keyframe is not determined through the above method steps, the second image can be determined from the multi-frame image data based on the first state information of the vehicle and the second state information of the obstacle; wherein, at the time of acquisition of the second image, the obstacle is the nearest vehicle in the same direction in the lane where the vehicle is located, and the vehicle and the obstacle are both stationary.

[0078] For example, if the CIPV (Cross-Issue Per Vehicle) period when the vehicle and the obstacle are both stationary and the obstacle is the vehicle is [T0-T1], then this time (i.e., time T0 and time T1) is considered to satisfy the conditions that the obstacle is the nearest vehicle traveling in the same direction in the lane where the vehicle is located, and the vehicle and the obstacle are both stationary. Therefore, the image data corresponding to time T0 and the image data corresponding to time T1 are determined as the aforementioned second image. In other words, if the obstacle is another vehicle, then time T0 is the moment when the vehicle in front has just stopped, and time T1 is the moment when the vehicle starts moving again after stopping.

[0079] The second image is designated as the interaction keyframe.

[0080] For example, the second image can be identified as an interactive keyframe, and there is a possibility that the vehicle will actively follow an obstacle within the target frame interval corresponding to the interactive keyframe.

[0081] In some examples, if the second image includes image data corresponding to time T0 and image data corresponding to time T1, then the target frame interval determined based on this interactive keyframe can be a time interval. , The corresponding frame interval, where, (i.e., the offset at the first moment) and (That is, the offset at the second moment) can be a natural number greater than or equal to 1, and its unit is seconds. and They can be the same or different.

[0082] The electronic device provided in this disclosure includes at least one processor configured to, in response to the absence of an interaction keyframe, determine a second image from multiple frames of image data based on first and second state information; wherein, at the time the second image is acquired, the obstacle is the nearest vehicle traveling in the same direction in the lane where the vehicle is located, and both the vehicle and the obstacle are stationary; the second image is then determined as an interaction keyframe. In this way, it is possible to determine an interaction keyframe in which the vehicle may actively follow the obstacle.

[0083] In some embodiments of this disclosure, such as Figure 2A As shown, the processor 101 performs the task of determining interactive keyframes from multi-frame image data based on first and second state information, and is further configured to: In response to the absence of an interaction keyframe and the first speed being less than the second speed, the first image is determined as the interaction keyframe.

[0084] For example, if the interaction keyframe is not determined through the above method steps, it can be further determined whether the first speed of the vehicle is less than the second speed of the obstacle; if so, the first image determined above is used as the interaction keyframe.

[0085] In some embodiments of this disclosure, such as Figure 2A As shown, the processor 101 performs the task of determining interactive keyframes from multi-frame image data based on first and second state information, and is further configured to: In response to the absence of an interaction keyframe, multiple third images are determined from the multiple image data based on the first and second state information; wherein, at the acquisition time of the third image, the lateral coordinates of the vehicle and the obstacle in the preset coordinate system are the same; based on the multiple third images, a fourth image is determined and the fourth image is identified as the interaction keyframe.

[0086] For example, if the interaction keyframe is not determined through the above method steps, multiple third images can be determined from the multiple image data based on the first state information and the second state information; wherein, at the time of acquisition of the third image, the lateral coordinates of the vehicle and the obstacle in the preset coordinate system are the same; further, the multiple third images are sorted according to the time dimension, and a fourth image is determined from the multiple third images according to the sorting result, and the fourth image is determined as the interaction keyframe.

[0087] For example, the preset coordinate system can be the vehicle's VCS coordinate system. When the obstacle's Y-axis coordinate in the vehicle's VCS coordinate system is 0, it is considered that the lateral alignment condition is met at that moment (i.e., the lateral coordinate values ​​are the same), and the image data corresponding to that moment is determined as the third image. Specifically, if the origin of the vehicle's VCS coordinate system is the center of the vehicle's rear axle, and the obstacle is another vehicle, then it can be determined whether the rear axle center of the other vehicle has a Y-axis coordinate of 0 in the vehicle's VCS coordinate system. Furthermore, multiple frames of third images can be sorted according to their acquisition time, and the third image acquired at the latest acquisition time can be used as the fourth image. In some examples, such as... Figure 3B As shown, Figure 3B The lateral coordinates of vehicle 31 and other vehicles 32 in the vehicle's VCS coordinate system are the same (that is, the Y-axis coordinates of both the vehicle and the obstacle are 0 in the vehicle's VCS coordinate system).

[0088] In some embodiments of this disclosure, such as Figure 2A As shown, the processor 101 executes the process of determining an interaction keyframe from multiple frames of image data based on first state information and second state information. It is also specifically configured to: in response to the fact that no interaction keyframe has been determined and the longitudinal coordinate of the obstacle in the preset coordinate system is greater than the longitudinal coordinate of the vehicle at the time of acquisition of the multiple frames of image data, determine the longitudinal distance between the vehicle and the obstacle in the preset coordinate system at the time of acquisition of the multiple frames of image data; determine the target longitudinal distance based on multiple longitudinal distances, and determine the image corresponding to the target longitudinal distance as the interaction keyframe.

[0089] For example, if the above method steps fail to determine the interaction keyframe, and the longitudinal coordinate of the obstacle in the preset coordinate system (such as the vehicle's VCS coordinate system) is greater than the longitudinal coordinate of the vehicle at the time of acquisition of multiple frames of image data (i.e., the vehicle is always behind the obstacle at the time of acquisition of multiple frames of image data), then the longitudinal distance between the vehicle and the obstacle in the preset coordinate system at the time of acquisition of each frame of image data can be further determined. Then, the multiple longitudinal distances are sorted according to their values, and the target longitudinal distance is determined from the multiple longitudinal distances based on the sorting results. The image corresponding to the target longitudinal distance is then identified as the interaction keyframe. For example, the longitudinal distance between the vehicle and the obstacle along the vehicle's forward direction in the vehicle's VCS coordinate system at the time of acquisition of each frame of image data can be determined, and the multiple longitudinal distances are sorted according to their values. Then, the longitudinal distance with the smallest value is determined as the target longitudinal distance based on the sorting results.

[0090] In some embodiments of this disclosure, such as Figure 2A As shown, the processor 101 executes the process of determining obstacle avoidance tags based on first state information, second state information, and multiple frames of target images, specifically configured as follows: Based on the first state information and the second state information, the lateral distance between the vehicle and the obstacle in the preset coordinate system and the relative position between the vehicle and the obstacle are determined at the acquisition time of the multi-frame target image.

[0091] For example, based on the first state information, the second state information, and the acquisition time of multiple target images, the first sub-state information of the vehicle and the second sub-state information of the obstacle can be determined first. Then, based on the first and second sub-state information, the lateral distance between the vehicle and the obstacle in a preset coordinate system at the acquisition time of the multiple target images, as well as the relative position between the vehicle and the obstacle, can be determined. Here, the first sub-state information refers to the vehicle's speed, position, and orientation at the acquisition time of the multiple target images, and the second sub-state information refers to the obstacle's speed, position, and orientation at the acquisition time of the multiple target images. The preset coordinate system can be the vehicle's VCS coordinate system.

[0092] For example, at the acquisition time of each frame of target image, the lateral distance between the vehicle and the obstacle in the vehicle's VCS coordinate system perpendicular to the vehicle's forward direction, as well as the relative position between the vehicle and the obstacle, can be determined.

[0093] In response to the lateral distance satisfying the first distance relationship or the relative position satisfying the position relationship, the obstacle avoidance label in the multi-frame target image is determined as the first avoidance label.

[0094] For example, in response to the lateral distance satisfying a first distance relationship or the relative position satisfying a positional relationship, the obstacle avoidance label in the multi-frame target image is determined to be "ignore", that is, at the time of acquisition of the multi-frame target image, the vehicle does not need to avoid the obstacle.

[0095] For example, at the time of acquiring at least one frame of the target image, if the lateral distance between the vehicle and the obstacle in the preset coordinate system is greater than or equal to a first preset lateral value (e.g., 8 meters), then the obstacle's avoidance label in the multi-frame target image is determined to be "ignore". Similarly, at the time of acquiring each frame of the target image, if the lateral distance between the vehicle and the obstacle in the preset coordinate system is greater than or equal to a second preset lateral value (e.g., 5 meters), then the obstacle's avoidance label in the multi-frame target image is determined to be "ignore". Furthermore, at the time of acquiring each frame of the target image, if the relative position indicates that the obstacle is behind the vehicle, then the obstacle's avoidance label in the multi-frame target image is determined to be "ignore".

[0096] The electronic device provided in this disclosure includes at least one processor configured to determine, based on first state information and second state information, the lateral distance between a vehicle and an obstacle in a preset coordinate system at the acquisition time of multiple target images, and the relative position between the vehicle and the obstacle; in response to the lateral distance satisfying a first distance relationship or the relative position satisfying a position relationship, to determine that the obstacle's avoidance label in the multiple target images is a first avoidance label; thus, it is possible to determine whether the obstacle's avoidance label in the multiple target images is "ignored".

[0097] In some embodiments of this disclosure, such as Figure 2A As shown, the processor 101 executes the process of determining obstacle avoidance tags based on first state information, second state information, and multiple frames of target images, and is further configured to: In response to the lateral distance satisfying the second distance relationship, the obstacle avoidance label in the multi-frame target image is determined as the second avoidance label.

[0098] For example, in embodiments of this disclosure, if the first avoidance label is not determined, it is possible to continue determining whether the lateral distance satisfies the second distance relationship. Further, in response to the lateral distance satisfying the second distance relationship, the avoidance label of the obstacle in the multi-frame target image is determined to be "following", that is, at the time of acquisition of the multi-frame target image, the vehicle's avoidance operation for the obstacle is a following operation.

[0099] For example, at the moment of acquiring each frame of the target image, if the lateral distance between the vehicle and the obstacle in the preset coordinate system is less than the third preset lateral value (such as 1 meter), then the obstacle avoidance label is determined to be "follow".

[0100] The electronic device provided in this disclosure includes at least one processor configured to determine, in response to a lateral distance satisfying a second distance relationship, an obstacle avoidance label in a multi-frame target image as a second avoidance label. Thus, it is possible to continue determining whether an obstacle avoidance label in the multi-frame target image is a second avoidance label even if a first avoidance label has not been determined.

[0101] In some embodiments of this disclosure, such as Figure 2A As shown, the processor 101 is further configured to: determine the longitudinal distance between the vehicle and the obstacle in a preset coordinate system at the acquisition time of the multi-frame target image based on the first state information and the second state information; determine the category attribute of the obstacle; and determine the obstacle's avoidance label in the multi-frame target image as a second sub-avoidance label based on the category attribute, lateral distance, longitudinal distance, third speed, and fourth speed.

[0102] For example, in this embodiment of the present disclosure, the longitudinal distance between the vehicle and the obstacle in a preset coordinate system, as well as the third speed of the vehicle and the fourth speed of the obstacle, can be determined based on the first sub-state information and the second sub-state information at the acquisition time of multiple target images. For example, the longitudinal distance between the vehicle and the obstacle along the vehicle's forward direction in the vehicle's VCS coordinate system, as well as the third speed of the vehicle and the fourth speed of the obstacle, can be determined at the acquisition time of each frame of target images. Further, the category attribute of the obstacle can be determined, such as whether the obstacle is a vehicle. Then, based on the category attribute, lateral distance, longitudinal distance, third speed, and fourth speed, the obstacle avoidance label in the multiple target images is determined as the second sub-avoidance label. The second sub-avoidance label is "actively following". In this way, multiple target images with the obstacle avoidance label "actively following" can be further filtered out and used as training samples for the end-to-end intelligent driving model.

[0103] For example, if all of the following conditions are met simultaneously, the obstacle avoidance label in the multi-frame target image is determined to be "actively following": ① The obstacle avoidance label is "following". ② The lateral distance between the vehicle and the obstacle is less than the fourth lateral preset value; in specific implementation, this can be achieved by determining that the vehicle's bounding box overlaps the obstacle's bounding box by at least 0.5m in the lateral direction. In this case, the fourth lateral preset value is equivalent to half the width of the vehicle's bounding box plus half the width of the obstacle's bounding box minus 0.5m. ③ The longitudinal distance between the vehicle and the obstacle is less than the first longitudinal preset value (e.g., 15m). ④ The longitudinal distance between the vehicle and the obstacle is greater than the second longitudinal preset value (e.g., 0m). ⑤ The obstacle's category attribute is vehicle. ⑥ The vehicle's third speed is less than the first preset speed (e.g., 0.3m / s). ⑦ The obstacle's fourth speed is less than the second preset speed (e.g., 0.3m / s).

[0104] In some embodiments of this disclosure, such as Figure 2A As shown, the processor 101 executes the process of determining obstacle avoidance tags based on first state information, second state information, and multiple frames of target images, and is further configured to: In response to the fact that the lateral distance does not satisfy the first distance relationship and the second distance relationship, and the relative position does not satisfy the position relationship, the obstacle avoidance label in the multi-frame target image is determined to be the third avoidance label.

[0105] For example, the third obstacle avoidance label in this embodiment of the present disclosure can be "detour". That is, at the time of acquisition of multiple frames of target images, the obstacle avoidance operation of the vehicle is a detour operation. For example, in the case where neither the first obstacle avoidance label nor the second obstacle avoidance label is determined, this embodiment of the present disclosure considers the obstacle avoidance label in the multiple frames of target images to be "detour". Furthermore, whether to detour to the left or right can be determined based on the left-right relationship when the vehicle and the obstacle are longitudinally aligned.

[0106] In some embodiments of this disclosure, such as Figure 2A As shown, the processor 101 is further configured to: determine a first moment and a second moment based on the first state information and the second state information; wherein, the first moment is the moment when the longitudinal coordinates of the vehicle and the obstacle in the preset coordinate system are the same, and the second moment is the moment when the lateral movement of the vehicle begins; and determine the obstacle's avoidance label in the multi-frame target image as a third sub-avoidance label based on the first moment, the second moment, the lateral distance, the longitudinal distance, and the relative position.

[0107] For example, in this embodiment of the present disclosure, the moment when the longitudinal coordinates of the vehicle and the obstacle in the preset coordinate system are the same, and the moment when the vehicle's lateral movement begins, can be determined based on the first sub-state information and the second sub-state information. For instance, the moment when the vehicle and the obstacle are longitudinally aligned can be determined and taken as the first moment, at which time the longitudinal coordinates of the vehicle and the obstacle in the preset coordinate system are the same. As another example, the moment when the vehicle initiates its detour can be determined and taken as the second moment, at which time the vehicle is considered to have begun lateral movement, and its lateral velocity is not zero. Further, based on the first moment, the second moment, the lateral distance, the longitudinal distance, and the relative position, the obstacle's avoidance label in the multi-frame target images can be determined as a third sub-avoidance label. The third sub-avoidance label is "active detour". In this way, multi-frame target images with the obstacle avoidance label "active detour" can be further filtered out and used as training samples for the end-to-end intelligent driving model.

[0108] For example, when the obstacle avoidance label is "detour," ① if the lateral distance between the vehicle and the obstacle is always greater than the fifth preset lateral value, then the obstacle avoidance label is determined to be "passive detour." In practice, this can be achieved by determining that the vehicle's detection frame and the obstacle's detection frame never overlap laterally. In this case, the fifth preset lateral value is determined by adding half the width of the vehicle's detection frame to half the width of the obstacle's detection frame. ② If the lateral distance between the vehicle and the obstacle is always less than the fifth preset lateral value, then the obstacle avoidance label is determined to be "passive detour." In practice, this can be achieved by determining that the vehicle's detection frame and the obstacle's detection frame always overlap laterally. ③ If the time difference between the second moment (detour initiation time) and the first moment (longitudinal alignment moment, such as the moment when the vehicle just begins to pass the obstacle) is less than the first difference (e.g., 0.5 seconds) or greater than the second difference (e.g., 12 seconds), then the obstacle avoidance label is determined to be "passive detour." ④ If the longitudinal distance between the vehicle and the obstacle at the second moment is less than the third preset longitudinal value (e.g., 3 meters) or greater than the fourth preset longitudinal value (e.g., 16 meters), then the obstacle avoidance label is determined to be "passive detour". ⑤ If the relative position indicates that the obstacle is always longitudinally in front of the vehicle, then the obstacle avoidance label is determined to be "passive detour". ⑥ When the detection frame of the vehicle and the detection frame of the obstacle overlap laterally, and the angle between the speed direction of the vehicle and the obstacle is greater than a preset angle (e.g., 45 degrees), if the vehicle speed is less than the third preset speed (e.g., 1 m / s) and the obstacle speed is greater than the fourth preset speed (e.g., 1 m / s), then the obstacle avoidance label is determined to be "passive detour"; if the obstacle speed is greater than the vehicle speed, and the speed difference between the obstacle speed and the vehicle speed is greater than a preset speed difference, then the obstacle avoidance label is determined to be "passive detour"; if the obstacle's orientation is opposite to the vehicle's orientation (e.g., the obstacle is traveling in the opposite direction relative to the vehicle), then the obstacle avoidance label is determined to be "passive detour"; if the vehicle is decelerating and the deceleration is less than a preset deceleration (e.g., -0.5 m / s²), then the obstacle avoidance label is determined to be "passive detour". Furthermore, except for the above situations, all other situations are labeled as "active detour".

[0109] Figure 2B The present disclosure provides a structural diagram of an electronic device 100, which includes at least one processor 101 and a memory 102.

[0110] The processor 101 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 100 to perform desired functions.

[0111] The memory 102 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 101 may execute one or more computer program instructions to implement the tag avoidance determination method and / or other desired functions of the various embodiments of this disclosure described above.

[0112] In one example, the electronic device 100 may also include an input device 103 and an output device 104, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0113] The input device 103 may also include, for example, a keyboard, a mouse, etc.

[0114] The output device 104 can output various information to the outside, including, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0115] Of course, for the sake of simplicity, Figure 2B Only some of the components of the electronic device 100 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 100 may include any other suitable components depending on the specific application.

[0116] Exemplary methods

[0117] Figure 4 This is a flowchart illustrating a method for determining an avoidance tag according to an embodiment of the present disclosure. The method can be performed by... Figure 2A At least one processor 101 in the electronic device 100 executes the commands. For example... Figure 4 As shown, the method may include the following steps S401 to S404.

[0118] Step S401: Determine the multi-frame image data.

[0119] Step S402: Determine the first state information of the vehicle at the moment of image data acquisition, and the second state information of the obstacles in the image data at the moment of acquisition.

[0120] Step S403: Based on the first state information, the second state information, and the multi-frame image data, determine the multi-frame target images.

[0121] Step S404: Based on the first state information, the second state information, and multiple frames of target images, determine the obstacle avoidance tag. The avoidance tag is used to indicate the vehicle's avoidance operation for the obstacle.

[0122] like Figure 5 As shown, based on the embodiment shown in Figure 2 above, step S403 may include steps S4031-S4032.

[0123] Step S4031: Based on the first state information and the second state information, determine the target frame interval, which is the frame interval where there is a possibility of interaction between the vehicle and the obstacle.

[0124] Step S4032: Determine the target images based on the target frame interval and multi-frame image data.

[0125] like Figure 6 As shown above, in the above Figure 5 Based on the illustrated embodiment, step S4031 may include steps S41-S43.

[0126] Step S41: Based on the first state information and the second state information, determine the interactive keyframes from the multi-frame image data.

[0127] Step S42: Based on the interactive keyframes, determine the index information corresponding to the interactive keyframes.

[0128] Step S43: Based on the index information, determine the target frame interval, which is the frame interval where there is a possibility of interaction between the vehicle and the obstacle.

[0129] In some embodiments, step S41, determining an interactive keyframe from multiple frames of image data based on first state information and second state information, includes: determining a first image from multiple frames of image data based on first state information and second state information; wherein, at the time the first image is acquired, the longitudinal coordinates of the vehicle and the obstacle in a preset coordinate system are the same; determining a first speed of the vehicle and a second speed of the obstacle based on the first state information, the second state information, and the first image; and determining the first image as an interactive keyframe in response to the first speed being greater than or equal to the second speed.

[0130] In some embodiments, step S41, determining an interaction keyframe from multiple frames of image data based on the first state information and the second state information, further includes: in response to not determining an interaction keyframe, determining a second image from multiple frames of image data based on the first state information and the second state information; wherein, at the time the second image is acquired, the obstacle is the nearest vehicle in the same direction in the lane where the vehicle is located, and the vehicle and the obstacle are both stationary; the second image is determined as the interaction keyframe.

[0131] In some embodiments, step S41, determining an interactive keyframe from multiple frames of image data based on first state information and second state information, further includes: determining the first image as an interactive keyframe in response to the absence of an interactive keyframe and the first speed being less than the second speed.

[0132] In some embodiments, step S41, determining the interaction keyframe from the multi-frame image data based on the first state information and the second state information, further includes: in response to not determining the interaction keyframe, determining multiple third images from the multi-frame image data based on the first state information and the second state information; wherein, at the acquisition time of the third image, the lateral coordinates of the vehicle and the obstacle in the preset coordinate system are the same; and determining a fourth image based on the multiple third images, and determining the fourth image as the interaction keyframe.

[0133] In some embodiments, step S41, determining the interaction keyframe from the multi-frame image data based on the first state information and the second state information, further includes: in response to the fact that no interaction keyframe has been determined and the longitudinal coordinate of the obstacle in the preset coordinate system is greater than the longitudinal coordinate of the vehicle at the time of acquisition of the multi-frame image data, determining the longitudinal distance between the vehicle and the obstacle in the preset coordinate system at the time of acquisition of the multi-frame image data; determining the target longitudinal distance based on the multiple longitudinal distances, and determining the image corresponding to the target longitudinal distance as the interaction keyframe.

[0134] In some embodiments, the interaction keyframes between the vehicle and each obstacle in the surrounding environment can be determined according to steps 1-6 below, and the target frame range can be obtained based on the determined interaction keyframes.

[0135] Step 1: If the vehicle is longitudinally aligned with the obstacle at a certain moment. If the vehicle's speed is greater than or equal to the obstacle's speed, then the key frame for this segment of driving data interaction is time [time]. The corresponding image data. Therefore, the target frame interval is the time interval [ -10, The frame interval corresponding to +5 is within the time interval where there is a possibility that the vehicle may actively avoid obstacles.

[0136] For example, if the target frame interval is not determined through step 1 above, then step 2 below is performed.

[0137] Step 2: If the vehicle and the obstacle are both stationary, and the obstacle is the vehicle's CIPV (Constant Interference Vehicle) within the time period [T0-T1], then the keyframes for the interaction of driving data in this segment are the image data corresponding to time T0 and the image data corresponding to time T1. Furthermore, the target frame interval is the frame interval corresponding to the time interval [T0-8, T1+8], within which there is a possibility that the vehicle actively follows the obstacle without circumventing it.

[0138] For example, if the target frame interval is not determined through step 2 above, then proceed to step 3 below.

[0139] Step 3: If the vehicle is longitudinally aligned with the obstacle. If the obstacle's speed is greater than the vehicle's speed, then the keyframe for this segment of driving data interaction is time [time]. The corresponding image data. Therefore, the target frame interval is the time interval [ -17, The frame interval corresponding to +3].

[0140] For example, if the target frame interval is not determined through step 3 above, then proceed to step 4 below.

[0141] Step 4: If the vehicle and the obstacle are laterally aligned for the last time... The key frame for the interaction of this segment of driving data is time. The corresponding image data. Therefore, the target frame interval is the time interval [ -12, The frame interval corresponding to +3].

[0142] For example, if the target frame interval is not determined through step 4 above, then proceed to step 5 below.

[0143] Step 5: If the vehicle never longitudinally crosses the obstacle, determine the moment when the longitudinal distance between the vehicle and the obstacle is minimized. and will time The corresponding image data is determined to be the keyframe for the interaction of this segment of driving data. Therefore, the target frame interval is the time interval […]. -15, The frame interval corresponding to +4].

[0144] Step 6: If no interaction keyframes have been determined above, then the obstacle will not be automatically labeled.

[0145] In actual use, there are usually multiple segments of driving data. By executing the methods in steps (1) to (6) above, a target frame interval in a segment of driving data (such as the multi-frame image data in step S401) can be determined according to priority, thereby realizing the true value labeling of obstacles while simplifying the data mining process.

[0146] Of course, the priority of the above steps can be ignored, and the next step can be executed regardless of whether the target frame interval has been determined, thus determining multiple target frame intervals.

[0147] like Figure 7 As shown above, in the above Figure 4Based on the illustrated embodiment, step S404 may include steps S4041-S4042.

[0148] Step S4041: Based on the first state information and the second state information, determine the lateral distance between the vehicle and the obstacle in the preset coordinate system at the acquisition time of the multi-frame target image, as well as the relative position between the vehicle and the obstacle.

[0149] Step S4042: In response to the lateral distance satisfying the first distance relationship or the relative position satisfying the position relationship, determine the obstacle avoidance label in the multi-frame target image as the first avoidance label.

[0150] like Figure 8 As shown above, in the above Figure 7 Based on the illustrated embodiment, step S404 may further include step S4043.

[0151] Step S4043: In response to the lateral distance satisfying the second distance relationship, determine the obstacle avoidance label in the multi-frame target image as the second avoidance label.

[0152] In some embodiments, after determining that the obstacle avoidance label in the multi-frame target image is the second avoidance label, the method further includes: determining, based on the first state information and the second state information, the longitudinal distance between the vehicle and the obstacle in a preset coordinate system at the acquisition time of the multi-frame target image, as well as the third speed of the vehicle and the fourth speed of the obstacle; determining the category attribute of the obstacle; and determining, based on the category attribute, the lateral distance, the longitudinal distance, the third speed, and the fourth speed, the obstacle avoidance label in the multi-frame target image is the second sub-avoidance label.

[0153] In some embodiments, step S404 may further include: in response to the lateral distance not satisfying the first distance relationship and the second distance relationship, and the relative position not satisfying the position relationship, determining the obstacle avoidance label in the multi-frame target image as a third avoidance label.

[0154] In some embodiments, after determining that the obstacle avoidance label in the multi-frame target image is the third avoidance label, the method further includes: determining a first time and a second time based on the first state information and the second state information; wherein, the first time is the time when the longitudinal coordinates of the vehicle and the obstacle in the preset coordinate system are the same, and the second time is the start time of the lateral movement of the vehicle; and determining the obstacle avoidance label in the multi-frame target image as the third sub-avoidance label based on the first time, the second time, the lateral distance, the longitudinal distance, and the relative position.

[0155] It is understood that the specific execution methods and corresponding beneficial effects of each step in the method for determining the avoidance tag in the above embodiments have been described in detail in the corresponding embodiments of the electronic device section. Please refer to the corresponding execution methods and beneficial technical effects of the above exemplary electronic device section, and they will not be repeated here.

[0156] It should be noted that the method for determining the avoidance tag provided in this disclosure can be executed in a single processor or in parallel in a large-scale cluster (i.e., multiple processors), and this disclosure does not impose any limitations on this.

[0157] Exemplary device

[0158] Figure 9 An apparatus for determining an avoidance tag is provided in an embodiment of this disclosure, such as Figure 9 As shown, the obstacle avoidance tag determination device 900 includes a data determination module 901, a status determination module 902, an image determination module 903, and a tag determination module 904.

[0159] Data determination module 901 is used to determine multi-frame image data; The state determination module 902 is used to determine the first state information of the vehicle at the moment of image data acquisition, and the second state information of the obstacles in the image data at the moment of acquisition. The image determination module 903 is used to determine multiple target images based on first state information, second state information and multiple frame image data; The tag determination module 904 is used to determine obstacle avoidance tags based on first state information, second state information and multiple frames of target images. The avoidance tags are used to indicate the vehicle's avoidance operation for the obstacle.

[0160] In some embodiments, such as Figure 10 As shown, the image determination module 903 includes a frame interval determination unit 9031 and a target image determination unit 9032.

[0161] The frame interval determination unit 9031 is used to determine the target frame interval based on the first state information and the second state information. The target frame interval is the frame interval in which there is a possibility of interaction between the vehicle and the obstacle. The target image determination unit 9032 is used to determine multiple target images based on the target frame interval and multiple frame image data.

[0162] In some embodiments, the frame interval determination unit 9031 is configured to determine an interaction key frame from multiple frames of image data based on first state information and second state information; determine index information corresponding to the interaction key frame based on the interaction key frame; and determine a target frame interval based on the index information, wherein the target frame interval is a frame interval in which there is a possibility of interaction between the vehicle and the obstacle.

[0163] In some embodiments, the frame interval determination unit 9031 is specifically used to determine a first image from multiple frames of image data based on first state information and second state information; wherein, at the time of acquisition of the first image, the longitudinal coordinates of the vehicle and the obstacle in a preset coordinate system are the same; based on the first state information, the second state information, and the first image, a first speed of the vehicle and a second speed of the obstacle are determined; in response to the first speed being greater than or equal to the second speed, the first image is determined as an interactive key frame; based on the interactive key frame, index information corresponding to the interactive key frame is determined; based on the index information, a target frame interval is determined, wherein the target frame interval is a frame interval in which there is a possibility of interaction between the vehicle and the obstacle.

[0164] In some embodiments, the frame interval determination unit 9031 is further specifically configured to, in response to the failure to determine an interaction keyframe, determine a second image from multiple frame image data based on first state information and second state information; wherein, at the time of acquisition of the second image, the obstacle is the nearest vehicle in the same direction in the lane where the vehicle is located, and the vehicle and the obstacle are both stationary; the second image is determined as an interaction keyframe; based on the interaction keyframe, index information corresponding to the interaction keyframe is determined; based on the index information, a target frame interval is determined, the target frame interval being the frame interval in which there is a possibility of interaction between the vehicle and the obstacle.

[0165] In some embodiments, the frame interval determination unit 9031 is further configured to, in response to the absence of an interaction key frame and the first speed being less than the second speed, determine the first image as an interaction key frame; determine the index information corresponding to the interaction key frame based on the interaction key frame; and determine the target frame interval based on the index information, wherein the target frame interval is a frame interval in which there is a possibility of interaction between the vehicle and the obstacle.

[0166] In some embodiments, the frame interval determination unit 9031 is further configured to, in response to the failure to determine an interaction keyframe, determine multiple third images from multiple frame image data based on first state information and second state information; wherein, at the acquisition time of the third image, the lateral coordinates of the vehicle and the obstacle in the preset coordinate system are the same; based on the multiple third images, determine a fourth image and identify the fourth image as an interaction keyframe; based on the interaction keyframe, determine the index information corresponding to the interaction keyframe; based on the index information, determine a target frame interval, wherein the target frame interval is the frame interval in which there is a possibility of interaction between the vehicle and the obstacle.

[0167] In some embodiments, the frame interval determination unit 9031 is further configured to, in response to the absence of an interaction keyframe and the fact that the longitudinal coordinate of the obstacle in the preset coordinate system is greater than the longitudinal coordinate of the vehicle at the time of acquisition of the multi-frame image data, determine the longitudinal distance between the vehicle and the obstacle in the preset coordinate system at the time of acquisition of the multi-frame image data; determine a target longitudinal distance based on multiple longitudinal distances, and determine the image corresponding to the target longitudinal distance as an interaction keyframe; determine the index information corresponding to the interaction keyframe based on the interaction keyframe; and determine a target frame interval based on the index information, wherein the target frame interval is a frame interval in which there is a possibility of interaction between the vehicle and the obstacle.

[0168] In some embodiments, such as Figure 11 As shown, the label determination module 904 includes a spatial position relationship determination unit 9041 and a first avoidance label determination unit 9042.

[0169] The spatial position relationship determination unit 9041 is used to determine the lateral distance between the vehicle and the obstacle in the preset coordinate system and the relative position between the vehicle and the obstacle at the acquisition time of the multi-frame target image based on the first state information and the second state information. The first obstacle avoidance label determination unit 9042 is used to determine the obstacle avoidance label in the multi-frame target image as the first obstacle avoidance label in response to the lateral distance satisfying the first distance relationship or the relative position satisfying the position relationship.

[0170] In some embodiments, the label determination module 904 is configured to determine the obstacle avoidance label in a multi-frame target image as a second avoidance label in response to the lateral distance satisfying a second distance relationship.

[0171] In some embodiments, the label determination module 904 is specifically configured to, in response to the lateral distance satisfying the second distance relationship, determine the longitudinal distance between the vehicle and the obstacle in a preset coordinate system at the acquisition time of the multi-frame target image, as well as the third speed of the vehicle and the fourth speed of the obstacle, based on the first state information and the second state information; determine the category attribute of the obstacle; and determine the obstacle's avoidance label in the multi-frame target image as a second sub-avoidance label based on the category attribute, lateral distance, longitudinal distance, third speed, and fourth speed.

[0172] In some embodiments, the label determination module 904 is further configured to determine the obstacle avoidance label in the multi-frame target image as a third avoidance label in response to the lateral distance not satisfying the first distance relationship and the second distance relationship, and the relative position not satisfying the position relationship.

[0173] In some embodiments, the label determination module 904 is further specifically configured to, in response to the lateral distance not satisfying the first distance relationship and the second distance relationship, and the relative position not satisfying the position relationship, determine a first moment and a second moment based on the first state information and the second state information; wherein, the first moment is the moment when the longitudinal coordinates of the vehicle and the obstacle in the preset coordinate system are the same, and the second moment is the start time of the lateral movement of the vehicle; based on the first moment, the second moment, the lateral distance, the longitudinal distance and the relative position, determine the obstacle's avoidance label in the multi-frame target image as a third sub-avoidance label.

[0174] This disclosure further provides an obstacle avoidance tag determination device, which includes at least one processor configured to: determine multiple frames of image data; determine first state information of a vehicle at the time of image data acquisition, and second state information of an obstacle in the image data at the time of acquisition; determine multiple target images based on the first state information, the second state information, and the multiple frames of image data; and determine an obstacle avoidance tag based on the first state information, the second state information, and the multiple frames of target images, wherein the obstacle avoidance tag is used to instruct the vehicle to perform an obstacle avoidance operation.

[0175] In some embodiments, the processor determines multiple target images based on first state information, second state information, and multiple frame image data. Specifically, it is configured to: determine a target frame interval based on the first state information and the second state information, wherein the target frame interval is a frame interval in which there is a possibility of interaction between the vehicle and the obstacle; and determine multiple target images based on the target frame interval and multiple frame image data.

[0176] In some embodiments, the processor determines the target frame interval based on first state information and second state information, specifically configured to: determine interactive keyframes from multi-frame image data based on the first state information and second state information; determine index information corresponding to the interactive keyframes based on the interactive keyframes; and determine the target frame interval based on the index information.

[0177] In some embodiments, the processor determines an interaction keyframe from multiple frames of image data based on first state information and second state information. Specifically, the processor is configured to: determine a first image from multiple frames of image data based on the first state information and second state information; wherein, at the time the first image is acquired, the longitudinal coordinates of the vehicle and the obstacle in a preset coordinate system are the same; determine a first speed of the vehicle and a second speed of the obstacle based on the first state information, the second state information, and the first image; and determine the first image as an interaction keyframe in response to the first speed being greater than or equal to the second speed.

[0178] In some embodiments, the processor determines an interaction keyframe from multiple frames of image data based on first state information and second state information. It is further configured to: in response to the failure to determine an interaction keyframe, determine a second image from multiple frames of image data based on the first state information and second state information; wherein, at the time of acquisition of the second image, the obstacle is the nearest vehicle in the same direction in the lane where the vehicle is located, and the vehicle and the obstacle are both stationary; and the second image is determined as the interaction keyframe.

[0179] In some embodiments, the processor determines an interactive keyframe from multiple frames of image data based on first state information and second state information, and is further configured to: determine the first image as an interactive keyframe in response to the absence of an interactive keyframe and the first speed being less than the second speed.

[0180] In some embodiments, the processor determines an interaction keyframe from multiple frames of image data based on first state information and second state information. It is further configured to: in response to the failure to determine an interaction keyframe, determine multiple third images from the multiple frames of image data based on the first state information and second state information; wherein, at the time the third image is acquired, the lateral coordinates of the vehicle and the obstacle in a preset coordinate system are the same; and based on the multiple third images, determine a fourth image and identify the fourth image as the interaction keyframe.

[0181] In some embodiments, the processor determines an interaction keyframe from multiple frames of image data based on first state information and second state information. Specifically, it is configured to: in response to the absence of an interaction keyframe and the fact that the longitudinal coordinate of the obstacle in the preset coordinate system is greater than the longitudinal coordinate of the vehicle at the time of acquisition of the multiple frames of image data, determine the longitudinal distance between the vehicle and the obstacle in the preset coordinate system at the time of acquisition of the multiple frames of image data; determine a target longitudinal distance based on multiple longitudinal distances, and determine the image corresponding to the target longitudinal distance as the interaction keyframe.

[0182] In some embodiments, the processor determines an obstacle avoidance label based on first state information, second state information, and multiple target images. Specifically, it is configured to: determine the lateral distance between the vehicle and the obstacle in a preset coordinate system and the relative position between the vehicle and the obstacle at the acquisition time of the multiple target images based on the first state information and the second state information; and determine the obstacle avoidance label in the multiple target images as a first avoidance label in response to the lateral distance satisfying a first distance relationship or the relative position satisfying a position relationship.

[0183] In some embodiments, the processor determines the obstacle avoidance label based on the first state information, the second state information, and the multi-frame target image, and is further configured to: in response to the lateral distance satisfying the second distance relationship, determine the obstacle avoidance label in the multi-frame target image as the second avoidance label.

[0184] In some embodiments, the processor is further configured to: determine, based on first state information and second state information, the longitudinal distance between the vehicle and the obstacle in a preset coordinate system at the acquisition time of the multi-frame target image, as well as the third speed of the vehicle and the fourth speed of the obstacle; determine the category attribute of the obstacle; and determine the avoidance label of the obstacle in the multi-frame target image as a second sub-avoidance label based on the category attribute, lateral distance, longitudinal distance, third speed, and fourth speed.

[0185] In some embodiments, the processor determines the obstacle avoidance label based on the first state information, the second state information, and the multi-frame target image, and is further configured to: in response to the lateral distance not satisfying the first distance relationship and the second distance relationship, and the relative position not satisfying the position relationship, determine the obstacle avoidance label in the multi-frame target image as a third avoidance label.

[0186] In some embodiments, the processor is further configured to: determine a first moment and a second moment based on first state information and second state information; wherein the first moment is the moment when the longitudinal coordinates of the vehicle and the obstacle in the preset coordinate system are the same, and the second moment is the start time of the lateral movement of the vehicle; and determine the obstacle's avoidance label in the multi-frame target image as a third sub-avoidance label based on the first moment, the second moment, the lateral distance, the longitudinal distance, and the relative position.

[0187] It is understood that the specific methods of operation of each module and the corresponding beneficial effects of the device for determining the avoidance tag in the above embodiments have been described in detail in the corresponding embodiments of the exemplary electronic device. Please refer to the corresponding operation methods and beneficial technical effects of the exemplary electronic device, which will not be repeated here.

[0188] Exemplary computer program products and computer-readable storage media

[0189] In addition to the methods and apparatus described above, embodiments of this disclosure may also provide a computer program product, including computer program instructions that, when executed by a processor, cause the processor to perform the steps in the method for determining avoidance tags described in the various embodiments of this disclosure in the "Exemplary Methods" section above.

[0190] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. These programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0191] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the method for determining avoidance tags described in the various embodiments of this disclosure in the "Exemplary Methods" section above.

[0192] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, but is not limited to, systems, apparatuses, or devices that are electrical, magnetic, optical, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0193] The basic principles of this disclosure have been described above with reference to specific embodiments. However, the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0194] Various modifications and variations can be made to this disclosure without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. An electronic device comprising at least one processor, said processor being configured to: Determine multiple frames of image data; Determine the first state information of the vehicle at the time of image data acquisition, and the second state information of the obstacles in the image data at the time of acquisition; Based on the first state information, the second state information, and the multi-frame image data, determine the multi-frame target images; Based on the first state information, the second state information, and the multi-frame target image, an obstacle avoidance tag is determined, and the obstacle avoidance tag is used to indicate the vehicle's avoidance operation for the obstacle.

2. The electronic device according to claim 1, wherein, The processor executes the process of determining multiple target images based on the first state information, the second state information, and the multiple frame image data, specifically configured as follows: Based on the first state information and the second state information, a target frame interval is determined, which is the frame interval in which there is a possibility of interaction between the vehicle and the obstacle; Based on the target frame interval and the multi-frame image data, the multi-frame target image is determined.

3. The electronic device according to claim 2, wherein, The processor executes a process based on the first state information and the second state information to determine the target frame interval, specifically configured as follows: Based on the first state information and the second state information, interactive keyframes are determined from the multi-frame image data; Based on the interaction keyframe, determine the index information corresponding to the interaction keyframe; Based on the index information, the target frame range is determined.

4. The electronic device according to claim 3, wherein, The processor performs the task of determining interactive keyframes from the multi-frame image data based on the first state information and the second state information, specifically configured as follows: Based on the first state information and the second state information, a first image is determined from the multi-frame image data; wherein, at the time of acquisition of the first image, the longitudinal coordinates of the vehicle and the obstacle in the preset coordinate system are the same. Based on the first state information, the second state information, and the first image, the first speed of the vehicle and the second speed of the obstacle are determined. In response to the first speed being greater than or equal to the second speed, the first image is determined as the interactive keyframe.

5. The electronic device according to claim 4, wherein, The processor, based on the first state information and the second state information, determines the interaction keyframes from the multi-frame image data, and is further configured to: In response to the failure to determine the interaction keyframe, a second image is determined from the multi-frame image data based on the first state information and the second state information; wherein, at the time of acquisition of the second image, the obstacle is the nearest vehicle in the same direction in the lane where the vehicle is located, and the vehicle and the obstacle are both stationary. The second image is identified as the interaction keyframe.

6. The electronic device according to claim 5, wherein, The processor, based on the first state information and the second state information, determines the interaction keyframes from the multi-frame image data, and is further configured to: In response to the failure to determine the interaction keyframe and the first speed being less than the second speed, the first image is determined as the interaction keyframe.

7. The electronic device according to claim 5, wherein, The processor, based on the first state information and the second state information, determines the interaction keyframes from the multi-frame image data, and is further configured to: In response to the failure to determine the interaction keyframe, based on the first state information and the second state information, multiple third images are determined from the multiple image data; wherein, at the time of acquisition of the third image, the lateral coordinates of the vehicle and the obstacle in the preset coordinate system are the same. Based on the multiple frames of the third image, a fourth image is determined, and the fourth image is identified as the interaction keyframe.

8. The electronic device according to claim 5, wherein, The processor, based on the first state information and the second state information, determines the interaction keyframes from the multi-frame image data, and is further configured to: In response to the failure to determine the interaction keyframe and the fact that the longitudinal coordinate of the obstacle in the preset coordinate system is greater than the longitudinal coordinate of the vehicle at the time of acquisition of the multi-frame image data, the longitudinal distance between the vehicle and the obstacle in the preset coordinate system at the time of acquisition of the multi-frame image data is determined. Based on the multiple longitudinal distances, the target longitudinal distance is determined, and the image corresponding to the target longitudinal distance is determined as the interactive keyframe.

9. The electronic device according to claim 1, wherein, The processor executes the process of determining the obstacle avoidance label based on the first state information, the second state information, and the multi-frame target image, specifically configured as follows: Based on the first state information and the second state information, the lateral distance between the vehicle and the obstacle in the preset coordinate system and the relative position between the vehicle and the obstacle are determined at the acquisition time of the multi-frame target images. In response to the lateral distance satisfying a first distance relationship or the relative position satisfying a positional relationship, the obstacle's avoidance label in the multi-frame target image is determined as a first avoidance label.

10. The electronic device according to claim 9, wherein, The processor executes the process of determining the obstacle avoidance label based on the first state information, the second state information, and the multi-frame target image, and is further specifically configured to: In response to the lateral distance satisfying the second distance relationship, the obstacle avoidance label in the multi-frame target image is determined as the second avoidance label.

11. The electronic device of claim 10, wherein the processor is further configured to: Based on the first state information and the second state information, the longitudinal distance between the vehicle and the obstacle in the preset coordinate system at the acquisition time of the multi-frame target images is determined, as well as the third speed of the vehicle and the fourth speed of the obstacle. Determine the category attributes of the obstacle; Based on the category attribute, the lateral distance, the longitudinal distance, the third velocity, and the fourth velocity, the obstacle avoidance label in the multi-frame target image is determined as the second sub-avoidance label.

12. The electronic device according to claim 10, wherein, The processor executes the process of determining the obstacle avoidance label based on the first state information, the second state information, and the multi-frame target image, and is further specifically configured to: In response to the fact that the lateral distance does not satisfy the first distance relationship and the second distance relationship, and the relative position does not satisfy the position relationship, the obstacle avoidance label in the multi-frame target image is determined to be the third avoidance label.

13. The electronic device of claim 12, wherein the processor is further configured to: Based on the first state information and the second state information, the first time point and the second time point are determined; wherein, The first moment is the moment when the longitudinal coordinates of the vehicle and the obstacle in the preset coordinate system are the same, and the second moment is the start time of the lateral movement of the vehicle. Based on the first time point, the second time point, the lateral distance, the longitudinal distance, and the relative position, the obstacle avoidance label in the multi-frame target image is determined as the third sub-avoidance label.

14. A method for determining an avoidance tag, the method comprising: Determine multiple frames of image data; Determine the first state information of the vehicle at the time of image data acquisition, and the second state information of the obstacles in the image data at the time of acquisition; Based on the first state information, the second state information, and the multi-frame image data, determine the multi-frame target images; Based on the first state information, the second state information, and the multi-frame target image, an obstacle avoidance tag is determined, and the obstacle avoidance tag is used to indicate the vehicle's avoidance operation for the obstacle.

15. The method according to claim 14, wherein, The step of determining multiple target images based on the first state information, the second state information, and the multiple frame image data includes: Based on the first state information and the second state information, a target frame interval is determined, which is the frame interval in which there is a possibility of interaction between the vehicle and the obstacle; Based on the target frame interval and the multi-frame image data, the multi-frame target image is determined.

16. The method according to claim 15, wherein, Determining the target frame interval based on the first state information and the second state information includes: Based on the first state information and the second state information, interactive keyframes are determined from the multi-frame image data; Based on the interaction keyframe, determine the index information corresponding to the interaction keyframe; Based on the index information, the target frame range is determined.

17. The method according to claim 14, wherein, The step of determining the obstacle avoidance label based on the first state information, the second state information, and the multi-frame target image includes: Based on the first state information and the second state information, the lateral distance between the vehicle and the obstacle in the preset coordinate system and the relative position between the vehicle and the obstacle are determined at the acquisition time of the multi-frame target images. In response to the lateral distance satisfying a first distance relationship or the relative position satisfying a positional relationship, the obstacle's avoidance label in the multi-frame target image is determined as a first avoidance label.

18. The method according to claim 17, wherein, The step of determining the obstacle avoidance label based on the first state information, the second state information, and the multi-frame target image further includes: In response to the lateral distance satisfying the second distance relationship, the obstacle avoidance label in the multi-frame target image is determined as the second avoidance label.

19. A device for determining an avoidance tag, comprising: The data determination module is used to determine multi-frame image data; The state determination module is used to determine the first state information of the vehicle at the time of image data acquisition, and the second state information of the obstacles in the image data at the time of acquisition. The image determination module is used to determine multiple target images based on the first state information, the second state information, and the multiple frame image data; The tag determination module is used to determine the obstacle avoidance tag based on the first state information, the second state information and the multi-frame target image, and the avoidance tag is used to indicate the vehicle's avoidance operation for the obstacle.

20. A computer-readable storage medium storing a computer program for performing the method for determining an avoidance tag as described in any one of claims 14-18.

Images