A method, apparatus, electronic device, and storage medium for determining prompt information.

Abstract

This disclosure relates to a method, apparatus, electronic device, and storage medium for determining prompt information. The method includes: acquiring road images captured by an image acquisition device; identifying the detection frame of a target vehicle using a first frame of road images captured by the image acquisition device as a detection image within a preset detection period; using other frames of road images captured by the image acquisition device within the detection period as tracking images; for each tracking image frame, determining the detection frame and wheel grounding parameters of the target vehicle in that tracking image frame based on the previously identified detection frame of the target vehicle and the tracking image itself; and determining whether to remind the guiding vehicle to pay attention to the target vehicle based on the ground equation of the road ahead of the guiding vehicle determined within the detection period, the detection frame of the target vehicle corresponding to the tracking image within the detection period, and the wheel grounding parameters of the target vehicle, thereby reducing the possibility of accidents and assisting the driver.

Description

Technical Field

[0001] This disclosure relates to the field of intelligent driving technology, specifically to a method, apparatus, electronic device, and non-transitory computer-readable storage medium for determining prompt information. Background Technology

[0002] In the field of intelligent driving technology, whether it is autonomous driving technology or driver assistance technology, timely detection of the behavior of other vehicles in the surrounding environment (such as cutting in or changing lanes) and determination of whether such behavior affects the driving of the vehicle, and then taking appropriate avoidance measures, are important links to ensure the safe driving of the vehicle and reduce traffic accidents.

[0003] To detect the behavior of other vehicles in the surrounding environment, it is first necessary to detect other vehicles in the surrounding environment. Currently, the detection of other vehicles in the surrounding environment is achieved by acquiring images of the surrounding environment using the vehicle's image sensors, and then using target detection algorithms to determine the category of the target in the image and the two-dimensional detection box information of the target. The two-dimensional detection box information can represent the position information of the target in the image.

[0004] However, even if the vehicle and its location are detected in the image, it is impossible to further detect the behavior of the vehicle in the image, and therefore it is impossible to determine whether the behavior of the vehicle in the image affects the driving of the vehicle itself.

[0005] The above description of the problem discovery process is only for the purpose of assisting in understanding the technical solution of this disclosure, and does not represent an admission that the above content is prior art. Summary of the Invention

[0006] To address at least one problem existing in the prior art, at least one embodiment of this disclosure provides a method, apparatus, electronic device, and non-transitory computer-readable storage medium for determining prompt information.

[0007] In a first aspect, embodiments of this disclosure provide a method for determining prompt information, the method comprising:

[0008] Acquire road images of the road ahead of the guide vehicle, captured by an image acquisition device mounted on the guide vehicle;

[0009] Within a preset detection period, the first frame of road image captured by the image acquisition device is used as the detection image, and the detection frame of at least one target vehicle located around the guide vehicle is identified from the detection image.

[0010] Other frames of road images captured by the image acquisition device within the detection period are used as tracking images. For each frame of tracking image, based on the detection box of the target vehicle previously identified in the tracking image and the tracking image itself, the detection box of the target vehicle in the tracking image and the wheel grounding wire parameters of the target vehicle are determined.

[0011] Based on the ground equation of the road ahead of the guiding vehicle determined during the detection period, the detection frame of the target vehicle corresponding to the tracking image during the detection period, and the wheel grounding wire parameters of the target vehicle, it is determined whether to remind the guiding vehicle to pay attention to the target vehicle.

[0012] Secondly, embodiments of this disclosure also provide an apparatus for determining prompt information, the apparatus comprising:

[0013] The acquisition unit is used to acquire road images of the road in front of the guide vehicle captured by the image acquisition device mounted on the guide vehicle;

[0014] The vehicle detection unit is used to identify the detection frame of at least one target vehicle located around the guide vehicle from the first frame of road image captured by the image acquisition device as the detection image within a preset detection period.

[0015] The vehicle tracking unit is used to take other frames of road images captured by the image acquisition device during the detection period as tracking images. For each frame of tracking image, based on the detection frame of the target vehicle previously identified in the tracking image and the tracking image itself, the unit determines the detection frame of the target vehicle in the tracking image and the wheel grounding wire parameters of the target vehicle.

[0016] The prompting unit is used to determine whether to remind the guiding vehicle to pay attention to the target vehicle based on the ground equation of the road ahead of the guiding vehicle determined within the detection period, the detection frame of the target vehicle corresponding to the tracking image within the detection period, and the wheel grounding wire parameters of the target vehicle.

[0017] Thirdly, embodiments of this disclosure also provide an electronic device, including: a processor and a memory; the processor executes the steps of the method as described in the first aspect by invoking a program or instructions stored in the memory.

[0018] Fourthly, embodiments of this disclosure also provide a non-transitory computer-readable storage medium for storing a program or instructions that cause a computer to perform the steps of the method as described in the first aspect.

[0019] Fifthly, embodiments of this disclosure also provide a computer program product, wherein the computer program product includes a computer program stored in a non-transitory computer-readable storage medium, and at least one processor of a computer reads from the storage medium and executes the computer program, causing the computer to perform the steps of the method as described in the first aspect.

[0020] As can be seen, in at least one embodiment of this disclosure, by acquiring a road image of the road ahead of the guiding vehicle, and performing target vehicle detection on the road image as a detection image, a detection box of the target vehicle can be obtained; by performing target vehicle tracking on the road image as a tracking image, a detection box of the target vehicle in the tracking image and the wheel ground wire parameters of the target vehicle can be obtained; and then, based on the detection box of the target vehicle, the wheel ground wire parameters, and the ground equation of the road ahead of the guiding vehicle, it can be determined whether to remind the guiding vehicle to pay attention to the target vehicle, thereby reducing the possibility of accidents and achieving the purpose of assisting the driver in driving. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings.

[0022] Figure 1 This is an exemplary architecture diagram of a vehicle guidance system provided in this disclosure embodiment;

[0023] Figure 2 This is an exemplary block diagram of an intelligent driving system provided in an embodiment of this disclosure;

[0024] Figure 3 This is an exemplary block diagram of a device for determining prompt information provided in an embodiment of this disclosure;

[0025] Figure 4 This is an exemplary block diagram of an electronic device provided in an embodiment of this disclosure;

[0026] Figure 5 This is an exemplary flowchart of a method for determining prompt information provided in an embodiment of this disclosure;

[0027] Figure 6 This is an exemplary application scenario diagram provided by an embodiment of this disclosure;

[0028] Figure 7 yes Figure 6 The diagram shows a projection of the wheel grounding wire in the application scenario. Detailed Implementation

[0029] To better understand the above-described objectives, features, and advantages of this disclosure, the present disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. It is to be understood that the described embodiments are only some, not all, of the embodiments of this disclosure. The specific embodiments described herein are merely for explaining this disclosure and are not intended to limit it. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure are within the scope of protection of this disclosure.

[0030] It should be noted that in this article, relational terms such as “first” and “second” are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0031] This disclosure provides a method, apparatus, electronic device, or non-transitory computer-readable storage medium for determining prompt information. By acquiring a road image of the road ahead of a guiding vehicle, and performing target vehicle detection on the road image as a detection image, a detection box of the target vehicle can be obtained. By tracking the road image as a tracking image, a detection box of the target vehicle in the tracking image and the wheel ground contact parameters of the target vehicle can be obtained. Then, based on the detection box of the target vehicle, the wheel ground contact parameters, and the ground equation of the road ahead of the guiding vehicle, it can be determined whether to remind the guiding vehicle to pay attention to the target vehicle, thereby reducing the possibility of accidents and achieving the purpose of assisting the driver.

[0032] The guide vehicle mentioned in this disclosure can be an intelligent driving vehicle, which is a vehicle equipped with different levels of intelligent driving systems. Intelligent driving systems include, for example, unmanned driving systems, assisted driving systems, driving assistance systems, highly automated driving systems, fully automated driving vehicles, etc.

[0033] This disclosure can be applied to electronic devices equipped with intelligent driving systems. In some embodiments, the electronic device may be an electronic device mounted on a guided vehicle. In some embodiments, the electronic device may be a non-vehicle-mounted electronic device. For example, the electronic device can be used to test intelligent driving algorithms.

[0034] This disclosure can be applied to different scenarios. For example, in AR (Augmented Reality) navigation scenarios, it can visually alert the driver to the target vehicle. It should be noted that the application scenarios of this disclosure are merely some examples or embodiments of this disclosure. Those skilled in the art can apply this disclosure to other similar situations without any creative effort.

[0035] To provide a clearer and more accurate explanation, this disclosure uses a vehicle guidance example to illustrate the method, apparatus, electronic device, or non-transitory computer-readable storage medium for determining the prompt information.

[0036] Figure 1 This is an exemplary overall architecture diagram of a guided vehicle provided in an embodiment of this disclosure. Figure 1 As shown, the vehicle guidance system includes: a sensor array, an intelligent driving system 100, a vehicle-level execution system, and other components that can be used to drive and control the vehicle, such as a brake pedal, a steering wheel, and an accelerator pedal.

[0037] A sensor array is used to collect data on the vehicle's external environment and detect the vehicle's position. The sensor array includes, but is not limited to, at least one of image acquisition devices (e.g., cameras), lidar, millimeter-wave radar, ultrasonic radar, GPS (Global Positioning System), and IMU (Inertial Measurement Unit).

[0038] In some embodiments, the sensor group is also used to collect vehicle dynamics data, and the sensor group includes, for example, at least one of wheel speed sensor, speed sensor, acceleration sensor, steering wheel angle sensor, and front wheel angle sensor.

[0039] The intelligent driving system 100 is used to acquire sensing data from a sensor array, wherein the sensing data includes, but is not limited to, images, videos, laser point clouds, millimeter waves, GPS information, vehicle status, etc. In some embodiments, the intelligent driving system 100 performs environmental perception and vehicle localization based on the sensing data, generating perception information and vehicle pose; the intelligent driving system 100 performs planning and decision-making based on the perception information and vehicle pose, generating planning and decision information; the intelligent driving system 100 generates vehicle control commands based on the planning and decision information and sends them to the vehicle's underlying execution system.

[0040] In some embodiments, the intelligent driving system 100 can be a software system, a hardware system, or a combination of software and hardware. For example, the intelligent driving system 100 is a software system running on an operating system, and the vehicle hardware system is a hardware system that supports the operation of the operating system.

[0041] In some embodiments, the intelligent driving system 100 can interact with a cloud server. In some embodiments, the intelligent driving system 100 interacts with the cloud server via a wireless communication network (e.g., including but not limited to GPRS network, Zigbee network, Wifi network, 3G network, 4G network, 5G network, etc.).

[0042] In some embodiments, a cloud server is used to interact with the vehicle. The cloud server can send environmental information, location information, control information, and other information required for intelligent driving to the vehicle. In some embodiments, the cloud server can receive sensor data, vehicle status information, vehicle driving information, and information related to vehicle requests from the vehicle. In some embodiments, the cloud server can remotely control the vehicle based on user settings or vehicle requests. In some embodiments, the cloud server can be a single server or a group of servers. The server group can be centralized or distributed. In some embodiments, the cloud server can be local or remote.

[0043] A vehicle-level execution system is used to receive vehicle control commands and control the vehicle's movement based on the vehicle control commands. In some embodiments, the vehicle-level execution system includes, but is not limited to, a steering system, a braking system, and a drive system. In some embodiments, the vehicle-level execution system may further include a low-level controller, which can parse the vehicle control commands and distribute them to corresponding systems such as the steering system, braking system, and drive system.

[0044] In some embodiments, the guide vehicle may also include Figure 1 The vehicle CAN bus (not shown) connects to the vehicle's underlying execution system. Information exchange between the intelligent driving system 100 and the vehicle's underlying execution system is transmitted via the vehicle CAN bus.

[0045] Figure 2 This is an exemplary block diagram of an intelligent driving system 200 provided in an embodiment of the present disclosure. In some embodiments, the intelligent driving system 200 can be implemented as follows: Figure 1 The intelligent driving system 100 or a part of the intelligent driving system 100 is used to control the driving of the vehicle.

[0046] like Figure 2 As shown, the intelligent driving system 200 can be divided into multiple modules, such as: perception module 201, planning module 202, control module 203, prompting module 204, and other modules that can be used for intelligent driving.

[0047] The perception module 201 is used for environmental perception and localization. In some embodiments, the perception module 201 is used to acquire sensor data, V2X (Vehicle to X) data, high-precision maps, and other data, and perform environmental perception and localization based on at least one of the above data to generate perception information and localization information. The perception information may include, but is not limited to, at least one of the following: obstacle information, road signs / markers, pedestrian / vehicle information, and drivable area. The localization information includes vehicle pose.

[0048] The planning module 202 is used for path planning and decision-making. In some embodiments, the planning module 202 generates planning and decision-making information based on the perception information and positioning information generated by the perception module 201. In some embodiments, the planning module 202 can also combine at least one of V2X data, high-precision maps, and other data to generate planning and decision-making information. The planning information may include, but is not limited to, planned paths; the decision-making information may include, but is not limited to, at least one of the following: behaviors (e.g., including but not limited to following, overtaking, stopping, detouring, etc.), vehicle heading, vehicle speed, desired vehicle acceleration, desired steering wheel angle, etc.

[0049] The control module 203 is used to generate control commands for the vehicle's underlying execution system based on planning and decision-making information, and to issue these control commands so that the vehicle's underlying execution system can control the vehicle's movement. These control commands may include, but are not limited to, steering wheel commands, lateral control commands, and longitudinal control commands.

[0050] The prompting module 204 is used to detect the target vehicle in the road image used as the detection image and obtain the detection box of the target vehicle; to track the target vehicle in the road image used as the tracking image and obtain the detection box of the target vehicle in the tracking image and the wheel ground line parameters of the target vehicle; and then, based on the detection box of the target vehicle, the wheel ground line parameters and the ground equation of the road ahead of the guiding vehicle, it determines whether to remind the guiding vehicle to pay attention to the target vehicle, thereby reducing the possibility of accidents and achieving the purpose of assisting the driver.

[0051] In some embodiments, the function of the prompting module 204 can be integrated into the perception module 201, the planning module 202, or the control module 203, or it can be configured as a module independent of the intelligent driving system 200. The prompting module 204 can be a software module, a hardware module, or a combination of software and hardware. For example, the prompting module 204 is a software module running on an operating system, and the vehicle hardware system is a hardware system that supports the operation of the operating system.

[0052] Figure 3 This is an exemplary block diagram of a device 300 for determining prompt information provided in an embodiment of the present disclosure. In some embodiments, the device 300 for determining prompt information may be implemented as follows: Figure 2The prompt module 204 or a part of the prompt module 204.

[0053] like Figure 3 As shown, the device 300 for determining the prompt information may include, but is not limited to, the following units: acquisition unit 301, vehicle detection unit 302, vehicle tracking unit 303, and prompt determination unit 304.

[0054] Acquisition Unit 301

[0055] The acquisition unit 301 is used to acquire environmental information in front of the vehicle. In some embodiments, the acquisition unit 301 acquires a road image of the road in front of the guide vehicle captured by an image acquisition device mounted on the guide vehicle.

[0056] It is understandable that a road image includes not only the road itself, but also environmental information such as vehicles moving or parked on the road, pedestrians, buildings on both sides of the road, and road signs. A road image can be understood as an environmental image guiding vehicles forward, which includes road information.

[0057] In some embodiments, the image acquisition frame rate of the image acquisition device is 20 frames per second, that is, the image acquisition device can capture 20 frames of road images per second. It should be understood that this embodiment is only an example and does not limit the specific value of the image acquisition frame rate. Those skilled in the art can set it according to actual needs.

[0058] In some embodiments, the image acquisition device may be a commercially available camera. The type and model of the camera can be selected according to actual needs, and this embodiment does not limit it.

[0059] Vehicle detection unit 302

[0060] The vehicle detection unit 302 is used to detect target vehicles in a road image, which is a detection image, and obtain the detection frame of the target vehicle. In some embodiments, the vehicle detection unit 302 uses a first frame of road image captured by an image acquisition device as the detection image within a preset detection period, and identifies the detection frame of at least one target vehicle located around the guide vehicle from the detection image.

[0061] In some embodiments, the preset detection period can be 500 milliseconds, that is, the image detection frame rate is 2 frames per second, that is, there are 2 road images per second as detection images. It is understood that this embodiment is only an example and does not limit the specific value of the detection period. Those skilled in the art can set it according to actual needs.

[0062] In some embodiments, the vehicle detection unit 302 can input a detection image into a target detection network to obtain a target information set output by the target detection network. Each target information set includes a target type and a target detection box. The vehicle detection unit 302 can filter target information of type "vehicle" from the target information set to obtain a detection box for at least one target vehicle around the guided vehicle.

[0063] The object detection network takes an image as input and outputs the types of different objects in the image and their bounding boxes. The bounding boxes are two-dimensional boxes that represent the location of the objects in the image.

[0064] In some embodiments, the object detection network can be trained to directly output detection boxes for target vehicles; that is, the input to the object detection network is the detection image, and the output is the detection boxes for all vehicles in the detection image. In this embodiment, the detection boxes include full vehicle boxes, rear vehicle boxes, and front vehicle boxes, wherein the full vehicle box is a rectangular box that can encompass the entire target vehicle; the rear vehicle box is a rectangular box that can encompass the rear of the target vehicle; and the front vehicle box is a rectangular box that can encompass the front of the target vehicle. In some embodiments, the object detection network is trained using the deep learning object detection toolkit mmdetection.

[0065] In some embodiments, the object detection network can be different networks, such as the SSD (Single Shot MultiBox Detector) network, or other object detection networks in the field of deep learning.

[0066] Vehicle tracking unit 303

[0067] The vehicle tracking unit 303 is used to track target vehicles on road images used as tracking images, obtaining the detection box of the target vehicle in the tracking image and the wheel ground contact line parameters of the target vehicle. The wheel ground contact line is the line connecting the wheel to the ground contact point. In some embodiments, the vehicle tracking unit 303 uses other frames of road images captured by the image acquisition device within the detection period as tracking images. For each frame of tracking image, based on the detection box of the target vehicle previously identified in that frame of tracking image and the tracking image itself, it determines the detection box of the target vehicle in that frame of tracking image and the wheel ground contact line parameters of the target vehicle. The wheel ground contact line parameters can be understood as the slope and intercept of the wheel ground contact line.

[0068] In some embodiments, if the image acquisition frame rate of the image acquisition device is 20 frames per second and the image detection frame rate of the vehicle detection unit 302 is 2 frames per second, then there are a total of 9 road images used as tracking images in one detection cycle. That is, the vehicle tracking unit 303 can track the target vehicle by 18 road images per second.

[0069] In some embodiments, detection or tracking is performed in real time. For example, the image acquisition device performs detection or tracking once for each frame of road image captured. That is, if the road image frame is used as a detection image, the vehicle detection unit 302 detects the target vehicle in the road image frame; if the road image frame is used as a tracking image, the vehicle tracking unit 303 tracks the target vehicle in the road image frame. The vehicle tracking unit 303 determines the detection frame of the target vehicle in the road image frame and the wheel grounding wire parameters of the target vehicle based on the detection frame of the target vehicle previously identified in the road image frame and the road image frame itself.

[0070] In some embodiments, detection and tracking are performed in batches. For example, the image acquisition device captures multiple frames of road images within a preset detection period, and performs batch detection and tracking on these multiple frames within the detection period. That is, the vehicle detection unit 302 uses the first frame of the road image within the detection period as the detection image to detect the target vehicle; the vehicle tracking unit 303 uses the other frames of the road image within the detection period as tracking images to track the target vehicle. Specifically, for each tracking image frame, the vehicle tracking unit 303 determines the detection frame of the target vehicle in that tracking image frame and the wheel grounding wire parameters of the target vehicle based on the detection frame of the target vehicle previously identified in that tracking image frame and the tracking image frame itself.

[0071] In some embodiments, the vehicle tracking unit 303 may determine a set of cropping boxes corresponding to the detection image based on the detection boxes of at least one target vehicle in the detection image. In some embodiments, the vehicle tracking unit 303 may, for any frame of tracking image (excluding the next frame of tracking image after the detection image), determine a set of cropping boxes corresponding to the frame of tracking image based on the set of detection boxes corresponding to the first frame of detection image preceding the frame of tracking image (there are no detection images between the first frame of detection image and the frame of tracking image), and based on the set of detection boxes corresponding to all tracking images between the frame of tracking image and the first frame of detection image.

[0072] In some embodiments, the vehicle tracking unit 303 determines the center position of any detection box in the detection image as the center position of the clipping box. In some embodiments, the vehicle tracking unit 303 may, for any frame of tracking image, determine the center position of the clipping box corresponding to the frame of tracking image based on the center position of the detection box of any target vehicle in the first frame of detection image preceding the frame of tracking image (where there is no detection image between the first frame of detection image and the frame of tracking image), and based on the center positions of the detection boxes of the same target vehicle in all tracking images between the frame of tracking image and the first frame of detection image.

[0073] For example, within a detection period, the center position of the cropping box in the second frame of the road image (as a tracking image) is the center position of the detection box of the target vehicle in the first frame of the road image (as a detection image); the center position of the cropping box in the third frame of the road image (as a tracking image) is predicted based on the center positions of the detection boxes of the same target vehicle in the first and second frames; the center position of the cropping box in the fourth frame of the road image (as a tracking image) is predicted based on the center positions of the detection boxes of the same target vehicle in the first, second, and third frames.

[0074] One possible prediction method is to connect the center positions of the detection boxes of the target vehicle in different frame images and make predictions on the extension of the connecting lines.

[0075] In some embodiments, the vehicle tracking unit 303 expands the length and width of any detection box in the detection image by a preset multiple to obtain a cropping box, which is used to crop the next frame of the tracking image of the detection image. Expanding the length and width of the detection box by a preset multiple is intended to ensure that the cropped image obtained by cropping the next frame of the tracking image of the detection image includes the target vehicle, facilitating target vehicle tracking while reducing data volume.

[0076] In some embodiments, for any frame of tracking image, the vehicle tracking unit 303 expands the length and width of the detection box of the same target vehicle in the previous frame of tracking image by a preset multiple to obtain the length and width of the clipping box corresponding to the detection box in the current frame of tracking image.

[0077] In some embodiments, the preset multiplier is 1.2, 1.5, or 1.6. It is understood that this embodiment is only illustrative and does not limit the specific value of the preset multiplier. Those skilled in the art can set it according to actual needs. It is understood that the processing of any frame of detected image by the vehicle tracking unit 303 is similar and will not be described in detail here.

[0078] In some embodiments, the vehicle tracking unit 303 may determine a set of cropped images corresponding to each frame of tracking image based on a set of cropped boxes corresponding to the detected image. In some embodiments, the vehicle tracking unit 303 may determine a set of cropped images corresponding to the next frame of tracking image of the detected image based on a set of cropped boxes corresponding to the detected image. In some embodiments, the vehicle tracking unit 303 may, for any frame of tracking image (excluding the next frame of tracking image of the detected image), determine a set of cropped boxes corresponding to the frame of tracking image based on a set of detection boxes corresponding to a first frame of detection image preceding the frame of tracking image (there are no detection images between the first frame of detection image and the frame of tracking image), and a set of detection boxes corresponding to all tracking images between the frame of tracking image and the first frame of detection image; and then crop the frame of tracking image based on the set of cropped boxes corresponding to the frame of tracking image to obtain a set of cropped images corresponding to the frame of tracking image.

[0079] In some embodiments, the vehicle tracking unit 303 can input the set of cropped images corresponding to each frame of the tracking image into the vehicle tracking network to obtain the detection box of the target vehicle in each frame of the tracking image and the wheel grounding line parameters of the target vehicle output by the vehicle tracking network. The wheel grounding line parameters are the slope and intercept of the wheel grounding line in the cropped image.

[0080] In some embodiments, the vehicle tracking network may employ a CNN (Convolutional Neural Networks) regression network or any tracking network from computer vision. In some embodiments, the vehicle tracking network may employ a deep neural network inference framework (MNN).

[0081] Confirmation prompt unit 304

[0082] The determination and prompting unit 304 is used to determine whether to remind the guiding vehicle to pay attention to the target vehicle, thereby reducing the possibility of accidents and assisting the driver. In some embodiments, the determination and prompting unit 304 can determine whether the target vehicle is a dangerous vehicle based on the ground equation of the road ahead of the guiding vehicle determined within the detection period, the detection frame of the target vehicle corresponding to the tracking image within the detection period, and the wheel ground wire parameters of the target vehicle. If it is a dangerous vehicle, it determines to remind the guiding vehicle to pay attention to the target vehicle. A dangerous vehicle is a vehicle that affects the driving behavior of the guiding vehicle and is likely to cause a traffic accident, such as vehicles changing lanes, vehicles cutting in, and vehicles encountered when the guiding vehicle is turning, driving to the left front, or driving to the right front.

[0083] For example, if the image acquisition device has an image acquisition frame rate of 20 frames per second and the vehicle detection unit 302 has an image detection frame rate of 2 frames per second, then in one detection cycle, there is 1 frame of road image used as the detection image and a total of 9 frames of road image used as the tracking image. The prompting unit 304 combines the detection and tracking results of these 10 frames to determine whether to remind the guiding vehicle to pay attention to the target vehicle.

[0084] In some embodiments, the determining prompting unit 304 can obtain the ground equation of the road ahead guiding the vehicle within the detection period through IPM (Inverse Perspective Mapping).

[0085] In some embodiments, the determining prompt unit 304 can determine the position and orientation of the target vehicle based on the detection frame of the target vehicle corresponding to the tracking image within the detection period and the wheel grounding wire parameters of the target vehicle. In some embodiments, the determining prompt unit 304 can transform the wheel grounding wire parameters from the image coordinate system to the world coordinate system based on the intrinsic and extrinsic parameters of the image sensor to obtain the orientation of the target vehicle. In some embodiments, the determining prompt unit 304 can transform the detection frame of the target vehicle corresponding to the tracking image within the detection period from the image coordinate system to the world coordinate system based on the intrinsic and extrinsic parameters of the image sensor to obtain the position of the target vehicle.

[0086] In some embodiments, the determining prompting unit 304 may determine whether to remind the guiding vehicle to pay attention to the target vehicle based on the ground equation of the road ahead of the guiding vehicle determined within the detection period, the wheel ground wire parameters of the target vehicle corresponding to the tracking image within the detection period, the position and orientation of the target vehicle, and the position and orientation of the vehicle.

[0087] In some embodiments, for any wheel grounding parameter, if the wheel grounding parameter is not parallel to the ground equation in the world coordinate system, the prompting unit 304 predicts the driving trajectory of the target vehicle based on the position and orientation of the target vehicle corresponding to the wheel grounding parameter; and predicts the trajectory of the vehicle based on the position and orientation of the vehicle itself; and then determines whether to remind the guiding vehicle to pay attention to the target vehicle based on the driving trajectory of the target vehicle and the driving trajectory of the vehicle itself.

[0088] Based on the driving trajectory of the target vehicle and the driving trajectory of the vehicle itself, the prompting unit 304 can determine whether the target vehicle is a vehicle that affects the driving behavior of the vehicle itself (the guiding vehicle) and is likely to cause a traffic accident. For example, this includes vehicles that change lanes, vehicles that cut in front of the vehicle, and vehicles encountered when the vehicle is turning, driving to the left front, or driving to the right front.

[0089] In some embodiments, after determining that the reminder unit 304 should pay attention to the target vehicle, it should provide a reminder in a visual, auditory, or combined manner so that the driver can be informed of the reminder in a timely manner.

[0090] In some embodiments, the division of units in the device 300 for determining prompt information is only a logical functional division. In actual implementation, there may be other division methods. For example, at least two units among the acquisition unit 301, vehicle detection unit 302, vehicle tracking unit 303, and prompt determination unit 304 can be implemented as one unit; the acquisition unit 301, vehicle detection unit 302, vehicle tracking unit 303, or prompt determination unit 304 can also be divided into multiple sub-units. It is understood that each unit or sub-unit can be implemented using electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application.

[0091] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. The electronic device can support the operation of an intelligent driving system. The electronic device can be an in-vehicle device for guiding the vehicle, or it can be a non-in-vehicle device.

[0092] like Figure 4 As shown, the electronic device includes at least one processor 401, at least one memory 402, and at least one communication interface 403. The various components of the electronic device are coupled together via a bus system 404. The communication interface 403 is used for information transmission with external devices. Understandably, the bus system 404 is used to implement communication between these components. In addition to a data bus, the bus system 404 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 4 The general designated all buses as Bus System 404.

[0093] It is understood that the memory 402 in this embodiment can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory.

[0094] In some implementations, memory 402 stores elements such as executable units or data structures, or subsets thereof, or extended sets thereof: operating systems and applications.

[0095] The operating system includes various system programs, such as the framework layer, core library layer, and driver layer, used to implement various basic tasks and handle hardware-based tasks. Application programs include various applications, such as media players and browsers, used to implement various application tasks. The program implementing the method for determining prompt information provided in the embodiments of this disclosure can be included in the application program.

[0096] In this embodiment of the disclosure, the processor 401 executes the steps of the method for determining prompt information provided in this embodiment of the disclosure by calling the program or instructions stored in the memory 402, specifically, the program or instructions stored in the application program.

[0097] The method for determining prompt information provided in this disclosure can be applied to, or implemented by, processor 401. Processor 401 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the hardware of processor 401 or by instructions in software form. Processor 401 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor.

[0098] The steps of the method for determining the prompt information provided in this disclosure can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software units in the decoding processor. The software units can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 402, and processor 401 reads the information in memory 402 and combines it with its hardware to complete the steps of the method.

[0099] Figure 5 This is an exemplary flowchart illustrating a method for determining a prompt message according to an embodiment of this disclosure. The method is executed by an electronic device; in some embodiments, the execution entity may also be an intelligent driving system supported by the electronic device. For ease of description, the following embodiments use an electronic device as the execution entity to illustrate the flowchart of the method for determining the prompt message.

[0100] like Figure 5 As shown, in step 501, the electronic device acquires a road image of the road in front of the guide vehicle taken by the image acquisition device mounted on the guide vehicle.

[0101] It is understandable that a road image includes not only the road itself, but also environmental information such as vehicles moving or parked on the road, pedestrians, buildings on both sides of the road, and road signs. A road image can be understood as an environmental image guiding vehicles forward, which includes road information.

[0102] In step 502, within a preset detection period, the electronic device uses the first frame of road image captured by the image acquisition device as the detection image, and identifies the detection frame of at least one target vehicle located around the guide vehicle from the detection image.

[0103] In some embodiments, the electronic device can input the detected image into the target detection network to obtain a set of target information output by the target detection network. Each target information in the target information set includes a target type and a target detection box. Then, target information with the target type of vehicle can be filtered from the target information set to obtain a detection box of at least one target vehicle around the guided vehicle.

[0104] The object detection network takes an image as input and outputs the types of different objects in the image and their bounding boxes. The bounding boxes are two-dimensional boxes that represent the location of the objects in the image.

[0105] In some embodiments, the object detection network can be trained to directly output detection boxes for target vehicles; that is, the input to the object detection network is the detection image, and the output is the detection boxes for all vehicles in the detection image. In this embodiment, the detection boxes include full vehicle boxes, rear vehicle boxes, and front vehicle boxes, wherein the full vehicle box is a rectangular box that can encompass the entire target vehicle; the rear vehicle box is a rectangular box that can encompass the rear of the target vehicle; and the front vehicle box is a rectangular box that can encompass the front of the target vehicle. In some embodiments, the object detection network is trained using the deep learning object detection toolkit mmdetection.

[0106] In some embodiments, the object detection network can be different networks, such as the SSD (Single Shot MultiBox Detector) network, or other object detection networks in the field of deep learning.

[0107] In step 503, the electronic device uses other frames of road images captured by the image acquisition device within the detection period as tracking images. For each tracking image frame, based on the detection bounding box of the target vehicle previously identified in that tracking image frame and the tracking image frame itself, it determines the detection bounding box of the target vehicle in that tracking image frame and the wheel grounding line parameters of the target vehicle. The wheel grounding line is the line connecting the wheel to the ground contact point. The wheel grounding line parameters can be understood as the slope and intercept of the wheel grounding line.

[0108] In some embodiments, the electronic device determines a set of cropping boxes corresponding to the detected image based on the detection boxes of at least one target vehicle in the detected image; then, based on the set of cropping boxes corresponding to the detected image, it determines a set of cropped images corresponding to each frame of the tracking image; thereby inputting the set of cropped images corresponding to each frame of the tracking image into the vehicle tracking network to obtain the detection box of the target vehicle in each frame of the tracking image and the wheel grounding line parameters of the target vehicle output by the vehicle tracking network. The wheel grounding line parameters are the slope and intercept of the wheel grounding line in the cropped image.

[0109] In some embodiments, the electronic device determines the center position of any detection box in the detection image as the center position of a cropping box, and expands the length and width of the detection box by a preset multiple to obtain the length and width of the cropping box, thus obtaining the cropping box corresponding to the detection box. This cropping box is used to crop the next frame tracking image of the detection image. Then, based on the set of cropping boxes corresponding to the detection image, a set of cropped images corresponding to the next frame tracking image of the detection image is determined. The preset multiple is 1.2, 1.5, or 1.6. It is understood that this embodiment is only illustrative and does not limit the specific value of the preset multiple; those skilled in the art can set it according to actual needs.

[0110] In some embodiments, for any frame of tracking image (excluding the next frame of tracking image after the detection image), the electronic device determines a set of cropping boxes corresponding to the frame of tracking image based on the set of detection boxes corresponding to the first frame of detection image preceding the frame of tracking image, and based on the set of detection boxes corresponding to all tracking images between the frame of tracking image and the first frame of detection image; thereby cropping the frame of tracking image based on the set of cropping boxes corresponding to the frame of tracking image to obtain a set of cropped images corresponding to the frame of tracking image.

[0111] In some embodiments, for any frame of tracking image (excluding the next frame of tracking image after the detection image), the electronic device determines the center position of the clipping box corresponding to the tracking image frame based on the center position of the detection box of the target vehicle in the first frame of detection image preceding the tracking image frame, and based on the center positions of the detection boxes of the same target vehicle in all tracking images between the tracking image frame and the first frame of detection image. Then, the length and width of the detection boxes of the same target vehicle in the previous frame of tracking image frame are both expanded by a preset multiple to obtain the length and width of the clipping box corresponding to the tracking image frame, thus obtaining the clipping box corresponding to the detection box. The preset multiple is 1.2, 1.5, or 1.6. It is understood that this embodiment is only illustrative and does not limit the specific value of the preset multiple; those skilled in the art can set it according to actual needs.

[0112] In some embodiments, the electronic device can input the set of cropped images corresponding to each frame of the tracking image into the vehicle tracking network to obtain the detection box of the target vehicle in each frame of the tracking image and the wheel grounding wire parameters of the target vehicle output by the vehicle tracking network.

[0113] In some embodiments, the vehicle tracking network may employ a CNN (Convolutional Neural Networks) regression network or any tracking network from computer vision. In some embodiments, the vehicle tracking network may employ a deep neural network inference framework (MNN).

[0114] In some embodiments, the electronic device performs detection or tracking in real time. For example, the electronic device performs detection or tracking once for each frame of road image captured by the image acquisition device. That is, if the road image frame is used as a detection image, the electronic device detects the target vehicle in that frame; if the road image frame is used as a tracking image, the electronic device tracks the target vehicle in that frame. The electronic device determines the detection frame of the target vehicle in that road image frame and the wheel grounding wire parameters of the target vehicle based on the detection bounding box of the target vehicle previously identified in that road image frame and the road image frame itself.

[0115] In some embodiments, the electronic device performs batch detection and tracking. For example, if an image acquisition device captures multiple frames of road images within a preset detection period, the electronic device performs batch detection and tracking on these multiple frames within the detection period. That is, the electronic device uses the first frame of the road image within the detection period as the detection image to detect the target vehicle; the electronic device uses the other frames of the road image within the detection period as tracking images to track the target vehicle. Specifically, for each tracking image frame, the electronic device determines the detection frame of the target vehicle in that tracking image frame and the wheel grounding wire parameters of the target vehicle based on the detection frame of the target vehicle previously identified in that tracking image frame and the tracking image frame itself.

[0116] In step 504, the electronic device determines whether to alert the guiding vehicle to the target vehicle based on the ground equation of the road ahead of the guiding vehicle determined during the detection period, the detection frame of the target vehicle corresponding to the tracking image during the detection period, and the wheel grounding wire parameters of the target vehicle.

[0117] For example, if the image acquisition device has an image acquisition frame rate of 20 frames per second and an image detection frame rate of 2 frames per second, then in one detection cycle, there is 1 frame of road image used as the detection image and a total of 9 frames of road image used as the tracking image. The electronic device combines the detection and tracking results of these 10 frames to determine whether to remind the guiding vehicle to pay attention to the target vehicle.

[0118] In some embodiments, the electronic device can obtain the ground equation of the road ahead of the vehicle during the detection period through IPM (Inverse Perspective Mapping).

[0119] In some embodiments, the electronic device determines the position and orientation of the target vehicle based on the detection frame of the target vehicle corresponding to the tracking image within the detection period and the wheel grounding wire parameters of the target vehicle; and detects the position and orientation of the vehicle itself; and then determines whether to remind the guiding vehicle to pay attention to the target vehicle based on the ground equation of the road ahead of the guiding vehicle determined within the detection period, the wheel grounding wire parameters of the target vehicle corresponding to the tracking image within the detection period, the position and orientation of the target vehicle, and the position and orientation of the vehicle itself.

[0120] In some embodiments, the electronic device transforms the wheel ground wire parameters from the image coordinate system to the world coordinate system based on the intrinsic and extrinsic parameters of the image sensor to obtain the orientation of the target vehicle; then, based on the intrinsic and extrinsic parameters of the image sensor, it transforms the detection box of the target vehicle corresponding to the tracking image within the detection period from the image coordinate system to the world coordinate system to obtain the position of the target vehicle.

[0121] In some embodiments, for any wheel grounding parameter, if the wheel grounding parameter is not parallel to the ground equation in the world coordinate system, the electronic device predicts the driving trajectory of the target vehicle based on the position and orientation of the target vehicle corresponding to the wheel grounding parameter; and predicts the trajectory of the vehicle based on the position and orientation of the vehicle; thereby determining whether to remind the guiding vehicle to pay attention to the target vehicle based on the driving trajectory of the target vehicle and the driving trajectory of the vehicle.

[0122] In some embodiments, the electronic device can determine whether the target vehicle is a vehicle that affects the driving behavior of the self-vehicle (leading vehicle) and is likely to cause a traffic accident based on the driving trajectory of the target vehicle and the driving trajectory of the self-vehicle. For example, this includes vehicles that change lanes, vehicles that cut in front of others, and vehicles encountered when the self-vehicle turns, drives to the left front, or drives to the right front.

[0123] In some embodiments, after determining that the electronic device should alert the target vehicle, it may do so visually, audibly, or in combination of both, so that the driver can be promptly informed of the alert.

[0124] In some embodiments, the image acquisition device has an image acquisition frame rate of 20 frames per second, meaning it can capture 20 frames of road images per second. The preset detection period is 500 milliseconds, meaning the image detection frame rate is 2 frames per second, or 2 road images per second are used as detection images. A total of 9 road images are used as tracking images within one detection period, meaning the electronic device can track the target vehicle using 18 frames of road images per second.

[0125] In some embodiments, within a detection period, the center position of the cropping box in the second frame road image (as a tracking image) is the center position of the detection box of the target vehicle in the first frame road image (as a detection image); the center position of the cropping box in the third frame road image (as a tracking image) is predicted based on the center positions of the detection boxes of the same target vehicle in the first and second frames; the center position of the cropping box in the fourth frame road image (as a tracking image) is predicted based on the center positions of the detection boxes of the same target vehicle in the first, second, and third frames. The prediction method can be as follows: connecting the center positions of the detection boxes of the target vehicle in different frames, and performing prediction on the extension of the connecting line.

[0126] Based on the method for determining prompt information provided in the above embodiments, taking an image detection frame rate of f frames per second as an example, the method flow for determining prompt information is described, including the following 5 steps:

[0127] 1. Target vehicle detection is performed using an object detection network in frames 0, f, ..., nf to obtain the full vehicle bounding box. The object detection network used is the SSD network from the deep learning field.

[0128] 2. The full vehicle frame obtained in frames 0, f, ..., nf is fixed at its center position and its length and width are enlarged by a preset multiple. Images are then cropped from frames 1, f+1, ..., nf+1 and fed into the vehicle tracking network to obtain the corresponding full vehicle frames in frames 1, f+1, ..., nf+1. Simultaneously, the slope and intercept of the grounding line are regressed. The vehicle tracking network is a CNN regression network.

[0129] 3. Based on the full vehicle frames obtained in frames 0 and 1, f and f+1, ..., nf and nf+1, the center positions of the full vehicle frames corresponding to frames 2, f+1, ..., nf+2 are inferred. The length and width of the full vehicle frames in frames 1, f+1, ..., nf+1 are then enlarged by a preset multiple. Images are cropped from frames 2, f+2, ..., nf+2 and fed into the vehicle tracking network to obtain the corresponding full vehicle frames in frames 2, f+2, ..., nf+2. Simultaneously, the wheel grounding wire parameters of the target vehicle are regressed. The wheel grounding wire parameters are the slope and intercept of the wheel grounding wire in the cropped image.

[0130] 4. Repeat step 3 in frames 3 to f-1, f+3 to 2f-1, ..., nf+3 to (n+1)f-1 to obtain the corresponding full vehicle frame and wheel grounding wire parameters.

[0131] 5. Obtain the ground equation of the road ahead of the guiding vehicle within the detection period using IPM; and determine the position and orientation of the target vehicle in the three-dimensional coordinate system based on the full vehicle frame and wheel grounding wire parameters of the target vehicle corresponding to the tracking image within the detection period; and detect the position and orientation of the self-vehicle (guiding vehicle) in the three-dimensional coordinate system; and then determine whether to remind the guiding vehicle to pay attention to the target vehicle based on the ground equation, wheel grounding wire parameters, the position and orientation of the target vehicle, and the position and orientation of the self-vehicle.

[0132] Figure 6 This is an exemplary application scenario diagram provided by an embodiment of the present disclosure. Based on the method for determining prompt information provided in the above embodiments, it is possible to obtain... Figure 6 The detection frames for different target vehicles, and the wheel grounding wires for different target vehicles. Figure 7 yes Figure 6 The diagram shown is a projection of the wheel grounding wire in the application scenario. This projection diagram is generated by IPM. Figure 6 Transformation generation. See also Figure 7 The white line represents the wheel grounding wire, while the black area can be understood as unrelated to the wheel grounding wire.

[0133] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art will understand that the embodiments of this disclosure are not limited to the described order of actions, because according to the embodiments of this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art will understand that the embodiments described in the specification are all optional embodiments.

[0134] This disclosure also proposes a non-transitory computer-readable storage medium that stores a program or instructions that cause a computer to perform steps, such as those in the embodiments of the method for determining prompt information. To avoid repetition, these steps will not be repeated here.

[0135] This disclosure also proposes a computer program product, wherein the computer program product includes a computer program stored in a non-transitory computer-readable storage medium, and at least one processor of the computer reads from the storage medium and executes the computer program, causing the computer to perform the steps of the methods for determining prompt information as described in the various embodiments, which will not be repeated here to avoid repetition.

[0136] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0137] Those skilled in the art will understand that although some embodiments described herein include certain features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of this disclosure and form different embodiments.

[0138] Those skilled in the art will understand that the descriptions of the various embodiments have different focuses, and for parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0139] Although embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present disclosure, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A method for determining a prompt message, wherein, The method includes: Acquire road images of the road ahead of the guide vehicle, captured by an image acquisition device mounted on the guide vehicle; Within a preset detection period, the first frame of road image captured by the image acquisition device is used as the detection image, and the detection frame of at least one target vehicle located around the guide vehicle is identified from the detection image. Other frames of road images captured by the image acquisition device within the detection period are used as tracking images. For each frame of tracking image, based on the detection box of the target vehicle previously identified in the tracking image and the tracking image itself, the detection box of the target vehicle in the tracking image and the wheel grounding wire parameters of the target vehicle are determined. The position and orientation of the target vehicle are determined based on the detection frame of the target vehicle corresponding to the tracking image within the detection period and the wheel grounding wire parameters of the target vehicle. Detect the vehicle's position and orientation; Based on the ground equation of the road ahead of the guiding vehicle determined within the detection period, the wheel ground wire parameters of the target vehicle corresponding to the tracking image within the detection period, the position and orientation of the target vehicle, and the position and orientation of the vehicle, it is determined whether to remind the guiding vehicle to pay attention to the target vehicle.

2. The method according to claim 1, wherein, The detection box that identifies at least one target vehicle located around the guide vehicle from the detected image includes: The detected image is input into the target detection network to obtain a set of target information output by the target detection network. Each piece of target information in the target information set includes a target type and a target detection box. Target information of vehicle type is filtered from the target information set to obtain a detection box of at least one target vehicle around the guiding vehicle.

3. The method according to claim 1, wherein, For each frame of the tracking image, based on the previously identified detection bounding boxes of the target vehicle in that frame and the tracking image itself, determining the detection bounding box of the target vehicle in that frame and the wheel grounding wire parameters of the target vehicle includes: Based on the detection bounding boxes of at least one target vehicle in the detection image, determine the set of cropping boxes corresponding to the detection image; Based on the set of cropping boxes corresponding to the detected images, determine the set of cropped images corresponding to each frame of the tracking image; The cropped image set corresponding to each frame of the tracking image is input into the vehicle tracking network to obtain the detection box of the target vehicle in each frame of the tracking image and the wheel grounding wire parameters of the target vehicle output by the vehicle tracking network.

4. The method according to claim 3, wherein, The step of determining the set of cropping boxes corresponding to the detected image based on the detection boxes of at least one target vehicle in the detected image includes: For any detection box in the detection image, the center position of the detection box is determined as the center position of the cropping box, and the length and width of the detection box are both expanded by a preset multiple to become the length and width of the cropping box, thus obtaining the cropping box corresponding to the detection box. The cropping box is used to crop the next frame of the tracking image of the detection image.

5. The method according to claim 4, wherein, The step of determining the set of cropped images corresponding to each frame of the tracking image based on the set of cropping boxes corresponding to the detected images includes: Based on the set of cropping boxes corresponding to the detected image, determine the set of cropping images corresponding to the next frame tracking image of the detected image.

6. The method according to claim 5, wherein, The process of determining the set of cropped images corresponding to each frame of tracking image also includes: For any frame of tracked image, the set of cropping boxes corresponding to the first frame of detected image preceding the tracked image is determined based on the set of detection boxes corresponding to all tracked images between the tracked image and the first frame of detected image. The tracking image of a frame is cropped based on the set of cropping boxes corresponding to that frame, resulting in a set of cropped images corresponding to that frame.

7. The method according to claim 6, wherein, The step of determining the cropping box set corresponding to any frame of tracking image, based on the set of detection boxes corresponding to the first frame of detection image preceding the tracking image, and based on the set of detection boxes corresponding to all tracking images between the tracking image and the first frame of detection image, includes: For any frame of tracking image, the center position of the cropping box corresponding to the frame of tracking image is determined based on the center position of the detection box of the target vehicle in the first frame of detection image preceding the frame of tracking image, and based on the center position of the detection box of the same target vehicle in all tracking images between the frame of tracking image and the first frame of detection image. The length and width of the detection box of the same target vehicle in the previous frame of the tracking image are both increased by a preset multiple to become the length and width of the corresponding cropping box in the current frame of the tracking image, thus obtaining the cropping box corresponding to the detection box.

8. The method according to claim 1, wherein, The step of determining the position and orientation of the target vehicle based on the detection frame of the target vehicle corresponding to the tracking image within the detection period and the wheel grounding wire parameters of the target vehicle includes: Based on the intrinsic and extrinsic parameters of the image sensor, the wheel ground wire parameters are transformed from the image coordinate system to the world coordinate system to obtain the orientation of the target vehicle; Based on the intrinsic and extrinsic parameters of the image sensor, the detection bounding box of the target vehicle corresponding to the tracking image within the detection period is transformed from the image coordinate system to the world coordinate system to obtain the position of the target vehicle.

9. The method according to claim 1, wherein, The step of determining whether to alert the guiding vehicle to the target vehicle based on the ground equation of the road ahead of the guiding vehicle determined within the detection period, the wheel ground wire parameters of the target vehicle corresponding to the tracking image within the detection period, the position and orientation of the target vehicle, and the position and orientation of the vehicle itself includes: For any wheel grounding wire parameter, if the wheel grounding wire parameter is not parallel to the ground equation in the world coordinate system, then based on the position and orientation of the target vehicle corresponding to the wheel grounding wire parameter, the trajectory of the target vehicle is predicted; and based on the position and orientation of the vehicle, the trajectory of the vehicle is predicted; based on the trajectory of the target vehicle and the trajectory of the vehicle, it is determined whether to remind the guiding vehicle to pay attention to the target vehicle.

10. A device for determining prompt information, wherein, The device includes: The acquisition unit is used to acquire road images of the road in front of the guide vehicle captured by the image acquisition device mounted on the guide vehicle; The vehicle detection unit is used to identify the detection frame of at least one target vehicle located around the guide vehicle from the first frame of road image captured by the image acquisition device as the detection image within a preset detection period. The vehicle tracking unit is used to take other frames of road images captured by the image acquisition device during the detection period as tracking images. For each frame of tracking image, based on the detection frame of the target vehicle previously identified in the tracking image and the tracking image itself, the unit determines the detection frame of the target vehicle in the tracking image and the wheel grounding wire parameters of the target vehicle. The determination prompting unit is used to determine the position and orientation of the target vehicle based on the detection frame of the target vehicle corresponding to the tracking image within the detection period and the wheel grounding wire parameters of the target vehicle; Detect the vehicle's position and orientation; Based on the ground equation of the road ahead of the guiding vehicle determined within the detection period, the wheel ground wire parameters of the target vehicle corresponding to the tracking image within the detection period, the position and orientation of the target vehicle, and the position and orientation of the vehicle, it is determined whether to remind the guiding vehicle to pay attention to the target vehicle.

11. An electronic device, wherein, include: Processor and memory; The processor executes the steps of the method as described in any one of claims 1 to 9 by invoking programs or instructions stored in the memory.

12. A non-transitory computer-readable storage medium, wherein, The non-transitory computer-readable storage medium stores a program or instructions that cause a computer to perform the steps of the method as described in any one of claims 1 to 9.

13. A computer program product, wherein, The computer program product includes a computer program stored in a non-transitory computer-readable storage medium, wherein at least one processor of the computer reads from the storage medium and executes the computer program, causing the computer to perform the steps of the method as described in any one of claims 1 to 9.

Images