Vehicle warning method and apparatus, and electronic device and computer-readable storage medium

Abstract

A vehicle warning method and apparatus, and an electronic device and a computer-readable storage medium. The method comprises: capturing, via a sensor, a plurality of images during the travelling of a vehicle, wherein the sensor is located at the tail of the vehicle (S110); executing target detection for each of the plurality of images, so as to obtain one or more bounding boxes of one or more objects and position information of the one or more bounding boxes (S120); on the basis of the one or more bounding boxes and the position information thereof, determining a motion trajectory of the one or more objects (S130); and on the basis of the motion trajectory of the one or more objects, determining whether to perform warning on the vehicle (S140).

Description

Vehicle warning methods and devices, electronic devices and computer-readable storage media Technical Field

[0001] This disclosure relates to the field of computer vision technology, and in particular to vehicle warning methods and devices, electronic devices and computer-readable storage media. Background Technology

[0002] When a vehicle is traveling on the road, the driver may not be able to observe moving objects behind them, such as other vehicles or pedestrians, in time due to obstructed vision or other reasons. In such situations, a collision may occur when the driver changes lanes and encounters a moving object behind them.

[0003] The methods described in this section are not necessarily methods that had been previously conceived or adopted. Unless otherwise specified, no method described in this section should be assumed to be prior art simply because it is included in this section. Similarly, unless otherwise specified, the issues mentioned in this section should not be considered to be accepted in any prior art. Summary of the Invention

[0004] According to one aspect of this disclosure, a vehicle warning method is provided, comprising: capturing a plurality of images of the vehicle during driving via sensors located at the rear of the vehicle; performing object detection for each of the plurality of images to obtain one or more bounding boxes of one or more objects and position information of the one or more bounding boxes; determining the motion trajectory of the one or more objects based on the one or more bounding boxes and their position information; and determining whether to issue a warning to the vehicle based on the motion trajectory of the one or more objects.

[0005] According to another aspect of this disclosure, a vehicle warning device is provided, comprising: a capture unit configured to capture a plurality of images of the vehicle during travel via sensors located at the rear of the vehicle; a target detection unit configured to perform target detection for each of the plurality of images to obtain one or more bounding boxes of one or more objects and position information of the one or more bounding boxes; a motion trajectory determination unit configured to determine the motion trajectory of the one or more objects based on the one or more bounding boxes and their position information; and a warning unit configured to determine whether to issue a warning to the vehicle based on the motion trajectory of the one or more objects.

[0006] According to another aspect of this disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored on the memory, wherein the processor is configured to execute the computer program to implement the steps of the vehicle warning method described above.

[0007] According to another aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the vehicle warning method described above.

[0008] According to another aspect of this disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, it implements the steps of the vehicle warning method described above.

[0009] Further features and advantages of this disclosure will become clear from the exemplary embodiments described below in conjunction with the accompanying drawings. Attached Figure Description

[0010] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0011] Figure 1 is a flowchart illustrating a vehicle warning method according to an exemplary embodiment;

[0012] Figure 2 is a schematic diagram illustrating model distillation according to an exemplary embodiment;

[0013] Figure 3 is a flowchart illustrating the determination of the motion trajectory of one or more objects based on one or more bounding boxes and their position information according to an exemplary embodiment;

[0014] Figure 4 is a flowchart illustrating a vehicle warning method according to another exemplary embodiment;

[0015] Figure 5 is a flowchart illustrating a process for determining whether to issue a warning to a vehicle based on the motion trajectory of one or more objects, according to an exemplary embodiment.

[0016] Figure 6 is a flowchart illustrating a process for determining whether to issue a warning to a vehicle based on the motion trajectory of one or more objects, according to another exemplary embodiment.

[0017] Figure 7 shows a structural block diagram of a vehicle warning device according to an embodiment of the present disclosure;

[0018] Figure 8 is a structural block diagram illustrating a computing device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0019] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0020] The terminology used in the description of the various examples described in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. Furthermore, the term "and / or" as used in this disclosure covers any one of the listed items and all possible combinations thereof.

[0021] When a vehicle is traveling on the road, the driver may not be able to observe moving objects behind them, such as other vehicles or pedestrians, in time due to obstructed vision. In such cases, a collision may occur when the driver changes lanes and encounters a moving object behind them. To address this, blind spot warning systems are being installed on vehicles to provide safety warnings to the driver by detecting moving objects behind the vehicle in real time.

[0022] Existing vehicle blind spot warning systems primarily rely on ranging sensors, such as radar or ultrasonic sensors, to detect the distance to moving objects behind the vehicle and issue safety warnings to the driver based on distance and speed information. However, the inventors discovered that these ranging sensors cannot classify or perceive the contours of moving objects behind the vehicle, thus failing to provide accurate and detailed warnings to the driver. Furthermore, some vehicles (such as motorcycles, electric bicycles, two-wheeled vehicles like self-balancing scooters and electric skateboards, and unicycles like self-balancing scooters) often tilt during operation (e.g., when turning or traversing bumpy roads), which may reduce the accuracy of these ranging sensors, leading to significant errors in moving object detection. Installing multiple ranging sensors on the vehicle to improve detection and warning accuracy would increase costs and make it difficult to align and fuse multiple inputs, increasing the probability of warning system malfunctions.

[0023] To address the aforementioned technical problems, this disclosure provides a novel vehicle warning method and apparatus, electronic device, computer-readable storage medium, and computer program product. As will be clearly seen in the following detailed description, the vehicle warning method according to embodiments of this disclosure can capture multiple images during vehicle operation, locate and identify objects behind the vehicle based on this visual information, determine their movement trends, and provide safety warnings to the driver based on the movement trends of one or more objects. Compared to traditional ranging sensors, target detection, localization, and movement trend determination based on image-based visual information are more efficient. Furthermore, since the vehicle according to embodiments of this disclosure uses only one image sensor as the target detection sensor, costs are reduced, it is easy to use, and has a low failure rate.

[0024] Exemplary embodiments of the vehicle warning method of this disclosure will now be described in further detail with reference to the accompanying drawings.

[0025] Figure 1 shows a flowchart of a vehicle warning method 100 according to an exemplary embodiment of the present disclosure. As shown in Figure 1, the vehicle warning method 100 may include: step S110, capturing multiple images of a vehicle during driving via a sensor located at the rear of the vehicle; step S120, performing object detection for each of the multiple images to obtain one or more bounding boxes of one or more objects and position information of the one or more bounding boxes; step S130, determining the motion trajectory of one or more objects based on the one or more bounding boxes and their position information; and step S140, determining whether to issue a warning to the vehicle based on the motion trajectory of the one or more objects.

[0026] Compared to traditional ranging sensors, the vehicle warning method according to embodiments of this disclosure can capture multiple images during vehicle movement using a visual sensor, and based on these images, locate and identify objects behind the vehicle and determine their movement trends. Since image-based visual information-based target detection and localization, as well as movement trend determination, are more efficient, and these operations can be achieved using only a single image sensor, costs can be reduced, operations simplified, and failure rates lowered.

[0027] According to some embodiments of this disclosure, the vehicle may be a two-wheeled vehicle or a unicycle.

[0028] In step S110, the sensor can be any suitable sensor capable of capturing images, such as a camera, video camera, webcam, etc. Such sensors can be standalone devices as described above, or they can be included in various electronic devices (e.g., mobile phones, computers, personal digital assistants, tablets, wearable devices, etc.). The scope of the subject matter claimed in this disclosure is not limited in the foregoing respects.

[0029] In step S110, the sensor can be located anywhere at the rear of the vehicle, as long as it can capture moving objects behind the vehicle in real time. For example, it can be installed above the rear wheels of the vehicle, or on the rear storage compartment of the vehicle, and preferably installed in the direction of the vehicle's centerline.

[0030] According to some embodiments of this disclosure, in step S120, performing object detection for each of the multiple images to obtain one or more bounding boxes of one or more objects and the location information of one or more bounding boxes may include: performing object detection for each of the multiple images based on a neural network model to obtain one or more bounding boxes of one or more objects and the location information of one or more bounding boxes.

[0031] In some examples, the location information of the bounding box of each object can be the center point of the bottom border of the bounding box. In other examples, the location information of the bounding box of each object can be the geometric center point of the bounding box. In still other examples, the location information of the bounding box of each object can be the center point of the top border of the bounding box. This location information can be, for example, the two-dimensional or three-dimensional coordinates of the center point.

[0032] It will be understood that the location information of the bounding box can refer to the coordinate information of any suitable point of the bounding box, and the scope of the subject matter claimed in this disclosure is not limited in the foregoing.

[0033] It will also be understood that the neural network model may be a YOLOv8 model or other models based on deep neural network algorithms to perform object detection for each image.

[0034] According to some embodiments of this disclosure, object detection for each of a plurality of images based on a neural network model may include: performing object detection on the plurality of images based on a student model network, wherein the student model network is obtained by: training a teacher model network, wherein the teacher model network has a first number of parameters; and obtaining a student model network based on the teacher model network, wherein the student model network has a second number of parameters less than the first number of parameters.

[0035] Figure 2 is a schematic diagram illustrating model distillation according to an exemplary embodiment. As shown in Figure 2, the object detection model 200 may include a teacher model network 210 with a relatively large number of parameters and strong learning ability, and a student model network 220 with a relatively small number of parameters and weak learning ability. During the training phase, the teacher model network 210 is first trained using training samples, and then distillation is performed based on the error listed in the following formula to obtain the student model network 220 through "knowledge transfer":

[0036] in, It's cross-entropy. p(x) is the output of the student model network, q(x) is the output of the teacher model network, l(x) is the logits output of the student model network, and z(x) is the logits output of the teacher model network.

[0037] Using a teacher model network to guide the training of a student model network allows for the "knowledge distillation" of the feature representations learned by the complex and highly capable teacher model network. These learned representations are then passed to the student model network, which has fewer parameters and weaker learning capabilities, resulting in a faster and more expressive student model network. Consequently, compared to outputs obtained solely from the student model network, the model's accuracy can be improved (e.g., by 10%).

[0038] According to some embodiments of this disclosure, object detection for each of a plurality of images based on a neural network model may include: quantizing the neural network model to convert the input data type of the neural network model into a predefined data type.

[0039] Typically, the input data to a neural network model can be floating-point data. This type of data can be computationally complex, limiting its application scenarios. By quantizing this type of model—converting floating-point data to a low-bit-width data type (e.g., 8-bit or 4-bit) suitable for edge devices (e.g., int 8)—the complexity of operations during convolution and other computations can be significantly reduced. This, in turn, reduces the model size, enabling deployment on hardware-constrained devices and reducing computation time.

[0040] For example, the data type of the parameters input to the neural network model can be fp32. Then, quantization can convert the data type of these parameters to int8. Next, convolution is performed on the quantized data (the result data type remains int8). Finally, dequantization converts the data type of the convolution result back to fp32 for output. In this example, because low-bit-width data types are used instead of floating-point data types for computation in the neural network, the complexity and size of the model can be reduced while maintaining model performance.

[0041] Furthermore, after quantizing the model, it can be fine-tuned and trained to avoid a decrease in accuracy due to quantization loss.

[0042] According to some embodiments of this disclosure, the vehicle warning method 100 may further include: in step S120, performing target detection for each of a plurality of images to obtain the category of one or more objects.

[0043] In some examples, neural network models can be used for object detection in images. When training such neural networks, object recognition can be performed on categories such as pedestrians, cars, and two-wheeled vehicles based on a large dataset, without recognizing other objects. This is because in real-world applications, objects related to vehicle safety are usually more relevant, thus reducing the size of the training dataset and improving the accuracy of object recognition.

[0044] In some examples, after object detection is performed on an image, a category ID can be assigned to each identified object. For example, Pedestrian 1, Pedestrian 2, Car 1, Car 2, etc., to facilitate subsequent target tracking of these objects.

[0045] Figure 3 is a flowchart illustrating the determination of the motion trajectory of one or more objects based on one or more bounding boxes and their position information according to an exemplary embodiment. According to some embodiments of this disclosure, as shown in Figure 3, step S130, determining the motion trajectory of one or more objects based on one or more bounding boxes and their position information, may include, for each of the one or more objects in each image: step S310, determining the confidence level between the object and each object in a previously captured image; step S320, determining that the object matches a corresponding object in a previously captured image based on a confidence level greater than a first confidence threshold; step S330, determining whether the confidence level is greater than a second confidence threshold based on a confidence level not greater than the first confidence threshold, wherein the second confidence threshold is less than the first confidence threshold; step S340, determining that the object matches a corresponding object in a previously captured image based on a confidence level greater than the second confidence threshold; and step S350, determining the motion trajectory of the object based on the bounding boxes and their position information of the matched objects in multiple images.

[0046] In step S310, the confidence level can be a value between 0 and 1, and represents the degree of confidence the model has in the detected target. For example, the confidence level can be calculated by the following two factors: (1) whether there is an object within the bounding box obtained by the target detection. If the bounding box accurately surrounds an object, the confidence level may be high, while if there is no object or only a part of the object is present in the bounding box, the confidence level may be low; (2) the matching degree between the object within the bounding box and each object in the previously captured image. If the matching degree between the object within the bounding box and an object in the previously captured image is high, the confidence level is high, while if the matching degree between the object within the bounding box and an object in the previously captured image is low, the confidence level is low.

[0047] In step S320, the first confidence threshold can be any suitable value, such as 0.3, 0.35, etc. If the confidence between the object within the identified bounding box and an object in a previously captured image is greater than the confidence threshold, it indicates that the two objects match.

[0048] In real-world applications, it's possible that an object in a captured image may be occluded at a certain moment. In this case, the confidence level between the occluded object and any object in the previously captured image may be less than a preset confidence threshold, but this does not mean that the object was not identified in that frame. To avoid the loss of an object in a frame due to occlusion, a second confidence threshold less than the first confidence threshold can be set, as described in step S330 below.

[0049] In steps S330-S340, the second confidence threshold can be any suitable value less than the first confidence threshold, or it can be a threshold range, for example, the second confidence threshold can be a threshold range of 0.1-0.3. When the confidence between the object within the identified bounding box and any object in the previously captured image is less than the first confidence threshold, the confidence value can be compared with the second confidence threshold. If the confidence value is not greater than the second confidence threshold, it indicates that there is no matching object in the previously captured image, meaning that the object is relatively likely to be missing in this frame. If the confidence value is greater than the second confidence value, it indicates that there is a matching object in the previously captured image, but the confidence value is low due to occlusion or other reasons.

[0050] In step S350, after performing confidence calculations and determining the matching results for each object in each image, the motion trajectory of each object can be determined based on the position information of the bounding boxes of the matching objects in multiple images. As mentioned above, the position information of the bounding boxes can refer to the coordinates of the center point of the lower boundary of the bounding box. By connecting the center points of the lower boundaries of the bounding boxes of a matching object in multiple images, the motion trajectory of that object can be obtained.

[0051] It will be understood that “image captured before this image” in this disclosure may refer to an image captured in the previous frame or to images captured in succession from previous frames, and the scope of the subject matter claimed in this disclosure is not limited in the foregoing.

[0052] During target tracking, performing two confidence comparisons on the detected objects can prevent the loss of objects in a frame due to occlusion, thus improving the accuracy of target tracking. Furthermore, since target tracking only uses the bounding boxes and positional information of the detected objects (preferably, the object category can also be combined for target tracking), without other complex operations, the efficiency of target tracking can be significantly improved.

[0053] According to some embodiments of this disclosure, the vehicle warning method 100 may further include: acquiring motion information from sensors; and wherein step S130, determining the motion trajectory of one or more objects based on one or more bounding boxes and their position information, may further include: for each of the one or more objects in each image, determining whether the object matches a corresponding object in a previously captured image based on both confidence level and motion information.

[0054] In some examples, motion information from sensors such as cameras can be further incorporated to track targets. For instance, vehicles such as two-wheeled vehicles may experience bumps or tilts during operation; by compensating for target tracking results by taking into account sensor motion information, the accuracy of target tracking can be further improved.

[0055] Figure 4 is a flowchart illustrating a vehicle warning method according to another exemplary embodiment. As shown in Figure 4, the vehicle warning method 400 may include steps S410-S440, which are similar to steps S110-S140 in the vehicle warning method 100 described with reference to Figure 1; and step S450, correcting the motion trajectory of one or more objects.

[0056] As mentioned above, vehicles may experience bumps and tilts during operation, which can affect not only the ability to track objects but also the accuracy of the tracked object's position. For example, when a vehicle turns or traverses bumpy roads, the trajectory of the detected object may fluctuate abnormally due to the camera's tilt or pitch. By correcting the trajectory after determining the movement of one or more objects and compensating for abnormal fluctuations, the object's movement trajectory can be made smoother and more accurate, thus improving the accuracy of warning information.

[0057] According to some embodiments of this disclosure, step S450, correcting the motion trajectory of one or more objects, may include: capturing the pose information of the vehicle during driving; and correcting the motion trajectory of one or more objects based on the pose information.

[0058] In some examples, IMU attitude sensors can be used to capture the vehicle's pose information during driving. An IMU attitude sensor is a high-performance 3D motion attitude measurement system based on MEMS technology. It includes motion sensors such as a three-axis gyroscope, a three-axis accelerometer, and a three-axis electronic compass. It can obtain temperature-compensated 3D attitude and orientation data through an embedded ARM processor. Utilizing quaternion-based 3D algorithms and data fusion technology, it can output zero-drift 3D attitude and orientation data in quaternion and Euler angle representation in real time.

[0059] By using data obtained from the IMU attitude sensor to correct and compensate for the trajectory of the tracked object, the effects of the vehicle body sensor floating up and down and tilting can be eliminated when the vehicle bumps and tilts.

[0060] According to some other embodiments of this disclosure, step S450, correcting the motion trajectory of one or more objects, may include: for each of the one or more objects, based on determining that the bounding box of the object is not detected in at least one of the plurality of images, compensating for the position information of the bounding box of the object in at least one image to obtain the corrected motion trajectory of the object, based on the position information of the bounding box of the object detected in the first image and the position information of the bounding box of the object detected in the second image, wherein the first image is an image captured before at least one of the plurality of images, and the second image is an image captured after at least one of the plurality of images.

[0061] As mentioned above, during vehicle operation, there may be instances where objects in captured images are occluded at some point. If an object is completely occluded in the corresponding image (i.e., no matching object is found in either previously captured or subsequently captured images), the object's motion trajectory will be missing.

[0062] According to embodiments of this disclosure, the missing trajectory points of an object in an image can be interpolated and supplemented based on the position information of the bounding box of the corresponding object in a previously captured image and the position information of the bounding box of the corresponding object in a subsequently captured image.

[0063] In some examples, linear interpolation algorithms can be used to interpolate and supplement missing trajectory points of an object in the image. For instance, if the lower bounding box coordinates of the corresponding object in a previously captured adjacent image are (x1, y1), and the lower bounding box coordinates of the corresponding object in a subsequently captured adjacent image are (x2, y2), then the coordinates of the missing trajectory points in the intermediate image can be obtained by calculating the slope m = (y2 - y1) / (x2 - x1). Continuing this example, considering the short time interval between capturing adjacent image frames, x = (x1 + x2) / 2 can be used as the x-coordinate of the missing trajectory point in the intermediate image, and the formula y = y1 + m * (x - x1) can be applied to calculate the estimated y-coordinate of the missing trajectory point.

[0064] In other examples, other nonlinear interpolation algorithms can be used to interpolate and supplement missing trajectory points of the object in the image. For example, a nonlinear fit can be performed based on the position coordinates of the lower boundary of the bounding box of the corresponding object in several previously captured adjacent images and the position coordinates of the lower boundary of the bounding box of the corresponding object in several subsequently captured adjacent images. Similarly, x = (x1 + x2) / 2 is used as the x-coordinate of the missing trajectory point in the intermediate image, and a nonlinear fitting formula is applied to calculate the estimated y-coordinate of the missing trajectory point.

[0065] It will be understood that any one or more previously captured images and any one or more subsequently captured images can be selected, depending on the actual needs, to interpolate and supplement missing trajectory points of an object in a given image. For example, if computational simplicity is desired, a linear interpolation algorithm can be chosen; if further improvement in the accuracy of the supplemented trajectory points is desired, multiple previously captured images and multiple subsequently captured images can be selected.

[0066] By correcting the trajectory and supplementing missing points, the trajectory of an object can be made smoother and more continuous, thus making it better suited for predicting the movement trend of the object and improving the accuracy of early warning information.

[0067] Figure 5 is a flowchart illustrating whether to issue a vehicle warning based on the motion trajectory of one or more objects according to an exemplary embodiment. As shown in Figure 5, step S140, determining whether to issue a vehicle warning based on the motion trajectory of one or more objects, may include: step S510, for each of the one or more objects, determining whether the bounding box of the object is located at the boundary of the corresponding image; step S520, determining whether to issue a vehicle warning based on the width change of the bounding box of the object in the corresponding image, based on the determination that the bounding box of the object is located at the boundary of the corresponding image; and step S530, determining whether to issue a vehicle warning based on the motion trajectory of the bounding box of the object, based on the determination that the bounding box of the object is not located at the boundary of the corresponding image.

[0068] According to some embodiments of this disclosure, the location information of one or more bounding boxes may be the midpoint of the lower boundary of the corresponding bounding box, and step S510, determining whether the bounding box of each of the one or more objects is located at the boundary of the corresponding image, may include: determining that the bounding box of the object is located at the boundary of the corresponding image based on the determination that at least a portion of the bounding box of the object is located outside the left or right boundary of the corresponding image; and determining that the bounding box of the object is not located at the boundary of the corresponding image based on the determination that the entire bounding box of the object is located in the corresponding image.

[0069] Specifically, when performing object detection on an image, one or more objects may be located outside the sensor's field of view. Consequently, the bounding boxes of these objects will be located at the left or right edge of the corresponding image, indicating object truncation. In contrast, if one or more objects are entirely within the sensor's field of view, the bounding boxes of these objects will be entirely within the corresponding image.

[0070] By applying different warning algorithms based on the object's location within the image, more accurate warning results can be obtained. The following section details how to predict the object's movement trend and how to issue warnings for vehicles based on whether the object is located at the boundary of the corresponding image.

[0071] According to some embodiments of this disclosure, step S520, determining whether to issue a vehicle warning based on the width change of the object's bounding box in the corresponding image after determining that the bounding box of the object is located at the boundary of the corresponding image, may include: determining whether the width of the object's bounding box in the corresponding image is less than a preset width threshold; issuing a vehicle warning based on the determination that the width of the object's bounding box in the corresponding image is less than the preset width threshold; determining whether the object's movement trajectory is moving away from or towards the vehicle based on the determination that the object's movement trajectory is moving away from the vehicle, and not issuing a vehicle warning based on the determination that the object's movement trajectory is moving towards the vehicle; and issuing a vehicle warning when the width of the object's bounding box in the corresponding image is less than the preset width threshold based on the determination that the object's movement trajectory is moving towards the vehicle.

[0072] In some examples, the preset width threshold can be any suitable pixel value set according to needs, such as 5, 8, 10, etc. A larger preset width threshold allows for earlier warnings to the vehicle, thus reducing the occurrence of vehicle collisions. A smaller preset width threshold results in later warnings to the vehicle, thus avoiding frequent warnings to the driver.

[0073] Continuing the example above, with a preset width threshold of 5, and when the bounding box of an object is located at the boundary of the corresponding image (e.g., the left boundary), if the width of the object's bounding box in the corresponding image is less than 5 pixels, it can be determined that the object is within the vehicle's warning area, and a warning can be issued to the vehicle's driver. If the width of the object's bounding box in the corresponding image is not less than 5 pixels, it can be determined that the object is relatively far from the vehicle. In this case, the movement trend of the object can be judged, i.e., whether to issue a warning to the vehicle's driver is determined by judging whether the object is moving closer to or further away from the vehicle.

[0074] According to some embodiments of this disclosure, step S520, determining whether to issue a vehicle warning based on the width change of the object's bounding box in the corresponding image, based on the determination that the object's bounding box is located at the boundary of the corresponding image, may include: determining whether the width of the object's bounding box in the corresponding image is less than a preset width threshold; determining whether the width change of the object's bounding box in the corresponding image exceeds a preset width change threshold; determining whether the width change of the object's bounding box in the corresponding image exceeds the preset width change threshold; determining whether the object's movement trajectory is moving away from or towards the vehicle; determining whether the object's movement trajectory is moving away from the vehicle and not issuing a vehicle warning; determining whether the object's movement trajectory is moving towards the vehicle and issuing a vehicle warning; and determining whether the width change of the object's bounding box in the corresponding image does not exceed the preset width change threshold and not issuing a vehicle warning.

[0075] In some examples, when an object's bounding box is located at the boundary of the corresponding image, if the width of the object's bounding box in the corresponding image is not less than a preset width threshold, the vehicle warning can be determined based on the object's movement speed. The object's movement speed can be determined by calculating the change in width of the object's bounding box between the corresponding image and its preceding adjacent image. If the object's movement speed is high, it can be further determined whether the object's trajectory is moving away from or towards the vehicle. If it is moving away from the vehicle, no warning is issued; if it is moving towards the vehicle, a warning is issued. If the object's movement speed is low, it can be assumed that the movement is due to vehicle vibration or bumps and does not pose a significant collision risk; therefore, no warning is issued.

[0076] In some examples, the preset width change threshold can also be any suitable pixel value set as needed. A smaller preset width change threshold results in earlier vehicle warnings, thus reducing the occurrence of vehicle collisions. A larger preset width threshold results in later vehicle warnings, thus avoiding frequent warnings to the driver.

[0077] When a vehicle body shakes or vibrates, it typically causes changes in the bounding box of the object in the corresponding image (e.g., a change in width). However, such changes usually do not affect the collision hazard factor. By setting a preset threshold for the amount of width change, premature and frequent warnings to the driver in this situation can be avoided.

[0078] It will be understood that the vehicle warning can also be determined by combining the object's acceleration. For example, the object's acceleration can be further determined, and the vehicle warning can be issued when the object's acceleration is large, while no warning is issued when the object's deceleration is large (i.e., the object is decelerating).

[0079] According to some embodiments of this disclosure, the motion trajectory of the object can be determined to be moving away from the vehicle based on the determination that the width of the bounding box of the object in the corresponding image increases, and the motion trajectory of the object can be determined to be moving closer to the vehicle based on the determination that the width of the bounding box of the object in the corresponding image decreases.

[0080] According to some embodiments of this disclosure, step S530, determining whether to issue a warning to a vehicle based on the motion trajectory of the object's bounding box, based on the determination that the object's bounding box is not located at the boundary of the corresponding image, may include: determining whether the distance between the object and the vehicle is less than a preset trajectory motion threshold based on the position information of the object's bounding box; issuing a warning to the vehicle based on the determination that the distance between the object and the vehicle is less than the preset trajectory motion threshold; determining whether the object's motion trajectory is moving away from or towards the vehicle based on the determination that the object's motion trajectory is moving away from the vehicle; not issuing a warning to the vehicle based on the determination that the object's motion trajectory is moving towards the vehicle; and issuing a warning to the vehicle when the distance exceeds the preset trajectory motion threshold based on the determination that the object's motion trajectory is moving towards the vehicle.

[0081] Similarly, the preset trajectory motion threshold can be any suitable pixel value set according to needs, such as 5, 8, 10, etc. A larger preset trajectory motion threshold allows for earlier warnings to the vehicle, thereby reducing the occurrence of vehicle collisions. A smaller preset trajectory motion threshold results in later warnings to the vehicle, thus avoiding frequent warnings to the driver.

[0082] Continuing the example above, if the preset trajectory motion threshold is 5 and the bounding box of an object is not located at the boundary of the corresponding image, and if the distance between the object and the vehicle is determined to be less than 5 pixels based on the object's bounding box location information (e.g., coordinate information), then the object is within the vehicle's warning area, and a warning can be issued to the driver. If the distance between the object and the vehicle is determined to be not less than 5 pixels based on the object's bounding box location information (e.g., coordinate information), then the motion trend of the object can be further judged, determining whether to issue a warning to the driver by judging whether the object is moving closer to or further away from the vehicle.

[0083] It will be understood that the preset width threshold and the preset trajectory motion threshold can be the same value or different values. By using adaptive thresholds based on the area where the object is located for warnings, it is possible to better adapt to the needs of the scenario, provide more accurate warnings to the vehicle driver, and minimize unnecessary warnings.

[0084] According to some embodiments of this disclosure, step S530, determining whether to issue a warning to a vehicle based on the motion trajectory of the object's bounding box, based on the determination that the object's bounding box is not located at the boundary of the corresponding image, may include: determining whether the distance between the object and the vehicle is less than a preset trajectory motion threshold based on the position information of the object's bounding box; determining whether the displacement change of the object's bounding box in the corresponding image and the previous adjacent image exceeds a preset trajectory motion change threshold; determining whether the object's motion trajectory is moving away from or towards the vehicle; not issuing a warning to the vehicle if the object's motion trajectory is moving away from the vehicle; issuing a warning to the vehicle if the object's motion trajectory is moving towards the vehicle; and not issuing a warning to the vehicle if the object's motion trajectory is moving towards the vehicle; and not issuing a warning to the vehicle if the object's bounding box does not exceed the preset trajectory motion change threshold.

[0085] Similar to the embodiment where the object's bounding box is located at the boundary of the corresponding image, when the object's bounding box is not located at the boundary of the corresponding image, if the distance between the object and the vehicle is less than a preset trajectory motion threshold, the vehicle's warning can be determined by considering the object's motion speed. The object's motion speed can be determined by calculating the displacement change of the object's bounding box in the corresponding image and its preceding adjacent image. If the object's motion speed is high, it can be further determined whether the object's trajectory is moving away from or towards the vehicle. If it is moving away from the vehicle, no warning is issued; if it is moving towards the vehicle, a warning is issued. If the object's motion speed is low, it can be assumed that the motion is due to vehicle vibration or bumps and does not pose a significant collision risk; therefore, no warning is issued.

[0086] Similarly, the preset trajectory motion change threshold can be any suitable pixel value set according to needs. The smaller the preset trajectory motion change threshold, the earlier the vehicle is warned, thereby reducing the occurrence of vehicle collisions. The larger the preset trajectory motion change threshold, the later the vehicle is warned, thereby avoiding frequent warnings to the driver.

[0087] When a vehicle body shakes or vibrates, it typically causes changes in the bounding box of the object in the corresponding image (for example, changes in positional information may lead to changes in the distance between the identified object and the vehicle). However, such changes usually do not affect the collision risk factor. By setting a preset threshold for the amount of trajectory motion change, premature and frequent warnings to the driver in such situations can be avoided.

[0088] It will be understood that the vehicle warning can also be determined by combining the object's acceleration. For example, the object's acceleration can be further determined, and the vehicle warning can be issued when the object's acceleration is large, while no warning is issued when the object's deceleration is large (i.e., the object is decelerating).

[0089] It will also be understood that the preset width change threshold and / or preset trajectory motion change threshold can vary based on the distance between the object and the vehicle. In some examples, the distance between the object and the vehicle is relatively large, and the probability of a collision is relatively low. In other examples, the distance between the object and the vehicle is relatively small, and the probability of a collision is relatively high. By adaptively adjusting the preset width change threshold and / or preset trajectory motion change threshold according to the distance between the object and the vehicle, warnings to the vehicle driver can be provided more flexibly and accurately.

[0090] Figure 6 is a flowchart illustrating whether to issue a warning to a vehicle based on the motion trajectory of one or more objects, according to another exemplary embodiment. As shown in Figure 6, step S140, determining whether to issue a warning to a vehicle based on the motion trajectory of one or more objects, may include: step S610, determining the positional relationship between one or more objects and the vehicle; and step S620, issuing a warning to the vehicle based on the positional relationship.

[0091] By determining the positional relationship between one or more objects and a vehicle, different warning algorithms can be set for the area where the object is located, thereby providing more accurate warning information. This allows vehicle drivers to better understand the cause of the warning and respond to the warning information in a timely manner.

[0092] According to some embodiments of this disclosure, the positional relationship may include one or more of the following: one or more objects are located in the blind spot of the vehicle; one or more objects are located in the adjacent lane of the vehicle; or one or more objects are located behind the vehicle.

[0093] Blind Spot Detection (BSD) can be enabled when one or more objects are in the vehicle's blind spot to monitor the driver's blind spot in real time and warn the driver when other road users appear in the blind spot.

[0094] Lane Changing Warning (LCW) can be activated when one or more objects are in the adjacent lane of the vehicle. It monitors the adjacent lane when the vehicle is changing lanes and warns the driver when other road users who may collide with the vehicle appear to the side or rear of the vehicle.

[0095] When one or more objects are located behind the vehicle, the Rear Collision Warning (RCW) can be activated to monitor the area behind the vehicle in real time and warn the driver when other road users who may be involved in a rear-end collision are present.

[0096] Figure 7 shows a structural block diagram of a vehicle warning device 700 according to an embodiment of the present disclosure. As shown in Figure 7, the vehicle warning device 700 may include: a capture unit 710 configured to capture multiple images of a vehicle in motion via a sensor located at the rear of the vehicle; a target detection unit 720 configured to perform target detection for each of the multiple images to obtain one or more bounding boxes of one or more objects and position information of the one or more bounding boxes; a motion trajectory determination unit 730 configured to determine the motion trajectory of one or more objects based on the one or more bounding boxes and their position information; and a warning unit 740 configured to determine whether to issue a warning to the vehicle based on the motion trajectory of the one or more objects.

[0097] Compared to traditional ranging sensors, the vehicle warning device according to embodiments of this disclosure can capture multiple images during vehicle operation based on a visual sensor, and thereby locate and identify objects behind the vehicle and determine their movement trends. Since image-based visual information-based target detection and localization, as well as movement trend determination, are more efficient, and these operations can be achieved using only a single image sensor, costs can be reduced, operation simplified, and the failure rate lowered.

[0098] According to some embodiments of this disclosure, the vehicle warning device 700 may further include: a unit configured to perform target detection on each of a plurality of images to obtain the category of one or more objects; and wherein determining the motion trajectory of one or more objects is further based on the category of one or more objects.

[0099] According to some embodiments of this disclosure, the target detection unit 720 may include: a target detection subunit configured to perform target detection on each of a plurality of images based on a neural network model to obtain one or more bounding boxes of one or more objects and position information of one or more bounding boxes.

[0100] According to some embodiments of this disclosure, the object detection subunit may include: a unit configured to perform object detection on multiple images based on a student model network, wherein the student model network is obtained by: training a teacher model network having a first number of parameters; and obtaining a student model network based on the teacher model network having a second number of parameters less than the first number of parameters.

[0101] According to some embodiments of this disclosure, the object detection subunit may include a unit configured to quantize a neural network model to convert the input data type of the neural network model into a predefined data type.

[0102] According to some embodiments of this disclosure, the motion trajectory determination unit 730 may include units that perform the following operations for each of one or more objects in each image: determining a confidence level between the object and each object in a previously captured image; determining that the object matches a corresponding object in a previously captured image based on a confidence level greater than a first confidence threshold; determining whether a confidence level is greater than a second confidence threshold based on a confidence level not greater than the first confidence threshold, wherein the second confidence threshold is less than the first confidence threshold; determining that the object matches a corresponding object in a previously captured image based on a confidence level greater than the second confidence threshold; and determining the motion trajectory of the object based on the bounding box and position information of the matched object in multiple images.

[0103] According to some embodiments of this disclosure, the vehicle warning device 700 further includes a unit configured to acquire motion information from sensors, and wherein the motion trajectory determination unit 730 may include a unit configured to determine, for each of one or more objects in each image, whether the object matches a corresponding object in a previously captured image based on both confidence level and motion information.

[0104] According to some embodiments of this disclosure, the vehicle warning device 700 may further include: a motion trajectory correction unit configured to correct the motion trajectory of one or more objects.

[0105] According to some embodiments of this disclosure, the motion trajectory correction unit may include: a unit configured to capture pose information of a vehicle during driving; and a unit configured to correct the motion trajectory of one or more objects based on the pose information.

[0106] According to some embodiments of this disclosure, the motion trajectory correction unit may include a unit configured to perform the following operations: for each of one or more objects, based on determining that the bounding box of the object is not detected in at least one of a plurality of images, compensating for the position information of the bounding box of the object in the at least one image, based on the position information of the bounding box of the object detected in a first image and the position information of the bounding box of the object detected in a second image, to obtain a corrected motion trajectory of the object, wherein the first image is an image captured before the at least one image of the plurality of images, and the second image is an image captured after the at least one image of the plurality of images.

[0107] According to some embodiments of this disclosure, the warning unit 740 may include a unit configured to perform the following operations: for each of one or more objects, determining whether the bounding box of the object is located at the boundary of the corresponding image; determining whether to issue a warning to the vehicle based on the width change of the bounding box of the object in the corresponding image, based on the determination that the bounding box of the object is located at the boundary of the corresponding image; and determining whether to issue a warning to the vehicle based on the motion trajectory of the bounding box of the object, based on the determination that the bounding box of the object is not located at the boundary of the corresponding image.

[0108] According to some embodiments of this disclosure, the location information of one or more bounding boxes may be the midpoint of the lower boundary of the corresponding bounding box, and wherein, for each of the one or more objects, determining whether the bounding box of the object is located at the boundary of the corresponding image may include: determining that the bounding box of the object is located at the boundary of the corresponding image based on determining that at least a portion of the bounding box of the object is located outside the left or right boundary of the corresponding image; and determining that the bounding box of the object is not located at the boundary of the corresponding image based on determining that the entire bounding box of the object is located within the corresponding image.

[0109] According to some embodiments of this disclosure, determining whether to issue a vehicle warning based on the width change of the object's bounding box in the corresponding image, based on the determination that the bounding box of the object is located at the boundary of the corresponding image, may include: determining whether the width of the object's bounding box in the corresponding image is less than a preset width threshold; issuing a vehicle warning based on the determination that the width of the object's bounding box in the corresponding image is less than the preset width threshold; determining whether the object's movement trajectory is moving away from or towards the vehicle based on the determination that the object's movement trajectory is moving away from the vehicle, and not issuing a vehicle warning based on the determination that the object's movement trajectory is moving towards the vehicle; and issuing a vehicle warning when the width of the object's bounding box in the corresponding image is less than the preset width threshold based on the determination that the object's movement trajectory is moving towards the vehicle.

[0110] According to some embodiments of this disclosure, determining whether to issue a vehicle warning based on the width change of the object's bounding box in the corresponding image, based on the determination that the bounding box of the object is located at the boundary of the corresponding image, may include: determining whether the width of the object's bounding box in the corresponding image is less than a preset width threshold; determining whether the width change of the object's bounding box in the corresponding image exceeds a preset width change threshold; determining whether the width change of the object's bounding box in the corresponding image exceeds a preset width change threshold; determining whether the object's movement trajectory is moving away from or towards the vehicle; determining that the object's movement trajectory is moving away from the vehicle, and not issuing a vehicle warning; determining that the object's movement trajectory is moving towards the vehicle, and issuing a vehicle warning; and determining that the width change of the object's bounding box in the corresponding image does not exceed a preset width change threshold, and not issuing a vehicle warning.

[0111] According to some embodiments of this disclosure, the motion trajectory of the object is determined to be moving away from the vehicle based on the determination that the width of the bounding box of the object in the corresponding image increases, and the motion trajectory of the object is determined to be moving closer to the vehicle based on the determination that the width of the bounding box of the object in the corresponding image decreases.

[0112] According to some embodiments of this disclosure, determining whether to issue a warning to the vehicle based on the motion trajectory of the object's bounding box, based on the determination that the bounding box of the object is not located at the boundary of the corresponding image, may include: determining whether the distance between the object and the vehicle is less than a preset trajectory motion threshold based on the so-called position information of the object's bounding box; issuing a warning to the vehicle based on the determination that the distance between the object and the vehicle is less than the preset trajectory motion threshold; determining whether the object's motion trajectory is moving away from or towards the vehicle based on the determination that the distance between the object and the vehicle is not less than the preset trajectory motion threshold; not issuing a warning to the vehicle based on the determination that the object's motion trajectory is moving away from the vehicle; and issuing a warning to the vehicle when the distance exceeds the preset trajectory motion threshold based on the determination that the object's motion trajectory is moving towards the vehicle.

[0113] According to some embodiments of this disclosure, determining whether to issue a warning to the vehicle based on the motion trajectory of the object's bounding box, based on the determination that the object's bounding box is not located at the boundary of the corresponding image, may include: determining whether the distance between the object and the vehicle is less than a preset trajectory motion threshold based on the position information of the object's bounding box; determining whether the displacement change of the object's bounding box in the corresponding image and the preceding adjacent image exceeds a preset trajectory motion change threshold; determining whether the object's motion trajectory is moving away from or towards the vehicle; not issuing a warning to the vehicle if the object's motion trajectory is moving away from the vehicle; issuing a warning to the vehicle if the object's motion trajectory is moving towards the vehicle; and not issuing a warning to the vehicle if the object's motion trajectory is moving towards the vehicle; and not issuing a warning to the vehicle if the object's bounding box does not exceed the preset trajectory motion change threshold.

[0114] According to some embodiments of this disclosure, the preset width change threshold and / or preset trajectory motion change threshold may vary based on the distance between the object and the vehicle.

[0115] According to some embodiments of this disclosure, the warning unit 740 may include: a unit configured to determine the positional relationship between one or more objects and the vehicle; and a unit configured to issue a warning to the vehicle based on the positional relationship.

[0116] According to some embodiments of this disclosure, the positional relationship includes one or more of the following: one or more objects are located in the blind spot of the vehicle; one or more objects are located in the adjacent lane of the vehicle; or one or more objects are located behind the vehicle.

[0117] According to some embodiments of this disclosure, the vehicle may be a two-wheeled vehicle or a unicycle.

[0118] It should be understood that the various units of the vehicle warning device 700 shown in FIG. 7 correspond to the various steps in the vehicle warning methods 100 and 400 described with reference to FIG. 1 and FIG. 4. Therefore, the operations, features, and advantages described above for methods 100 and 400 also apply to the vehicle warning device 700 and its constituent units. For the sake of brevity, some operations, features, and advantages will not be repeated here.

[0119] Another aspect of this disclosure may include an electronic device that may include a memory, a processor, and a computer program stored in the memory, wherein the processor is configured to execute the computer program to implement the steps of the vehicle warning method described above.

[0120] Another aspect of this disclosure may include a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the vehicle warning method described above.

[0121] Another aspect of this disclosure may include a computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the steps of the vehicle warning method described above.

[0122] Referring to FIG8, a computing device 800 will now be described, which is an example of a hardware device that can be applied to various aspects of this disclosure. The computing device 800 can be any machine configured to perform processing and / or computation, and can be, but is not limited to, a workstation, server, desktop computer, laptop computer, tablet computer, personal digital assistant, smartphone, in-vehicle computer, access control system, time and attendance device, or any combination thereof. The aforementioned vehicle warning device can be implemented wholly or at least partially by the computing device 800 or similar devices or systems. While the computing device 800 represents one example of several types of computing platforms, the computing device 800 may include more or fewer elements and / or different element arrangements than shown in FIG8, and does not limit the scope of the claimed subject matter in these respects.

[0123] In some embodiments, computing device 800 may include elements connected to or communicating with bus 802 (possibly via one or more interfaces). For example, computing device 800 may include bus 802, one or more processors 804, one or more input devices 806, and one or more output devices 808. The one or more processors 804 may be any type of processor and may include, but are not limited to, one or more general-purpose processors and / or one or more dedicated processors (e.g., special-purpose processing chips). Input devices 806 may be any type of device capable of inputting information to computing device 800 and may include, but are not limited to, a mouse, keyboard, touchscreen, microphone, and / or remote control. Output devices 808 may be any type of device capable of presenting information and may include, but are not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Computing device 800 may also include or be connected to a non-transitory storage device 810. The non-transitory storage device can be any storage device that is non-transitory and capable of storing data, and can include, but is not limited to, disk drives, optical storage devices, solid-state storage, floppy disks, flexible disks, hard disks, magnetic tapes or any other magnetic media, optical discs or any other optical media, ROM (read-only memory), RAM (random access memory), cache memory and / or any other memory chip or cartridge, and / or any other medium from which a computer can read data, instructions and / or code. The non-transitory storage device 810 can be detached from an interface. The non-transitory storage device 810 embodies one or more non-transitory computer-readable media storing a program including instructions that, when executed by one or more processors of the computing device 800, cause the computing device 800 to perform the vehicle warning methods 100, 400 and their variations described above. The computing device 800 may also include a communication device 812. The communication device 812 can be any type of device or system that enables communication with external devices and / or with a network, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication devices and / or chipsets, such as Bluetooth™ devices, 802.11 devices, WiFi devices, WiMax devices, cellular communication devices and / or the like.

[0124] In some embodiments, the computing device 800 may also include a working memory 814, which may be any type of memory that can store programs (including instructions) and / or data useful for the operation of the processor 804, and may include, but is not limited to, random access memory and / or read-only memory devices.

[0125] The software elements (programs) may reside in the working memory 814, including but not limited to the operating system 816, one or more application programs 818, drivers, and / or other data and code. Instructions for performing the methods and steps described above may be included in one or more application programs 818, and the electronic circuitry of the vehicle warning device described above can be implemented by the processor 804 reading and executing the instructions of one or more application programs 818. The executable code or source code of the instructions of the software elements (programs) may be stored in a non-transitory computer-readable storage medium (e.g., the storage device 810 described above) and may be stored in the working memory 814 during execution (possibly compiled and / or installed). The executable code or source code of the instructions of the software elements (programs) may also be downloaded from a remote location.

[0126] It should also be understood that various modifications can be made depending on specific requirements. For example, custom hardware can also be used, and / or specific elements can be implemented using hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. For example, some or all of the disclosed methods and apparatus can be implemented by programming hardware (e.g., programmable logic circuits including field-programmable gate arrays (FPGAs) and / or programmable logic arrays (PLAs)) using logic and algorithms according to this disclosure in assembly language or hardware programming languages ​​(such as Verilog, VHDL, C++).

[0127] It should also be understood that the aforementioned methods can be implemented using a server-client model. For example, the client can use a camera to capture image data and send the image data to the server for further processing. The client can also perform a portion of the processing described above and send the resulting data to the server. The server can receive data from the client, execute the aforementioned methods or another portion thereof, and return the execution result to the client. The client can receive the execution result of the method from the server and, for example, present it to the user via an output device.

[0128] It should also be understood that the components of computing device 800 can be distributed across a network. For example, some processing can be performed using one processor, while other processing can be performed simultaneously by another processor located far away from that processor. Other components of computing device 800 can also be distributed similarly. In this way, computing device 800 can be understood as a distributed computing system that performs processing on multiple processors at multiple locations.

[0129] While embodiments or examples of this disclosure have been described with reference to the accompanying drawings, it should be understood that the methods, systems, and devices described above are merely exemplary embodiments or examples, and the scope of the invention is not limited by these embodiments or examples, but only by the granted claims and their equivalents. Various elements in the embodiments or examples may be omitted or replaced by their equivalents. Furthermore, the steps may be performed in a different order than that described in this disclosure. Further, various elements in the embodiments or examples may be combined in various ways. Importantly, as the technology evolves, many elements described herein can be replaced by equivalents that appear after this disclosure.

Claims

1. A vehicle warning method, comprising: Multiple images of the vehicle during its journey are captured via sensors located at the rear of the vehicle; For each of the plurality of images, object detection is performed to obtain one or more bounding boxes of one or more objects and the location information of the one or more bounding boxes; Based on the one or more bounding boxes and their position information, determine the motion trajectory of the one or more objects; as well as Whether to issue a warning to the vehicle is determined based on the movement trajectory of the one or more objects.

2. The method according to claim 1, further comprising: Perform object detection on each of the plurality of images to obtain the category of the one or more objects; and The determination of the motion trajectory of the one or more objects is further based on the category of the one or more objects.

3. The method of claim 1, wherein, For each of the plurality of images, object detection is performed to obtain one or more bounding boxes of one or more objects and the location information of the one or more bounding boxes, including: Based on a neural network model, target detection is performed on each of the multiple images to obtain one or more bounding boxes of one or more objects and the location information of the one or more bounding boxes.

4. The method of claim 3, wherein, Target detection for each of the plurality of images based on a neural network model includes: Target detection is performed on the multiple images based on the student model network, wherein the student model network is obtained through the following operations: Training a teacher model network, the teacher model network having a first number of parameters; and A student model network is obtained based on the teacher model network, and the student model network has a second parameter quantity that is smaller than the first parameter quantity.

5. The method of claim 3, wherein, Target detection based on a neural network model for the multiple images includes: The neural network model is quantized to convert the input data type of the neural network model into a predefined data type.

6. The method of claim 2, wherein, Determining the motion trajectory of the one or more objects based on the one or more bounding boxes and their position information includes: For each of the one or more objects in each image: Determine the confidence level between this object and each object in previously captured images; Based on the determination that the confidence level is greater than a first confidence threshold, it is determined that the object matches a corresponding object in a previously captured image. Based on determining that the confidence level is not greater than a first confidence threshold, it is then determined whether the confidence level is greater than a second confidence threshold, wherein the second confidence threshold is less than the first confidence threshold; Based on the determination that the confidence level is greater than the second confidence threshold, it is determined that the object matches a corresponding object in a previously captured image; and Based on the bounding box and position information of the object matched in the multiple images, the motion trajectory of the object is determined.

7. The method of claim 6, further comprising acquiring motion information from the sensor; and in, Determining the motion trajectory of the one or more objects based on the one or more bounding boxes and their position information further includes: For each of the one or more objects in each image, the object is determined to match a corresponding object in a previously captured image based on both the confidence level and the motion information.

8. The method according to any one of claims 1 to 7, further comprising: The motion trajectory of the one or more objects is corrected.

9. The method according to claim 8, wherein, Correcting the motion trajectory of the one or more objects includes: Capture the vehicle's pose information during driving; and The motion trajectory of the one or more objects is corrected based on the pose information.

10. The method according to claim 8, wherein, Correcting the motion trajectory of the one or more objects includes: For each of the one or more objects, based on the determination that the bounding box of the object is not detected in at least one of the plurality of images, the position information of the bounding box of the object in the at least one image is compensated based on the position information of the bounding box of the object detected in the first image and the position information of the bounding box of the object detected in the second image to obtain the corrected motion trajectory of the object. Wherein, the first image is an image captured before the at least one image among the plurality of images, and the second image is an image captured after the at least one image among the plurality of images.

11. The method according to any one of claims 1 to 7, wherein, Determining whether to issue a warning to the vehicle based on the movement trajectory of the one or more objects includes: For each of the one or more objects, determine whether the bounding box of that object is located at the boundary of the corresponding image; Based on the determination that the bounding box of the object is located at the boundary of the corresponding image, a decision is made on whether to issue a warning to the vehicle based on the width variation of the bounding box of the object in the corresponding image; and Based on the determination that the bounding box of the object is not located at the boundary of the corresponding image, a warning is issued to the vehicle based on the motion trajectory of the bounding box of the object.

12. The method according to claim 11, wherein, The position information of the one or more bounding boxes is the midpoint of the lower boundary of the corresponding bounding box, and Specifically, determining whether the bounding box of each of the one or more objects is located at the boundary of the corresponding image includes: Based on the determination that at least a portion of the object's bounding box lies outside the left or right boundary of the corresponding image, the bounding box of the object is determined to be located at the boundary of the corresponding image; and Based on the determination that the entire bounding box of the object is located within the corresponding image, it is determined that the bounding box of the object is not located at the boundary of the corresponding image.

13. The method according to claim 11, wherein, Determining whether to issue a warning to the vehicle based on the width variation of the object's bounding box within the corresponding image, based on the determined location of the object's bounding box at the boundary of the corresponding image, includes: Determine whether the width of the bounding box of the object in the corresponding image is less than a preset width threshold; Based on the determination that the width of the object's bounding box in the corresponding image is less than the preset width threshold: Provide warnings to the vehicles; and Based on the determination that the width of the bounding box of the object in the corresponding image is not less than the preset width threshold: Determine whether the object's trajectory is moving away from or towards the vehicle; Based on the determination that the object's movement trajectory is moving away from the vehicle, no warning is issued to the vehicle; and If the motion trajectory of the object is determined to be close to the vehicle, a warning is issued to the vehicle when the width of the object's bounding box in the corresponding image is less than the preset width threshold.

14. The method according to claim 11, wherein, Determining whether to issue a warning to the vehicle based on the width variation of the object's bounding box within the corresponding image, based on the determined location of the object's bounding box at the boundary of the corresponding image, includes: Determine whether the width of the bounding box of the object in the corresponding image is less than a preset width threshold; Based on the determination that the width of the bounding box of the object in the corresponding image is less than the preset width threshold: Determine whether the width change of the bounding box of the object in the corresponding image and the previous adjacent image of the corresponding image exceeds a preset width change threshold; Based on the determination that the width change of the bounding box of the object in the corresponding image and the preceding adjacent image of the corresponding image exceeds the preset width change threshold: Determine whether the object's trajectory is moving away from or towards the vehicle; Based on the determination that the object's movement trajectory is moving away from the vehicle, no warning is issued to the vehicle; and Based on the determination that the object's movement trajectory indicates it is approaching the vehicle, an early warning is issued to the vehicle. If the width change of the bounding box of the object in the corresponding image and the previous adjacent image of the corresponding image does not exceed the preset width change threshold, no warning is issued for the vehicle.

15. The method according to claim 13 or 14, wherein, Based on the determination that the width of the object's bounding box in the corresponding image increases, the motion trajectory of the object is determined to be moving away from the vehicle, and based on the determination that the width of the object's bounding box in the corresponding image decreases, the motion trajectory of the object is determined to be moving closer to the vehicle.

16. The method according to claim 11, wherein, Determining whether to issue a warning to the vehicle based on the motion trajectory of the object's bounding box, which determines that the object's bounding box is not located at the boundary of the corresponding image, includes: Based on the position information of the bounding box of the object, determine whether the distance between the object and the vehicle is less than a preset trajectory motion threshold; Based on the determination that the distance between the object and the vehicle is less than the preset trajectory movement threshold: Provide warnings to the vehicles; and Based on the determination that the distance between the object and the vehicle is not less than the preset trajectory motion threshold: Determine whether the object's trajectory is moving away from or towards the vehicle; Based on the determination that the object's movement trajectory is moving away from the vehicle, no warning is issued to the vehicle; and If the movement trajectory of the object is determined to be close to the vehicle, an early warning is issued to the vehicle when the distance exceeds the preset trajectory movement threshold.

17. The method according to claim 11, wherein, Determining whether to issue a warning to the vehicle based on the motion trajectory of the object's bounding box, which determines that the object's bounding box is not located at the boundary of the corresponding image, includes: Based on the position information of the bounding box of the object, determine whether the distance between the object and the vehicle is less than a preset trajectory motion threshold; Based on the determination that the distance between the object and the vehicle is less than the preset trajectory movement threshold: Determine whether the displacement change of the bounding box of the object in the corresponding image and the previous adjacent image of the corresponding image exceeds a preset trajectory motion change threshold; Based on the determination that the displacement change of the bounding box of the object in the corresponding image and the preceding adjacent image of the corresponding image exceeds the preset trajectory motion change threshold: Determine whether the object's trajectory is moving away from or towards the vehicle; Based on the determination that the object's movement trajectory is moving away from the vehicle, no warning is issued to the vehicle; and Based on the determination that the object's movement trajectory indicates it is approaching the vehicle, an early warning is issued to the vehicle. If the displacement change of the bounding box of the object in the corresponding image and the previous adjacent image of the corresponding image does not exceed the preset trajectory motion change threshold, no warning is issued for the vehicle.

18. The method according to claim 14 or 17, wherein, The preset width change threshold and / or the preset trajectory motion change threshold are based on the distance between the object and the vehicle.

19. The method according to any one of claims 1 to 7, wherein, Determining whether to issue a warning to the vehicle based on the movement trajectory of the one or more objects includes: Determine the positional relationship between the one or more objects and the vehicle; and The vehicle is given an early warning based on the location relationship.

20. The method according to claim 19, wherein, The positional relationship includes one or more of the following: The one or more objects are located in the blind spot of the vehicle; The one or more objects are located in the adjacent lane of the vehicle; or The one or more objects are located behind the vehicle.

21. The method according to any one of claims 1 to 7, wherein, The vehicle is a two-wheeled vehicle or a one-wheeled vehicle.

22. A vehicle warning device, comprising: The capture unit is configured to capture multiple images of the vehicle during its journey via sensors located at the rear of the vehicle. The object detection unit is configured to perform object detection for each of the plurality of images to obtain one or more bounding boxes of one or more objects and position information of the one or more bounding boxes; The motion trajectory determination unit is configured to determine the motion trajectory of the one or more objects based on the one or more bounding boxes and their position information; as well as The warning unit is configured to determine whether to issue a warning to the vehicle based on the movement trajectory of the one or more objects.

23. An electronic device, comprising: Memory, processor, and computer program stored on said memory, The processor is configured to execute the computer program to implement the steps of the method according to any one of claims 1 to 21.

24. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 21.

25. A computer program product comprising a computer program, wherein, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 21.

Images