Target speed determination method, apparatus, device, storage medium, and vehicle

Abstract

The present disclosure relates to a target speed determination method, device, equipment, storage medium and vehicle, the method comprising: acquiring image information, the image information comprising an identified target, a distance between the target and a vehicle, and a target state of the target; inputting the distance and the target state into a Kalman filter, adjusting a convergence parameter of the Kalman filter according to the target state, and outputting the target speed. The present disclosure inputs the distance between the target and the vehicle and the target state in the image information into the Kalman filter, adjusts the convergence parameter of the Kalman filter according to the target state, and outputs the target speed, thereby improving the efficiency of obtaining the target speed, providing information for timely decision of an AEB system, improving the accuracy of the target speed determination method, improving the safety of vehicle driving, ensuring the safety of cyclists or pedestrians, and improving the driving experience.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method, apparatus, device, storage medium, and vehicle for determining a target speed. Background Technology

[0002] With the advancement of technology and the improvement of people's living standards, the number of cars is constantly increasing, leading to traffic congestion and road overcrowding. Therefore, some people choose non-motorized vehicles or walking for convenience. This makes road conditions increasingly complex, and drivers may encounter cyclists or pedestrians. To ensure the safety of cyclists or pedestrians, the Automatic Emergency Braking (AEB) system adjusts the vehicle's speed in a timely manner based on the speed of the pedestrian or cyclist to avoid accidents and improve the driving experience.

[0003] Normally, vehicles use filters to measure the speed of pedestrians or cyclists. However, in some special situations, such as pedestrians or cyclists crossing the road or entering the frame, the speed of the target cannot be obtained quickly. As a result, the AEB system cannot make timely decisions and cannot ensure the safety of cyclists or pedestrians. Summary of the Invention

[0004] To solve the above-mentioned technical problems, or at least partially solve them, this disclosure provides a target speed determination method, apparatus, device, storage medium, and vehicle to ensure the safety of cyclists or pedestrians and improve the driving experience of drivers.

[0005] In a first aspect, embodiments of this disclosure provide a method for determining a target speed, including:

[0006] Acquire image information, which includes the identified target, the distance between the target and the vehicle, and the target status;

[0007] The distance and the target state are input into a Kalman filter. The convergence parameters of the Kalman filter are adjusted according to the target state, and the target speed is output. The convergence parameters of the Kalman filter include the speed difference between the target speed and the vehicle speed, and the Kalman filter noise. The target speed includes the absolute speed of the target relative to the ground and the relative speed of the target relative to the vehicle.

[0008] In some embodiments, obtaining image information includes:

[0009] The image information contains identified targets, including pedestrians and cyclists.

[0010] In some embodiments, obtaining image information includes:

[0011] Obtain the distance between the target and the vehicle as shown in the image information;

[0012] The image information is detected based on the trained detection model to obtain a target detection box for the target, which includes a pedestrian detection box or a cyclist detection box.

[0013] Based on the pixel coordinates and pixel height of the target detection box in the image information, the distance between the target and the vehicle is determined.

[0014] In some embodiments, obtaining image information includes:

[0015] Obtain the target state of the target in the image information;

[0016] The image information is detected based on the trained detection model to obtain a target detection box for the target, which includes a pedestrian detection box or a cyclist detection box.

[0017] The target state is identified based on the pixel coordinates of the target detection box in the image information.

[0018] In some embodiments, determining the distance between the target and the vehicle based on the pixel coordinates and pixel height of the target detection bounding box in the image information includes:

[0019] Based on the pixel coordinates of the target detection box in the image information, the first distance between the target and the vehicle is determined by inverse perspective transformation;

[0020] Based on the pixel height of the target detection box in the image information, the second distance between the target and the vehicle is determined according to the pinhole imaging principle;

[0021] The smaller of the first distance and the second distance is determined to be the distance between the target and the vehicle.

[0022] In some embodiments, identifying the target state based on the pixel coordinates of the target detection box in the image information includes:

[0023] The image information is detected based on the trained detection model to obtain other detection boxes for targets other than the target.

[0024] Based on the pixel coordinates of the target detection box and the other detection boxes in the image information, the relative positional relationship between the target and the other targets is determined;

[0025] Based on the correspondence between the relative positional relationship and the state, the target state is identified.

[0026] In some embodiments, the target state includes at least one of the following: occlusion state, cut-in state, cut-out state, road center state, image edge state, long distance state, cross-traffic state, and near the front of the vehicle state.

[0027] In some embodiments, the distance and the target state are input into a Kalman filter, the convergence parameters of the Kalman filter are adjusted according to the target state, and the target velocity is output, including:

[0028] When the target state is in the cutting-in state or the cutting-out state, the vehicle speed is obtained, and the target initial speed is obtained based on the distance and the vehicle speed.

[0029] Adjust the velocity difference value of the Kalman filter, and determine the target velocity based on the initial target velocity and the velocity difference value.

[0030] In some embodiments, the distance and the target state are input into a Kalman filter, the convergence parameters of the Kalman filter are adjusted according to the target state, and the target velocity is output, including:

[0031] When the target state is any state other than the cut-in state and the cut-out state, select the Kalman filter noise corresponding to the target state;

[0032] The target velocity is output based on the distance, the target state, and the Kalman filter noise corresponding to the target state.

[0033] In a second aspect, embodiments of this disclosure provide a target speed determination device, comprising:

[0034] The acquisition module is used to acquire image information, including the identified target, the distance between the target and the vehicle, and the target status;

[0035] The output module is used to input the distance and the target state into a Kalman filter, adjust the convergence parameters of the Kalman filter according to the target state, and output the target speed. The convergence parameters of the Kalman filter include the speed difference between the target speed and the vehicle speed, and the Kalman filter noise. The target speed includes the absolute speed of the target relative to the ground and the relative speed of the target relative to the vehicle.

[0036] Thirdly, embodiments of this disclosure provide an electronic device, including:

[0037] Memory;

[0038] Processor; and

[0039] Computer programs;

[0040] The computer program is stored in the memory and configured to be executed by the processor to implement the method as described in the first aspect.

[0041] Fourthly, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the method described in the first aspect.

[0042] Fifthly, embodiments of this disclosure also provide a computer program product, which includes a computer program or instructions that, when executed by a processor, implement the method described in the first aspect.

[0043] In a sixth aspect, embodiments of this disclosure also provide a vehicle, including: a target speed determining device as described in the second aspect; or an electronic device as described in the third aspect; or a computer-readable storage medium as described in the fourth aspect.

[0044] The target speed determination method, apparatus, device, storage medium, and vehicle provided in this disclosure improve the efficiency of acquiring target speed by inputting the distance between the target and the vehicle and the target state from the image information into a Kalman filter, adjusting the convergence parameters of the Kalman filter according to the target state, and outputting the target speed. This provides information for timely decision-making by the AEB system, improves the accuracy of the target speed determination method, enhances vehicle driving safety, protects the safety of cyclists or pedestrians, and improves the driving experience. Attached Figure Description

[0045] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0046] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 A flowchart of the target speed determination method provided in this embodiment of the disclosure;

[0048] Figure 2 A flowchart of the target speed determination method provided in this embodiment of the disclosure;

[0049] Figure 3 A flowchart of the distance detection method provided in this embodiment of the disclosure;

[0050] Figure 4 This is a flowchart of a target state recognition method provided in an embodiment of the present disclosure;

[0051] Figure 5 A flowchart of the target speed determination method provided in this embodiment of the disclosure;

[0052] Figure 6 This is a schematic diagram of the target speed determination device provided in an embodiment of the present disclosure;

[0053] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0054] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0055] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0056] This disclosure provides a method for determining a target velocity, which will be described below with reference to specific embodiments.

[0057] Figure 1 This is a flowchart illustrating a target speed determination method provided in an embodiment of this disclosure. The method can be executed by a target speed determination device, which can be implemented in software and / or hardware. This device can be configured in an electronic device, such as a server or terminal, where the terminal specifically includes in-vehicle systems, vehicles, etc. Furthermore, this method can be applied to application scenarios involving determining target speeds, such as determining pedestrian speeds, determining cyclist speeds, etc. It is understood that the target speed determination method provided in this embodiment can also be applied to other scenarios. The following describes... Figure 1 The method for determining the target velocity is described below. The specific steps of this method are as follows:

[0058] S101. Acquire image information, the image information including the identified target, the distance between the target and the vehicle, and the target status of the target.

[0059] The vehicle is equipped with a vehicle-mounted infotainment system and camera equipment. The infotainment system and camera equipment are connected via communication methods, such as wired, Bluetooth, wireless network, fiber optic cable, etc. The camera equipment includes radar, cameras, etc.

[0060] The vehicle's infotainment system acquires real-time images of the vehicle's surroundings while it is in motion, specifically images of the environment within a 120° radius in front of the vehicle. These images include identified targets, the distance between the targets and the vehicle, and the target's status.

[0061] S102. Input the distance and the target state into a Kalman filter, adjust the convergence parameters of the Kalman filter according to the target state, and output the target speed. The convergence parameters of the Kalman filter include the speed difference between the target speed and the vehicle speed, and the Kalman filter noise. The target speed includes the absolute speed of the target relative to the ground and the relative speed of the target relative to the vehicle.

[0062] The vehicle's infotainment system inputs the distance between the target and the vehicle, and the target's state, obtained from the aforementioned image information, into a Kalman filter. Based on different target states, it selects different convergence parameters for the Kalman filter. These convergence parameters include the speed difference between the target and the vehicle's speed, and the Kalman filter noise. The output is the target speed, which includes the target's absolute speed relative to the ground and its relative speed to the vehicle. It is understandable that because pedestrians or cyclists move at relatively uniform speeds, the speed difference between the target and the vehicle's speed will increase as the vehicle's speed increases.

[0063] Optionally, the less accurate the data obtained by visual observation, the greater the noise of the Kalman filter.

[0064] This embodiment of the present disclosure improves the efficiency of obtaining target speed by inputting the distance between the target and the vehicle and the target state from the image information into a Kalman filter, adjusting the convergence parameters of the Kalman filter according to the target state, and outputting the target speed. This provides information for timely decision-making by the AEB system, improves the accuracy of the target speed determination method, enhances vehicle driving safety, protects the safety of cyclists or pedestrians, and improves the driving experience.

[0065] Figure 2 A flowchart of a target speed determination method provided in this disclosure embodiment. The specific steps of the method are as follows:

[0066] S201. Obtain image information.

[0067] The vehicle is equipped with a vehicle-mounted infotainment system and camera equipment. The infotainment system and camera equipment are connected via communication methods, such as wired, Bluetooth, wireless network, fiber optic cable, etc. The camera equipment includes radar, cameras, etc.

[0068] The vehicle's infotainment system uses camera equipment to capture real-time images of the vehicle's surroundings while it is in motion, specifically images of the environment within a 120° radius in front of the vehicle.

[0069] S202. Detect the image information based on the trained detection model to obtain target detection boxes for the target, wherein the target includes pedestrians or cyclists, and the target detection boxes include pedestrian detection boxes or cyclist detection boxes.

[0070] The target can be a pre-defined target within the image acquisition range in front of the vehicle, such as a pedestrian or cyclist. The target detection box for the target is the bounding box of the detected target in the image information, and its shape is generally rectangular. In other embodiments, it can also be other shapes, and this embodiment does not make a specific limitation.

[0071] The height of pedestrians is idealized and obtained by discretizing the height of pedestrians. That is, the height should be continuous. In this embodiment, a height corresponding to an age group is set as the height template of pedestrians for detection training. That is, the image information labeled with the height template is input into the detection model to train the detection model and obtain the trained detection model.

[0072] The height of the cyclist is idealized. The cyclist can be understood as someone pushing or riding a non-motorized vehicle or a two-wheeled motorcycle. That is, the height of the cyclist is set as a fixed value, which is used as the height template of the cyclist. The image information labeled with this height template is input into the detection model to train the detection model and obtain the trained detection model.

[0073] Understandably, multiple detection models can be built and trained separately to obtain image information of multiple height templates and height templates as an image sample set. Based on this image sample set, multiple detection models can be used for detection, and the model with the best detection performance can be selected.

[0074] The aforementioned image information is input into a trained detection model, such as a convolutional neural network, to obtain the output of the detection model, which is the target detection box for the target in the image information. The target includes pedestrians or cyclists, and the target detection box includes either a pedestrian detection box or a cyclist detection box. A pedestrian detection box can be understood as the smallest bounding rectangle of a pedestrian in the image information, and a cyclist detection box can be understood as the smallest bounding rectangle of a cyclist in the image. It is understood that the image information may include multiple different targets, and the number of each target may not be unique. Therefore, the image information may include one or more target detection boxes. This embodiment only uses one target detection box as an example for explanation.

[0075] S203. Based on the pixel coordinates and pixel height of the target detection box in the image information, determine the distance between the target and the vehicle.

[0076] The pixel coordinates of the target detection box are used to reflect the position of the target detection box in the image information, that is, the coordinate position of the pixel corresponding to a point on the target detection box or a point inside the target detection box in the image information.

[0077] Optionally, the pixel coordinates of the intersection of the diagonals of the target detection box in the image information can be used as the pixel coordinates of the target detection box in the image information. It is understood that in other embodiments, the pixel coordinates of the target detection box or other points in the target detection box can also be selected as the pixel coordinates of the target detection box in the image information. For example, the selected points can be the midpoint of the bottom edge of the target detection box, the midpoint of the left side of the target detection box, the lower left corner vertex of the target detection box, the lower right corner vertex of the target detection box, and other points.

[0078] Pixel height refers to the physical distance that represents one pixel on an image or screen.

[0079] The vehicle's system determines the distance between the target and the vehicle based on the pixel coordinates and pixel height of the target detection box in the aforementioned image information, combined with camera parameters and the vehicle's position.

[0080] S204. Based on the pixel coordinates of the target detection box in the image information, identify the target state.

[0081] The vehicle's infotainment system identifies the target's state based on the pixel coordinates of the target detection box in the aforementioned image information.

[0082] Specifically, the vehicle's infotainment system acquires a pre-calibrated sample dataset, inputs this dataset into a motion state model for training, and obtains a trained motion state model. Based on the pixel coordinates of the target detection box in the aforementioned image information, the system uses the trained motion state model to determine the target's motion state, thereby determining the target's state. It is understandable that the vehicle's infotainment system can also acquire the target state through other methods.

[0083] S205. Input the distance and the target state into a Kalman filter, adjust the convergence parameters of the Kalman filter according to the target state, and output the target speed. The convergence parameters of the Kalman filter include the speed difference between the target speed and the vehicle speed, and the Kalman filter noise. The target speed includes the absolute speed of the target relative to the ground and the relative speed of the target relative to the vehicle.

[0084] Kalman filtering is an algorithm that uses the state equations of a linear system to optimally estimate the system state using observed input and output data. Since the observed data includes noise and interference from the system, the optimal estimation can also be viewed as a filtering process. Data filtering is a data processing technique that removes noise and restores accurate data. Kalman filtering, given known measurement variance, can estimate the state of a dynamic system from a series of data containing measurement noise. Because it is easy to implement in computer programming and can update and process field-acquired data in real time, Kalman filtering is currently the most widely used filtering method and has found good applications in communication, navigation, guidance, and control, among other fields.

[0085] The vehicle's infotainment system inputs the aforementioned distance and target state into a Kalman filter. Based on different target states, it selects different convergence parameters for the Kalman filter and outputs the target speed. These convergence parameters include the speed difference between the target speed and the vehicle's speed, and the Kalman filter noise. The target speed includes the target's absolute speed relative to the ground and its relative speed to the vehicle. It is understandable that because pedestrians or cyclists move at relatively uniform speeds, the speed difference between the target speed and the vehicle's speed will increase as the vehicle's speed increases.

[0086] Optionally, the less accurate the data obtained by visual observation, the greater the noise of the Kalman filter.

[0087] This embodiment acquires image information; it then detects the image information using a trained detection model to obtain target detection boxes, which include pedestrians or cyclists. These target detection boxes provide a data foundation for determining the distance between the target and the vehicle and for identifying the target's state. Based on the pixel coordinates and pixel height of the target detection boxes in the image information, the distance between the target and the vehicle is determined, providing a data foundation for outputting the target's speed. Based on the pixel coordinates of the target detection boxes in the image information, the target's state is identified using a motion state model, providing a data foundation for outputting the target's speed. Finally, the distance and the target's state are input into a Kalman scattering model. The filter adjusts the convergence parameters of the Kalman filter according to the target state and outputs the target speed. The convergence parameters of the Kalman filter include the speed difference between the target speed and the vehicle speed, and the Kalman filter noise. The target speed includes the absolute speed of the target relative to the ground and the relative speed of the target relative to the vehicle. By clearly defining the target speed, the AEB system can adjust the vehicle speed in a timely manner based on the target speed. It has a wide range of applications and is easy to implement, improving the accuracy of the target speed determination method. At the same time, it provides reliable speed data for the AEB system, facilitating timely decision-making by the AEB system, improving vehicle driving safety, ensuring the safety of cyclists or pedestrians, and enhancing the driving experience.

[0088] Figure 3 A flowchart of the distance determination method provided in the embodiments of this disclosure is shown below. Figure 2 As shown, the specific steps included in this method are as follows:

[0089] S301, Obtain image information.

[0090] Specifically, the implementation process and principle of S301 and S201 are the same, and will not be repeated here.

[0091] S302. Detect the image information based on the trained detection model to obtain target detection boxes for the target, wherein the target includes pedestrians or cyclists, and the target detection boxes include pedestrian detection boxes or cyclist detection boxes.

[0092] Specifically, the implementation process and principle of S302 and S202 are the same, and will not be repeated here.

[0093] S303. Based on the pixel coordinates of the target detection box in the image information, the first distance between the target and the vehicle is determined by inverse perspective transformation.

[0094] Inverse perspective transformation is the reverse process of perspective transformation. It mainly combines camera parameters to map a point in the image information from the image coordinate system to the world coordinate system or a coordinate system with the vehicle as the origin, thereby eliminating the interference and error of perspective on image detection and recognition tasks.

[0095] Camera calibration parameters include the camera's intrinsic and extrinsic parameters. Intrinsic parameters include the camera's focal length and pixel size, while extrinsic parameters include the camera's position and pitch angle.

[0096] The vehicle-mounted system determines the inverse perspective transformation matrix of the target detection box based on the camera calibration parameters. Based on the inverse perspective transformation matrix, it applies inverse perspective transformation to the pixel coordinates of the target detection box in the image information to obtain the correspondence between the image coordinate system and the world coordinate system or the vehicle coordinate system (i.e., the coordinate system with the vehicle as the origin). It then performs a one-to-one correspondence between the pixel coordinates of the target detection box in the image coordinate system and the world coordinate system or the vehicle coordinate system to obtain the position of the target detection box in the world coordinate system or the vehicle coordinate system. Based on the position of the target in the vehicle coordinate system and the position information of the vehicle, the first distance between the target and the vehicle can be calculated.

[0097] S304. Based on the pixel height of the target detection box in the image information, determine the second distance between the target and the vehicle according to the pinhole imaging principle.

[0098] The system acquires the real-world height data of the target using a visual sensor and applies Kalman filtering to obtain the target's real-world height. Simultaneously, it detects the pixel height of the initial detection box corresponding to the target in the original image. Based on the pixel height of the initial detection box in the original image, the target's real-world height, and the camera's focal length, a second distance from the target to the vehicle is determined according to the pinhole imaging principle.

[0099] S305. Determine the smaller of the first distance and the second distance as the distance between the target and the vehicle.

[0100] Based on the second distances obtained in S303 and S304, the vehicle's system determines the smaller of the first and second distances as the distance between the target and the vehicle. It is understood that since two different ranging methods are used in the above steps, the first and second distances obtained may be different or the same. When the first and second distances are different, the smaller of the first and second distances is selected as the distance between the target and the vehicle. When the first and second distances are the same, either the first or second distance is selected as the distance between the target and the vehicle to fully ensure the driving safety of the vehicle.

[0101] This disclosure embodiment detects the distance between the target and the vehicle using different methods, and selects the smaller value among different ranging results as the distance between the target and the vehicle. This avoids unstable detection results due to errors in the ranging system or environmental influences, providing more accurate data for subsequent determination of the target speed and improving vehicle driving safety.

[0102] Figure 4 This is a flowchart of the target state recognition method provided in the embodiments of this disclosure, such as... Figure 4 As shown, the specific steps included in this method are as follows:

[0103] S401, Obtain image information.

[0104] Specifically, the implementation process and principle of S401 and S201 are the same, and will not be repeated here.

[0105] S402. Detect the image information based on the trained detection model to obtain target detection boxes for the target, wherein the target includes pedestrians or cyclists, and the target detection boxes include pedestrian detection boxes or cyclist detection boxes.

[0106] Specifically, the implementation process and principle of S402 and S202 are the same, and will not be repeated here.

[0107] S403. Detect the image information based on the trained detection model to obtain other detection boxes for targets other than the target.

[0108] The vehicle's system inputs image information into a trained detection model. The trained detection model can identify other targets in the image information besides the targets mentioned above, that is, targets other than pedestrians and cyclists, such as fences, other vehicles, etc., and assign other targets besides the targets mentioned above to other detection boxes.

[0109] S404. Based on the pixel coordinates of the target detection box and the other detection boxes in the image information, determine the relative positional relationship between the target and the object, and between the target and the vehicle.

[0110] After obtaining the target detection box and other detection boxes, the vehicle's system can determine the relative positional relationship between the target and the object, as well as between the target and the vehicle, based on the pixel coordinates of the target detection box and other detection boxes in the image information.

[0111] Optionally, after obtaining the target detection box and other detection boxes, the vehicle system can determine the size of the target detection box and other detection boxes in the image information based on their pixel coordinates. The size of the target detection box and other detection boxes can be compared to determine the relative positional relationship between the target and the object. This relative positional relationship includes whether the target is behind other cyclists, behind a fence, behind other vehicles, or whether the target is within the lane.

[0112] Optionally, after obtaining the target detection box and other detection boxes, the vehicle system can, based on the pixel coordinates of the target detection box and other detection boxes in the image information, project and transform the pixel coordinates of the target detection box in the image information to the vehicle's coordinate system to obtain the projected coordinates of the target detection box in the vehicle coordinate system; based on the projected coordinates of the target detection box in the vehicle coordinate system, the relative positional relationship between the target and the vehicle can be determined, including the target being close to the front of the vehicle and the target being at a distance from the vehicle.

[0113] Optionally, after obtaining the target detection box and other detection boxes, the vehicle system can determine the left distance between the left boundary of the target detection box and the left edge of the image information, and the right distance between the right boundary of the target detection box and the right edge of the road image, based on the pixel coordinates of the target detection box and other detection boxes in the image information. If the left or right distance is less than a preset distance threshold, the target detection box is determined to be at the edge of the road image, and the target state is determined to be the image edge state.

[0114] Optionally, the target's walking direction is determined based on image information and historical image information; if the angle between the walking direction and the vehicle's driving direction is between 0 and 180 degrees, the target's walking direction is determined to intersect with the vehicle's driving direction; based on the intersection of the target's walking direction and the vehicle's driving direction, the target's state is determined to be the cutting-in state.

[0115] S405. Based on the correspondence between the relative positional relationship and the state, identify the target state.

[0116] The vehicle's infotainment system stores a correspondence between relative positional relationships and target states. After obtaining the relative positional relationships between the target and objects, and between the target and the vehicle, the system can determine the target state corresponding to the relative positional relationship based on this correspondence. For example, when the target is behind an object, the system can determine that the target's state is that it is occluded by that object.

[0117] Optionally, the target state includes at least one of the following: occlusion state, cutting in state, cutting out state, road center state, image edge state, long distance state, crossing state, and approaching the front of the vehicle state.

[0118] This disclosure improves the accuracy of target state identification by specifically describing the types of target states, which can provide more target state information for the automatic braking system and improve vehicle driving safety.

[0119] Figure 5 A flowchart of the target speed determination method provided in the embodiments of this disclosure is shown below. Figure 5 As shown, the specific steps included in this method are as follows:

[0120] S501, Obtain image information.

[0121] Specifically, the implementation process and principle of S501 and S201 are the same, and will not be repeated here.

[0122] S502. Detect the image information based on the trained detection model to obtain target detection boxes for the target, wherein the target includes pedestrians or cyclists, and the target detection boxes include pedestrian detection boxes or cyclist detection boxes.

[0123] Specifically, the implementation process and principle of S502 and S202 are the same, and will not be repeated here.

[0124] S503. Based on the pixel coordinates and pixel height of the target detection box in the image information, determine the distance between the target and the vehicle.

[0125] Specifically, the implementation process and principle of S503 and S203 are the same, and will not be repeated here.

[0126] S504. Based on the pixel coordinates of the target detection box in the image information, identify the target state.

[0127] The vehicle's infotainment system identifies the target's state based on the pixel coordinates of the target detection box in the aforementioned image information.

[0128] Specifically, the vehicle's infotainment system acquires a pre-calibrated sample dataset, inputs this dataset into a motion state model for training, and obtains a trained motion state model. Based on the pixel coordinates of the target detection box in the aforementioned image information, the system uses the trained motion state model to determine the target's motion state, thereby determining the target's state. It is understandable that the vehicle's infotainment system can also acquire the target state through other methods.

[0129] Optionally, the target state includes at least one of the following: occlusion state, cutting in state, cutting out state, road center state, image edge state, long distance state, crossing state, and approaching the front of the vehicle state.

[0130] S505. When the target state is in the cutting-in state or the cutting-out state, obtain the vehicle speed and obtain the target initial speed based on the distance and the vehicle speed.

[0131] When the target is in the cutting state, it is assumed that the current target speed is greater than the vehicle speed. The vehicle speed is converged and obtained. Based on the distance between the target and the vehicle in any two frames of image information and the time difference between the two frames of image information, the speed difference between the target speed and the vehicle speed is determined. The vehicle speed is added to the speed difference to obtain the target initial speed. The target initial speed will increase as the vehicle speed increases.

[0132] When the target state is cut off, it is assumed that the current target vehicle speed is less than the vehicle speed. The vehicle speed is converged and obtained. Based on the distance between the target and the vehicle in any two frames of image information and the time difference between the two frames of image information, the speed difference between the target speed and the vehicle speed is determined. The vehicle speed is subtracted from the speed difference to obtain the target initial speed. The target initial speed will increase as the vehicle speed increases.

[0133] S506. Adjust the velocity difference of the Kalman filter, and determine the target velocity based on the initial target velocity and the velocity difference.

[0134] Before filtering with a Kalman filter, it is necessary to pre-initialize the Kalman filter to speed up the convergence of the filter speed measurement. The specific function for initializing the filter can be the LineLeastForLateralSpeed ​​function.

[0135] The initial target velocity and velocity difference are input into a Kalman filter. The velocity difference value of the Kalman filter is adjusted to determine the target velocity. It can be understood that the output target velocity can be the target's absolute velocity relative to the ground and its relative velocity relative to the vehicle; it can also be the output target's absolute velocity relative to the ground, from which the relative velocity is calculated; or it can be the output target's relative velocity relative to the vehicle, from which the absolute velocity is calculated.

[0136] Specifically, the weighted least squares method is used to fit the Video Comparison Summary (VCS) distance of the first 8–16 frames of the target observation to obtain an approximate VCS velocity. The fitting points are a minimum of 8 frames and a maximum of 16 frames. If the autocorrelation coefficient in the first 8–16 frames is higher than 0.6, the fitting is stopped, and the longitudinal velocity of the Kalman filter is reset using the fitted VCS velocity.

[0137]

[0138] State variables of the Kalman master filter It is four-dimensional, in which, Let X be the velocity in the world coordinate system. Let x be the velocity in the Y direction in the world coordinate system, and let x and y be the X and Y coordinates in the world coordinate system, respectively.

[0139] The state transition equation F of the Kalman filter is:

[0140]

[0141] The observation Z of the Kalman filter has two dimensions: the height h of the detection box and the x-coordinate of the midpoint of the bottom edge in the image domain. The observation equation is:

[0142]

[0143]

[0144]

[0145] Where Px is the X coordinate of the state variable, Py is the Y coordinate of the state variable, and center_x is the X coordinate of the camera vanishing point in the image domain.

[0146] The state variable X of the pedestrian acceleration Kalman filter is a four-dimensional vector, consisting of X-direction acceleration, X-direction velocity, Y-direction acceleration, and Y-direction velocity:

[0147]

[0148] The state transition equation F is:

[0149]

[0150] The observation vector Z of the Kalman filter is a 2-dimensional vector, representing the velocities in the X and Y directions:

[0151]

[0152] S507. When the target state is any state other than the cut-in state and the cut-out state, select the Kalman filter noise corresponding to the target state.

[0153] When the target state is any state other than the cut-in state and the cut-out state, that is, the target state is the center of the road state, the edge of the image state, the far distance state, the crossing state, or the state close to the front of the vehicle, the Kalman filter noise corresponding to the target state is selected.

[0154] S508. Based on the distance, the target state, and the Kalman filter noise corresponding to the target state, output the target velocity.

[0155] The distance and target state are input into the Kalman filter, and the Kalman filter noise corresponding to the target state is selected to output the target velocity. It can be understood that the more inaccurate the human observation or the greater the degree of occlusion of the target in the image information, the greater the Kalman noise.

[0156] Specifically, the weighted least squares method is used to fit the Video Comparison Summary (VCS) distance of the first 8–16 frames of the target observation to obtain an approximate VCS velocity. The fitting points are a minimum of 8 frames and a maximum of 16 frames. If the autocorrelation coefficient in the first 8–16 frames is higher than 0.6, the fitting is stopped, and the longitudinal velocity of the Kalman filter is reset using the fitted VCS velocity.

[0157]

[0158] State variables of the Kalman master filter For four dimensions, Let X be the velocity in the world coordinate system. Let x be the velocity in the Y direction in the world coordinate system, and let x and y be the X and Y coordinates in the world coordinate system, respectively.

[0159] The state transition equation F of the Kalman filter is:

[0160]

[0161] The observation Z of the Kalman filter has two dimensions: the height h of the detection box and the x-coordinate of the midpoint of the bottom edge in the image domain. The observation equation is:

[0162]

[0163]

[0164]

[0165] Where Px is the X coordinate of the state variable, Py is the Y coordinate of the state variable, and center_x is the X coordinate of the camera vanishing point in the image domain.

[0166] The state variable X of the pedestrian acceleration Kalman filter is a four-dimensional vector, consisting of X-direction acceleration, X-direction velocity, Y-direction acceleration, and Y-direction velocity:

[0167]

[0168] The state transition equation F is:

[0169]

[0170] The observation vector Z of the Kalman filter is a 2-dimensional vector, representing the velocities in the X and Y directions:

[0171]

[0172] This disclosure describes in detail how the distance between the target and the vehicle, and the target state are input into a Kalman filter to output the target speed. It specifically states the speed of the target in each state, thereby improving the applicability of the target speed determination method and enhancing vehicle driving safety.

[0173] Figure 6 This is a schematic diagram of the target speed determination device provided in the embodiments of this disclosure. The target speed determination device may be a vehicle infotainment system as described in the above embodiments, or it may be a component or assembly within the vehicle infotainment system. The target speed determination device provided in the embodiments of this disclosure can execute the processing flow provided in the target speed determination method embodiments, such as... Figure 6As shown, the target speed determination device 60 includes: an acquisition module 61 and an output module 62; wherein, the acquisition module 61 is used to acquire image information, the image information including the identified target, the distance between the target and the vehicle, and the target state of the target; the output module 62 is used to input the distance and the target state into a Kalman filter, adjust the convergence parameters of the Kalman filter according to the target state, and output the target speed, the convergence parameters of the Kalman filter including the speed difference between the target speed and the vehicle speed, and Kalman filter noise, the target speed including the absolute speed of the target relative to the ground and the relative speed of the target relative to the vehicle.

[0174] Optionally, the acquisition module 61 is also used to acquire identified targets in the image information, including pedestrians and cyclists.

[0175] Optionally, the acquisition module 61 is further configured to acquire the distance between the target and the vehicle in the image information; detect the image information based on the trained detection model to acquire a target detection box for the target, the target detection box including a pedestrian detection box or a cyclist detection box; and determine the distance between the target and the vehicle based on the pixel coordinates and pixel height of the target detection box in the image information.

[0176] Optionally, the acquisition module 61 is further configured to acquire the target state of the target in the image information; and to identify the target state based on the pixel coordinates of the target detection box in the image information.

[0177] Optionally, the acquisition module 61 is further configured to determine a first distance between the target and the vehicle based on the pixel coordinates of the target detection box in the image information through inverse perspective transformation; determine a second distance between the target and the vehicle based on the pixel height of the target detection box in the image information according to the pinhole imaging principle; and determine the smaller of the first distance and the second distance as the distance between the target and the vehicle.

[0178] Optionally, the acquisition module 61 is further configured to detect the image information based on the trained detection model, and acquire other detection boxes for targets other than the target; determine the relative positional relationship between the target and the other targets based on the pixel coordinates of the target detection box and the other detection boxes in the image information; and identify the target state based on the correspondence between the relative positional relationship and the state.

[0179] Optionally, the target state includes at least one of the following: occlusion state, cutting in state, cutting out state, road center state, image edge state, long distance state, crossing state, and approaching the front of the vehicle state.

[0180] Optionally, the output module 62 is further configured to: obtain the vehicle speed when the target state is in a cut-in state or a cut-out state; obtain the target initial speed based on the distance and the vehicle speed; adjust the speed difference value of the Kalman filter; and determine the target speed based on the target initial speed and the speed difference value.

[0181] Optionally, the output module 62 is further configured to select a Kalman filter noise corresponding to the target state when the target state is in a state other than the cut-in state and the cut-out state, and output the target speed using the Kalman filter noise corresponding to the state.

[0182] Figure 6 The target speed determination device shown in the embodiment can be used to execute the technical solution of the above-described target speed determination method embodiment. Its implementation principle and technical effect are similar, and will not be repeated here.

[0183] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. The electronic device may be a terminal as described in the above embodiments. The electronic device provided in this embodiment of the present disclosure can execute the processing flow provided in the target speed determination method embodiment, such as... Figure 7 As shown, the electronic device 70 includes: a memory 71, a processor 72, a computer program, and a communication interface 73; wherein the computer program is stored in the memory 71 and configured to be executed by the processor 72 using the target speed determination method as described above.

[0184] In addition, this disclosure also provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the target speed determination method described in the above embodiments.

[0185] Furthermore, this disclosure also provides a computer program product, which includes a computer program or instructions that, when executed by a processor, implement the target speed determination method as described above.

[0186] In addition, this disclosure also provides a vehicle that includes a target speed determining device as described in the above embodiments; or an electronic device as described in the above embodiments; or a computer-readable storage medium as described in the above embodiments.

[0187] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0188] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0189] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0190] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to:

[0191] Acquire image information, which includes the identified target, the distance between the target and the vehicle, and the target status;

[0192] The distance and the target state are input into a Kalman filter. The convergence parameters of the Kalman filter are adjusted according to the target state, and the target speed is output. The convergence parameters of the Kalman filter include the speed difference between the target speed and the vehicle speed, and the Kalman filter noise. The target speed includes the absolute speed of the target relative to the ground and the relative speed of the target relative to the vehicle.

[0193] In addition, the electronic device can also perform other steps in the target velocity determination method described above.

[0194] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0195] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0196] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0197] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0198] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0199] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0200] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for determining a target velocity, characterized in that, The method includes: Acquire image information, which includes the identified target, the distance between the target and the vehicle, and the target state, wherein the target state includes at least one of the following: occlusion state, cutting in state, cutting out state, road center state, image edge state, long distance state, crossing state, and approaching the front of the vehicle state. The distance and the target state are input into a Kalman filter. The convergence parameters of the Kalman filter are adjusted according to the target state, and the target speed is output. The convergence parameters of the Kalman filter include the speed difference between the target speed and the vehicle speed, and the Kalman filter noise. The target speed includes the absolute speed of the target relative to the ground and the relative speed of the target relative to the vehicle. The distance and the target state are input into a Kalman filter, and the convergence parameters of the Kalman filter are adjusted according to the target state to output the target velocity, including: When the target state is in the cutting-in state or the cutting-out state, the vehicle speed is obtained, and the target initial speed is obtained based on the distance and the vehicle speed. Adjust the velocity difference value of the Kalman filter, and determine the target velocity based on the initial target velocity and the velocity difference value.

2. The method according to claim 1, characterized in that, Acquire image information, including: The image information contains identified targets, including pedestrians and cyclists.

3. The method according to claim 1, characterized in that, Acquire image information, including: Obtain the distance between the target and the vehicle as shown in the image information; The image information is detected based on the trained detection model to obtain a target detection box for the target, which includes a pedestrian detection box or a cyclist detection box. Based on the pixel coordinates and pixel height of the target detection box in the image information, the distance between the target and the vehicle is determined.

4. The method according to claim 1, characterized in that, Acquire image information, including: Obtain the target state of the target in the image information; The image information is detected based on the trained detection model to obtain a target detection box for the target, which includes a pedestrian detection box or a cyclist detection box. The target state is identified based on the pixel coordinates of the target detection box in the image information.

5. The method according to claim 3, characterized in that, Based on the pixel coordinates and pixel height of the target detection box in the image information, the distance between the target and the vehicle is determined, including: Based on the pixel coordinates of the target detection box in the image information, the first distance between the target and the vehicle is determined by inverse perspective transformation; Based on the pixel height of the target detection box in the image information, the second distance between the target and the vehicle is determined according to the pinhole imaging principle; The smaller of the first distance and the second distance is determined to be the distance between the target and the vehicle.

6. The method according to claim 4, characterized in that, Based on the pixel coordinates of the target detection box in the image information, the target state is identified, including: The image information is detected based on the trained detection model to obtain other detection boxes for targets other than the target. Based on the pixel coordinates of the target detection box and the other detection boxes in the image information, the relative positional relationship between the target and the other targets is determined; Based on the correspondence between the relative positional relationship and the state, the target state is identified.

7. The method according to claim 1, characterized in that, The distance and the target state are input into a Kalman filter, and the convergence parameters of the Kalman filter are adjusted according to the target state to output the target velocity, including: When the target state is any state other than the cut-in state and the cut-out state, select the Kalman filter noise corresponding to the target state; The target velocity is output based on the distance, the target state, and the Kalman filter noise corresponding to the target state.

8. A target velocity determination device, characterized in that, The device includes: The acquisition module is used to acquire image information, which includes the identified target, the distance between the target and the vehicle, and the target state. The target state includes at least one of the following: occlusion state, cutting in state, cutting out state, road center state, image edge state, long distance state, crossing state, and approaching the front of the vehicle state. The output module is used to input the distance and the target state into the Kalman filter, adjust the convergence parameters of the Kalman filter according to the target state, and output the target speed. The convergence parameters of the Kalman filter include the speed difference between the target speed and the vehicle speed, and the Kalman filter noise. The target speed includes the absolute speed of the target relative to the ground and the relative speed of the target relative to the vehicle. The output module is used for: When the target state is in the cutting-in state or the cutting-out state, the vehicle speed is obtained, and the target initial speed is obtained based on the distance and the vehicle speed. Adjust the velocity difference value of the Kalman filter, and determine the target velocity based on the initial target velocity and the velocity difference value.

9. An electronic device, characterized in that, include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and configured to be executed by the processor to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.

11. A vehicle, characterized in that, include: The target velocity determination device as described in claim 8; Or, the electronic device as described in claim 9; Alternatively, the computer-readable storage medium as described in claim 10.

Images