Vehicle control method and device, electronic equipment and vehicle

Abstract

The invention discloses a vehicle control method and device, electronic equipment and a vehicle, and the method comprises the steps: collecting multiple frames of images in the driving direction of the vehicle in the driving process of the vehicle; if it is determined that a target sensing object exists according to the multiple frames of images, the braking decision result of the vehicle is determined according to sensing information of the target sensing object, and the sensing information comprises the movement speed of the target sensing object and the movement direction of the target sensing object; according to the braking decision result of the vehicle, the vehicle is controlled to execute preset operation, the preset operation comprises one of parking, deceleration driving and continuous driving, in the process, the vehicle can make a decision according to the visual information, and the vehicle is controlled based on the decision result, including parking, deceleration driving or continuous driving. And the condition of emergency braking caused by false triggering is reduced.

Description

Technical Field

[0001] The present application belongs to the field of vehicle control technology, and in particular relates to a vehicle control method, device, electronic equipment and vehicle. Background Art

[0002] The car's Automatic Emergency Braking (AEB) system is an advanced vehicle safety technology. The AEB system uses vehicle sensors such as radar to monitor the driving environment in front of the vehicle in real time, automatically detects the distance to the vehicle or obstacle in front, and when the system determines that there is a risk of collision, it will automatically apply brakes to reduce or avoid traffic accidents.

[0003] At present, the above-mentioned emergency braking is based on radar perception and is prone to false triggering of the AEB function, causing the vehicle to brake urgently due to false triggering. Summary of the Invention

[0004] The embodiments of the present application provide a vehicle control method, device, electronic device, and vehicle, which can make decisions based on visual information and control the vehicle based on the decision results, including stopping, slowing down, or continuing to drive, thereby reducing emergency braking caused by false triggering.

[0005] In a first aspect, an embodiment of the present application provides a vehicle control method, the method comprising:

[0006] During the driving of the vehicle, collecting multiple frames of images in the driving direction of the vehicle;

[0007] If it is determined according to the multiple frames of images that a target perception object exists, determining a braking decision result of the vehicle based on perception information of the target perception object, the perception information including a moving speed and a moving direction of the target perception object;

[0008] According to the braking decision result of the vehicle, the vehicle is controlled to perform a preset operation, where the preset operation includes one of the following: parking, decelerating and continuing to drive.

[0009] In one embodiment of the present application, after acquiring the plurality of frames of images in the direction of travel of the vehicle, if it is determined based on the plurality of frames of images that a target perception object exists, and before determining a braking decision result of the vehicle based on perception information of the target perception object, the method further includes:

[0010] Performing target detection on each of the multiple frames of images to obtain a first detection result for each of the frames of images, where the first detection result includes at least one detected perceived object and a confidence level of the perceived object, or the first detection result does not include the perceived object;

[0011] Deleting the first detection results that do not include the perceived object from the first detection results of the multiple frames of images to obtain J second detection results, where J is a positive integer;

[0012] Filter the J second detection results according to the confidence level of the perceived object included in each of the J second detection results to obtain I second detection results, where I is a positive integer and is less than or equal to J;

[0013] Determine the target perception object based on I of the second detection results.

[0014] In one embodiment of the present application, determining the target perception object according to one second detection result includes:

[0015] For an image corresponding to each of the one second detection results, if a center point of a perceived object in the image is identified to be located in a preset area of ​​the image, selecting the perceived object as a candidate perceived object;

[0016] If the same candidate perception object appears in the images corresponding to I of the second detection results more than a preset number of times, the candidate perception object is determined as the target perception object.

[0017] In one embodiment of the present application, determining the braking decision result of the vehicle based on the perception information of the target perception object includes:

[0018] Determining a motion trajectory of the target perception object according to first coordinate information of the target perception object in each frame of image;

[0019] Obtaining a motion speed of the target perception object according to a motion trajectory of the target perception object and a duration corresponding to a plurality of frames of images where the target perception object is located;

[0020] Determining a moving direction of the target perception object according to the moving trajectory of the target perception object;

[0021] A braking decision result of the vehicle is determined according to the moving speed of the target perception object and the moving direction of the target perception object.

[0022] In one embodiment of the present application, determining the movement direction of the target perception object according to the movement trajectory of the target perception object includes:

[0023] generating a first vector of the target perception object according to the motion trajectory of the target perception object, wherein the first vector is used to represent the running direction of the target perception object;

[0024] The moving direction of the target perception object is determined according to the angle between the first vector and the second vector, and the second vector is determined according to the moving trajectory of the vehicle.

[0025] In one embodiment of the present application, determining the braking decision result of the vehicle according to the movement speed and the movement direction of the target perception object includes:

[0026] Multiplying the movement speed of the target perception object by a preset time length to obtain a first predicted position point of the target perception object;

[0027] If the angle between the first vector and the second vector is less than a preset angle, and the distance between the first predicted position point and the second predicted position point is less than a preset distance, it is determined that the braking decision result includes a second indication, the second predicted position point is determined based on the movement speed of the vehicle and a preset duration, and the second indication is used to instruct the vehicle to stop or decelerate.

[0028] In one embodiment of the present application, determining the braking decision result of the vehicle according to the movement speed and the movement direction of the target perception object includes:

[0029] Multiplying the movement speed of the target perception object by a preset time length to obtain a first predicted position point of the target perception object;

[0030] If the angle between the first vector and the second vector is greater than or equal to a preset angle, and the first predicted position point is within a first area, it is determined that the braking decision result includes a second indication, and the second indication is used to instruct the vehicle to stop or decelerate, wherein the first area is determined based on the vehicle's body width, body length, movement speed and the preset duration.

[0031] In one embodiment of the present application, the number of the target perception objects is M, where M is a positive integer; and determining the braking decision result of the vehicle according to the movement speed and the movement direction of the target perception objects includes:

[0032] Calculating the distances between the M target perception objects and the vehicle respectively, and screening the M target perception objects according to the M distances to obtain N target perception objects, where N is a positive integer less than or equal to M;

[0033] The motion speeds, motion directions, categories of the N target perception objects, and the distances between the target perception objects and the vehicle are input into a decision model to obtain a braking decision result of the vehicle, wherein the decision model is trained using multiple training samples extracted from sample videos, and the decision model outputs a probability corresponding to each of the N target perception objects based on the motion speeds, motion directions, categories, and the distances between the perception objects and the vehicle, and:

[0034] When the probability corresponding to the existence of at least one target perception object among the N target perception objects is greater than a preset probability, determining that the braking decision result of the vehicle includes a second instruction, where the second instruction is used to instruct the vehicle to stop or decelerate;

[0035] When the probabilities corresponding to the N target perception objects are all less than the preset probability, it is determined that the braking decision result of the vehicle includes a third indication, and the third indication is used to instruct the vehicle to continue driving.

[0036] In one embodiment of the present application, the decision model includes multiple base models, each base model includes multiple decision units, the sum of the weights of the decision units in the base model is 1, and the sum of the weights of each base model is 1;

[0037] The decision model is trained according to the following process:

[0038] Acquire multiple training samples, wherein the training samples include the category of the perceived object, the movement speed of the perceived object, the movement direction of the perceived object, the distance between the perceived object and the sample vehicle, and labeling results of the training samples; the labeling results are used to indicate the braking decision results of the sample video;

[0039] In each round of training, a training sample is input to each decision unit, and the probability of the perception object included in the training sample output by each decision unit is obtained;

[0040] For each of the base models, performing weighted calculation according to the weight of each of the decision units in the base model and the probability of the output of each of the decision units in the base model to obtain the probability of the output of the base model;

[0041] Performing weighted calculation according to the weight of each base model and the probability output by the base model to obtain a total probability;

[0042] Determine a sample decision result based on the total probability;

[0043] According to the difference between the sample decision result and the labeling result corresponding to the training sample, the weight of each base model and the weight of each decision unit are adjusted.

[0044] In a second aspect, an embodiment of the present application provides a vehicle control device, which is applied to a vehicle and includes:

[0045] An acquisition module, configured to acquire multiple frames of images in the direction of travel of the vehicle during the vehicle's travel;

[0046] a decision module, configured to, if it is determined based on the multiple frames of image that a target perception object exists, determine a braking decision result for the vehicle based on perception information of the target perception object, the perception information including a moving speed and a moving direction of the target perception object;

[0047] A control module is used to control the vehicle to perform a preset operation according to the braking decision result of the vehicle, and the preset operation includes one of the following: stopping, decelerating and continuing to drive.

[0048] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory storing computer program instructions;

[0049] When the processor executes the computer program instructions, the vehicle control method as described in the first aspect is implemented.

[0050] In a fourth aspect, an embodiment of the present application provides a vehicle comprising the electronic device as described in the third aspect.

[0051] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the vehicle control method as described in the first aspect is implemented.

[0052] In a sixth aspect, an embodiment of the present application provides a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the vehicle control method as described in the first aspect.

[0053] The vehicle control method, device, electronic device and vehicle of the embodiments of the present application collect multiple frames of images in the direction of vehicle travel during vehicle driving; if it is determined based on the multiple frames of images that a target perception object exists, the braking decision result of the vehicle is determined based on the perception information of the target perception object, and the perception information includes the movement speed and movement direction of the target perception object. Further, based on the braking decision result of the vehicle, the vehicle is controlled to perform a preset operation, and the preset operation includes one of the following: stopping, decelerating and continuing to drive. During the above process, the vehicle can make decisions based on visual information and control the vehicle based on the decision results, including stopping, decelerating or continuing to drive, thereby reducing emergency braking caused by false triggering. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0055] Figure 1 This is a flow chart of a vehicle control method provided by an embodiment of the present application;

[0056] Figure 2 This is a schematic diagram of target detection provided by an embodiment of the present application;

[0057] Figure 3 This is a schematic diagram of the motion trajectory provided in the embodiment of the present application. Figure 1 ;

[0058] Figure 4 This is a schematic diagram of the motion trajectory provided in the embodiment of the present application. Figure 2 ;

[0059] Figure 5 It is a motion trajectory and vector diagram provided by the embodiment of the present application;

[0060] Figure 6 This is another flow chart of the vehicle control method provided in the embodiment of the present application;

[0061] Figure 7 is a schematic structural diagram of a vehicle control device provided in an embodiment of the present application;

[0062] Figure 8 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present application by illustrating the examples of the present application.

[0064] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, the elements defined by the phrase "comprising..." do not exclude the presence of other identical elements in the process, method, article, or device comprising the elements.

[0065] In each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to the user's identity or characteristics, such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of such data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, such as historical driving video data, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the disclosed embodiment will be obtained.

[0066] The vehicle can be a private car, such as a sedan, SUV, MPV, or pickup truck. It can also be a commercial vehicle, such as a van, bus, small truck, or large trailer. It can be a gasoline vehicle or a new energy vehicle. When the vehicle is a new energy vehicle, it can be a hybrid vehicle or a pure electric vehicle.

[0067] In order to solve the problems of the prior art, the embodiments of the present application provide a vehicle control method, device, electronic device and vehicle. The vehicle control method provided by the embodiments of the present application is first introduced below.

[0068] Figure 1FIG. 1 shows a flow chart of a vehicle control method provided by an embodiment of the present application. Figure 1 As shown, the vehicle control method provided in the embodiment of the present application is applied to an electronic device, such as a server, and includes the following steps 101 to 103, wherein:

[0069] Step 101 : During the driving process of the vehicle, a plurality of frames of images are collected in the driving direction of the vehicle.

[0070] In this embodiment, while the vehicle is traveling, a camera or imaging device disposed at the front end of the vehicle collects multiple frames of images in the direction of the vehicle's travel, and target detection is performed on each frame of the image to determine the target perception object present in the multiple frames of the image.

[0071] Step 102: If it is determined based on the multiple frames of images that a target perception object exists, a braking decision result of the vehicle is determined based on perception information of the target perception object, where the perception information includes a movement speed and a movement direction of the target perception object.

[0072] In this embodiment, the target perception object can be a vehicle, a person, an obstacle, etc. If the existence of the target perception object is determined based on multiple frames of images, the perception information is obtained from the multiple frames of images, including the movement speed of the target perception object, the movement direction of the target perception object and the category of the target perception object. The categories include vehicles, people, obstacles, etc. The braking decision result of the vehicle is determined based on the perception information of the target perception object. The braking decision result includes corresponding instructions, and the corresponding instructions are used to instruct the vehicle to perform preset operations.

[0073] Step 103: Control the vehicle to perform a preset operation according to the braking decision result of the vehicle, where the preset operation includes one of the following: stopping, decelerating and continuing to drive.

[0074] In this embodiment, the vehicle is controlled to stop, reduce travel, or continue travel based on the braking decision result of the vehicle. For example, the braking decision result includes indication a, which is used to instruct the vehicle to reduce travel or stop, and the braking result includes indication b, which is used to instruct the vehicle to continue travel.

[0075] In this embodiment, during the driving of the vehicle, multiple frames of images are collected in the driving direction of the vehicle; if it is determined based on the multiple frames of images that there is a target perception object, the braking decision result of the vehicle is determined based on the perception information of the target perception object, and the perception information includes the movement speed of the target perception object and the movement direction of the target perception object. Further, based on the braking decision result of the vehicle, the vehicle is controlled to perform a preset operation, and the preset operation includes one of the following: stopping, decelerating and continuing to drive. During the above process, the vehicle can make decisions based on visual information and control the vehicle based on the decision results, including stopping, decelerating or continuing to drive, thereby reducing emergency braking caused by false triggering, improving driving safety and response speed, and enhancing user experience.

[0076] In one embodiment of the present application, after acquiring multiple frames of images in the direction of travel of the vehicle, if it is determined based on the multiple frames of images that a target perception object exists, and before determining a braking decision result of the vehicle based on perception information of the target perception object, the method further includes:

[0077] Performing target detection on each of the multiple frames of images to obtain a first detection result for each of the frames of images, where the first detection result includes at least one detected perceived object and a confidence level of the perceived object, or the first detection result does not include the perceived object;

[0078] Deleting the first detection results that do not include the perceived object from the first detection results of the multiple frames of images to obtain J second detection results, where J is a positive integer;

[0079] Filter the J second detection results according to the confidence level of the perceived object included in each of the J second detection results to obtain I second detection results, where I is a positive integer and is less than or equal to J;

[0080] Determine the target perception object based on I of the second detection results.

[0081] In this embodiment, target detection is performed on each of the multiple image frames using an object detection algorithm, such as the YOLO detection algorithm, to obtain a first detection result for each frame. The first detection result includes at least one detected perceived object and a confidence score for the perceived object. It may also include a detection frame for the perceived object, the perceived object's category, and coordinate information for the perceived object. The coordinate information may be pixel coordinate information or coordinate information in the vehicle's coordinate system. Alternatively, if there are no people, vehicles, or obstacles in the vehicle's travel direction, the first detection result may not include the perceived object.

[0082] The first detection results that do not include the perceived object in the first detection results of the multiple-frame images are deleted to obtain J second detection results. Specifically, if the first detection result does not include the perceived object, the first detection result is not retained, the first detection result that does not include the perceived object is deleted, and the first detection result that includes at least one detected perceived object in the first detection results of the multiple-frame images is retained, that is, J second detection results are obtained, where J is a positive integer, and each of the J second detection results includes at least one detected perceived object and the confidence of the perceived object.

[0083] See also Figure 2 The number outside the detection box of the perceived object is the confidence level. Usually, the confidence level ranges from 0 to 1. The higher the value, the more reliable the detection result. Usually, a confidence threshold is pre-set, such as 0.5, to filter out detection boxes below the threshold, which can reduce false detections. Specifically, according to the confidence level of the perceived object, for each of the J second detection results, the confidence level included in the second detection result is compared with the confidence threshold. If the confidence level included in the second detection result is greater than or equal to the confidence threshold, the second detection result is retained; if the confidence level included in the second detection result is less than the confidence threshold, the second detection result is not retained. The J second detection results are filtered in the above manner to obtain I second detection results, where I is a positive integer and I is less than or equal to J.

[0084] The detection result obtained by the detection algorithm may also include coordinate information of the detected perception object in the image, and the coordinate information may be the coordinate information of the center point of the perception object. Further, the target perception object is determined based on the I second detection results.

[0085] Optionally, a depth estimation model is used to estimate the depth of the target perception object. The depth estimation model can use DepthAnything v2, which is a transformer-based deep learning network model that can estimate the actual distance of each pixel of the input image from the camera. According to the average distance of the pixels in the perception target frame, the actual distance of the target perception target from the camera, that is, the longitudinal distance, is obtained. The 3D coordinates of the pixel in the camera coordinate system, that is, the lateral distance, are calculated based on the camera intrinsic parameters and the depth value of the pixel at the center of the perception target frame. Assume that the pixel at the center of the perception target is (u, v), the depth is z, and the camera intrinsic parameter K is expressed as:

[0086]

[0087] The specific calculation method for the lateral distance is Xc = (u-cx) * z / fx; the depth estimation model DepthAnythingv2 is used to estimate the pixel depth, that is, the longitudinal distance between the target pixel and the vehicle. The average pixel depth in the target frame is taken as the longitudinal distance between the target and the vehicle, that is, the true depth of the target. The lateral distance and longitudinal distance constitute the coordinate information of the target perception object.

[0088] By performing target detection on images in driving scenarios, various targets in the images can be accurately identified.

[0089] In one embodiment of the present application, determining the target perception object according to one second detection result includes:

[0090] For an image corresponding to each of the one second detection results, if a center point of a perceived object in the image is identified to be located in a preset area of ​​the image, selecting the perceived object as a candidate perceived object;

[0091] If the same candidate perception object appears in the images corresponding to I of the second detection results more than a preset number of times, the candidate perception object is determined as the target perception object.

[0092] See also Figure 2 , taking one frame of multiple frames as an example, the detected perception object will be surrounded by a detection frame and marked with its type or category, including bus, car, etc. The image also includes a preset area, i.e. Figure 2 The trapezoidal frame area in front of the vehicle is obtained through the vehicle's camera calibration. For example, the horizontal distance between the left and right sides of the preset area and the center point of the vehicle is 1.5 meters, and the longitudinal distance between the top edge of the preset area and the vehicle is 50 meters. The perception object whose center point is within the preset area is regarded as the subsequent perception object.

[0093] For an image corresponding to each of the I second detection results, if the center point of the identified perceived object is located in a preset area of ​​the image, the identified perceived object is used as a candidate perceived object, such as Figure 2 As shown, the center points of all perception objects are not located in the preset area of ​​the image, and there are no candidate perception objects in this frame image, which means that the identified perception objects are far away from the vehicle and are not candidate perception objects.

[0094] For example, the multiple frames of images are ten frames of images. If the number of times the same candidate perception object appears in the images corresponding to I second detection results is greater than the preset number of times, such as the preset number of times is eight times, the same candidate perception object may appear in nine of the ten frames of images, or appear in all ten frames of images, and the candidate perception object is determined as the target perception object.

[0095] Combining the occurrence position and the number of occurrences to screen the perception objects and obtain the target perception objects can improve the accuracy of target screening.

[0096] In one embodiment of the present application, determining the braking decision result of the vehicle based on the perception information of the target perception object includes:

[0097] Determining a motion trajectory of the target perception object according to first coordinate information of the target perception object in each frame of image;

[0098] Obtaining a motion speed of the target perception object according to a motion trajectory of the target perception object and a duration corresponding to a plurality of frames of images where the target perception object is located;

[0099] Determining a moving direction of the target perception object according to the moving trajectory of the target perception object;

[0100] A braking decision result of the vehicle is determined according to the moving speed of the target perception object and the moving direction of the target perception object.

[0101] In this embodiment, the first coordinate information of the target perception object in each frame of the image may be the coordinate information in the pixel coordinate system. The first coordinate information of the target perception object in each frame of the image is subjected to coordinate transformation to obtain the second coordinate information corresponding to the target perception object. The second coordinate information may be the coordinate information in the vehicle coordinate system with the vehicle as the origin. The second coordinate information corresponding to the target perception object is transformed to obtain the third coordinate information corresponding to the target perception object. The third coordinate information may be the coordinate information in the global coordinate system. The third coordinate information corresponding to the target perception object in each frame of the image is connected to obtain the motion trajectory of the target perception object.

[0102] like Figure 3 As shown, Figure 3The motion trajectory of the ego vehicle, i.e., the execution subject of this embodiment, hereinafter referred to as the ego vehicle, is obtained. The motion trajectory of the ego vehicle (also referred to as the driving trajectory) is the motion trajectory of the time period corresponding to the collected multiple frames of images. The driving trajectory of the ego vehicle is determined based on the motion speed and motion direction (also referred to as the driving direction) of the ego vehicle. For example, the second coordinate information (0, 0) of the vehicle position corresponding to the first frame of image is the origin, and the initial driving direction of the vehicle is the y-axis direction. According to the vehicle speed v1 corresponding to the first frame of image and the time t between the two frames of image, the second coordinate information (0, 0) of the vehicle position corresponding to the second frame of image is calculated. The coordinate information (x1, y1) = (0, v1 × t), the vehicle speed corresponding to the second frame image is v2, and the angle between the driving direction and the y-axis is a. Then, the second coordinate information of the vehicle corresponding to the third frame image is (x2, y2) = (x1 + v2 × t × sin (a), y1 + v2 × t × cos (a)). The above method can be used to obtain the second coordinate information of the vehicle corresponding to each frame image. The third coordinate information is converted to obtain the third coordinate information of the vehicle. The third coordinate points are connected in sequence to obtain the motion trajectory of the vehicle, that is, the coordinate information in the global coordinate system.

[0103] It should be noted that if the target perception object does not appear in a certain frame image, the first, second, and third coordinate information of the target perception object can be marked as empty, which does not affect the generation of the motion trajectory.

[0104] For example, in a frame of image, the horizontal distance and vertical distance of a target perception object are m and n respectively, that is, the second coordinate information of the target perception object in the ego-vehicle coordinate system corresponding to the frame of image is (m, n), the angle between the y-axis of the ego-vehicle coordinate system and the y-axis of the global coordinate system is a, and the coordinates of the origin of the ego-vehicle coordinate system in the global coordinate system are (x1, y1), then the coordinates of the target perception object in the global coordinate system can be calculated as (x, y) = (x1 + m*cos(a) + n*sin(a), y1 - m*sin(a) + n*cos(a)). According to this method, the third coordinate information of each target perception object in the global coordinate system is calculated in turn, and the third coordinate points are connected in turn to obtain the motion trajectory of the target perception object, such as Figure 4 As shown in FIG, when the target perception object is a vehicle, the motion trajectory of the vehicle and the motion trajectory of the ego vehicle are obtained by the above method.

[0105] Among them, the multi-frame images in the time length corresponding to the multi-frame images where the target perception object is located and the aforementioned multi-frame images may be the same or different. If the target perception object appears in every frame of the multi-frame images, the multi-frame images where it is located are the aforementioned multi-frame images. If the target perception object does not appear in every frame of the multi-frame images, the multi-frame images where it is located are the images where the target perception object appears.

[0106] Furthermore, the motion speed of the target perception object is calculated based on the motion trajectory of the target perception object and the duration corresponding to the multi-frame images where the target perception object is located. Specifically, the motion distance of the target perception object is obtained based on the motion trajectory of the target perception object, and the motion distance of the target perception object is divided by the duration corresponding to the multi-frame images where the target perception object is located to obtain the motion speed of the target perception object, that is, the average motion speed of the target perception object in the multi-frame images where the target perception object is located.

[0107] Furthermore, the motion direction of the target perception object is determined based on the motion trajectory of the target perception object and the motion trajectory of the vehicle. The motion direction is relative to the vehicle and is divided into lateral motion and longitudinal motion.

[0108] In the above steps, a decision is made based on the movement speed and movement direction of the target perception object to obtain a braking decision result of the vehicle. The braking decision result includes corresponding instructions, and the vehicle is controlled to perform preset operations according to the corresponding instructions.

[0109] Visual information includes a large amount of detailed environmental information. The movement of the target perceived object is obtained through visual information. This rich information provides a comprehensive basis for the vehicle's decision-making, enabling the vehicle to better understand the surrounding environment and make appropriate responses.

[0110] In one embodiment of the present application, determining the movement direction of the target perception object according to the movement trajectory of the target perception object includes:

[0111] generating a first vector of the target perception object according to the motion trajectory of the target perception object, wherein the first vector is used to represent the running direction of the target perception object;

[0112] The moving direction of the target perception object is determined according to the angle between the first vector and the second vector, and the second vector is determined according to the moving trajectory of the vehicle.

[0113] In this embodiment, a first vector of the target perception object is generated according to the motion trajectory of the target perception object, and the first vector is used to characterize the running direction of the target perception object. A second vector of the vehicle is generated according to the motion trajectory of the vehicle. The motion direction of the target perception object is determined according to the angle between the first vector and the second vector. For example, if the angle between the first vector and the second vector is greater than or equal to a preset angle, the preset angle can be set to 60°, and the motion direction of the target perception object is determined to be lateral motion. The motion direction here is relative to the vehicle, that is, the motion direction of the target perception object is lateral motion relative to the motion direction of the vehicle; if the angle between the first vector and the second vector is less than the preset angle, the motion direction of the target perception object is determined to be longitudinal motion, that is, the motion direction of the target perception object is longitudinal motion relative to the motion direction of the vehicle.

[0114] See also Figure 5 As shown, the target perceives the motion trajectory of the object and the vehicle, generates their respective vectors according to the motion trajectory, and determines the motion direction according to the angle between the first vector and the second vector. Figure 5 It can be seen that the angle between the first vector and the second vector is greater than the preset angle, and the movement direction of the target perception object is determined to be lateral movement.

[0115] The motion trajectory generated by visual information can accurately determine the motion direction of the target perception object.

[0116] In one embodiment of the present application, determining the braking decision result of the vehicle according to the movement speed and the movement direction of the target perception object includes:

[0117] Multiplying the movement speed of the target perception object by a preset time length to obtain a first predicted position point of the target perception object;

[0118] If the angle between the first vector and the second vector is less than a preset angle, and the distance between the first predicted position point and the second predicted position point is less than a preset distance, it is determined that the braking decision result includes a second indication, the second predicted position point is determined based on the movement speed of the vehicle and a preset duration, and the second indication is used to instruct the vehicle to stop or decelerate.

[0119] In this embodiment, the preset duration can be set to 2 seconds. The target perception object's reachable position within the preset duration, such as 2 seconds, is estimated, and the target perception object's velocity is multiplied by the preset duration to obtain a first predicted target perception object position. Furthermore, the vehicle's reachable position within the preset duration, such as 2 seconds, is estimated, and the vehicle's velocity is multiplied by the preset duration to obtain a second predicted vehicle position.

[0120] If the angle between the first vector and the second vector is less than the preset angle, it means that the target perception object is a longitudinally moving object, and the target perception object and the vehicle are moving in the same direction. It is necessary to determine whether there is a possibility of collision between the target perception object and the vehicle in front. The distance between the first predicted position point and the second predicted position point is less than the preset distance, which means that the distance between the target perception object and the vehicle is relatively close, and there is a possibility that the first predicted position point and the second predicted position point coincide with each other, that is, if the vehicle continues to move forward, it may collide with the target perception object and needs to stop or slow down. It is determined that the braking decision result includes a second indication, and the second indication is used to instruct the vehicle to stop or slow down.

[0121] Among them, if the angle between the first vector and the second vector is less than a preset angle, and the distance between the first predicted position point and the second predicted position point is greater than or equal to a preset distance, it is determined that the braking decision result includes a third indication, and the third indication is used to instruct the vehicle to continue driving.

[0122] If the angle between the first vector and the second vector is less than the preset angle, it means that the target perception object is a longitudinally moving object, and the target perception object and the vehicle are moving in the same direction. It is necessary to determine whether there is a possibility of collision between the target perception object and the vehicle in front. The distance between the first predicted position point and the second predicted position point is greater than or equal to the preset distance, which means that the distance between the target perception object and the vehicle is relatively far, that is, the vehicle will not collide with the target perception object if it continues to move forward, and the vehicle can continue to drive. It is determined that the braking decision result includes a third indication, and the third indication is used to instruct the vehicle to continue driving.

[0123] By estimating the position points of the longitudinally moving target perception objects, it is possible to accurately determine whether a collision occurs so that the vehicle can make decisions.

[0124] In one embodiment of the present application, determining the braking decision result of the vehicle according to the movement speed and the movement direction of the target perception object includes:

[0125] Multiplying the movement speed of the target perception object by a preset time length to obtain a first predicted position point of the target perception object;

[0126] If the angle between the first vector and the second vector is greater than or equal to a preset angle, and the first predicted position point is within a first area, it is determined that the braking decision result includes a second indication, and the second indication is used to instruct the vehicle to stop or decelerate, wherein the first area is determined based on the vehicle's body width, body length, movement speed and the preset duration.

[0127] In this embodiment, if the angle between the first vector and the second vector is greater than or equal to the preset angle, it means that the target perception object is a laterally moving object, and it is necessary to determine whether there is a possibility of collision between the target perception object and the vehicle. Optionally, the lateral component velocity can be calculated for the laterally moving target perception object. The lateral component velocity is calculated based on the movement speed and the angle of the target perception object. The angle is the angle between the movement direction of the target perception object in the last frame of the multi-frame image and the movement direction of the vehicle. The lateral component velocity of the target perception object is multiplied by the preset time length to obtain the first predicted position point of the target perception object.

[0128] For a laterally moving target perception object, the target perception object and the vehicle are moving in different directions. To estimate the position point that the target perception object can reach within a preset time, such as 2 seconds, the entire vehicle body needs to be taken into consideration. Specifically, the first area range is determined based on the vehicle body width, vehicle body length, movement speed, and the preset time length. The vehicle body width and vehicle body length are multiplied to obtain the vehicle body area. The movement speed is multiplied by the preset time length to obtain the second predicted position point of the vehicle. A first area is generated with the second predicted position point as the center, wherein the first area is the same size as the vehicle body area. If the first predicted position point is within the first area range, there is a possibility that the first predicted position point and the first area range overlap. If the vehicle continues to move forward, it may collide with the target perception object and need to stop or slow down. That is, it is determined that the braking decision result includes a second instruction, and the second instruction is used to instruct the vehicle to stop or slow down.

[0129] Among them, if the angle between the first vector and the second vector is greater than or equal to a preset angle, and the first predicted position point is not within the first area, it is determined that the braking decision result includes a third indication, and the third indication is used to instruct the vehicle to continue driving.

[0130] If the angle between the first vector and the second vector is greater than or equal to the preset angle, it means that the target perception object is a laterally moving object, the first predicted position point is not within the first area, the first predicted position point does not overlap with the first area, the vehicle continues to move forward without colliding with the target perception object, the vehicle can continue to drive, and it is determined that the braking decision result includes a third indication, which is used to instruct the vehicle to continue driving.

[0131] By estimating the position of the laterally moving target perception object, it is possible to accurately determine whether a collision has occurred, allowing the vehicle to make decisions.

[0132] In a scenario, when there are multiple target perception objects, take the target perception object as an example. The multiple vehicles include vehicles moving laterally and vehicles moving longitudinally. In order to reduce calculation, the vehicles that are farther away can be filtered first, such as Figure 6 As shown, the vehicles are screened according to the first threshold and the second threshold, and the longitudinal targets with a lateral distance greater than the first threshold, i.e., the longitudinally moving vehicles, are discarded, and the lateral targets with a longitudinal distance greater than the second threshold, i.e., the laterally moving vehicles, are discarded. The first threshold and the second threshold are set according to actual needs; finally, the vehicles retained Figure 6The target perception objects A, B, and C are located in the middle. For lateral targets, the current speed is used to simultaneously calculate the positions of the ego vehicle and the target vehicle within 2 seconds (i.e., the first predicted position point mentioned above), and a horizontal and vertical range (i.e., the first area range mentioned above) is drawn with the ego vehicle as the center. If the target point is within this range (i.e., the first predicted position point mentioned above is within the first area range), AEB is triggered, i.e., the vehicle is controlled to stop or reduce travel. If it is not within this range, AEB is not triggered, and the vehicle is controlled to continue traveling. For longitudinal targets, the current speed is used to simultaneously calculate the positions of the ego vehicle and the target vehicle within 2 seconds (i.e., the first predicted position point mentioned above). If the target vehicle point coincides with the ego vehicle point (i.e., the distance between the first predicted position point and the second predicted position point mentioned above is less than the preset distance), AEB is triggered, i.e., the vehicle is controlled to stop or reduce travel. If they do not coincide, AEB is not triggered, and the vehicle is controlled to continue traveling.

[0133] In one embodiment of the present application, the number of target perception objects is M, where M is a positive integer;

[0134] The determining of the braking decision result of the vehicle according to the movement speed and the movement direction of the target perception object includes:

[0135] Calculating the distances between the M target perception objects and the vehicle respectively, and screening the M target perception objects according to the M distances to obtain N target perception objects, where N is a positive integer less than or equal to M;

[0136] The motion speeds, motion directions, categories of the N target perception objects, and the distances between the target perception objects and the vehicle are input into a decision model to obtain a braking decision result of the vehicle, wherein the decision model is trained using multiple training samples extracted from sample videos, and the decision model outputs a probability corresponding to each of the N target perception objects based on the motion speeds, motion directions, categories, and the distances between the perception objects and the vehicle, and:

[0137] When the probability corresponding to the existence of at least one target perception object among the N target perception objects is greater than a preset probability, determining that the braking decision result of the vehicle includes a second instruction, where the second instruction is used to instruct the vehicle to stop or decelerate;

[0138] When the probabilities corresponding to the N target perception objects are all less than the preset probability, it is determined that the braking decision result of the vehicle includes a third indication, and the third indication is used to instruct the vehicle to continue driving.

[0139] In this embodiment, the number of target perception objects is M, where M is a positive integer. The distances between the M target perception objects and the vehicle are calculated respectively, and the M target perception objects are screened according to the M distances. For example, for each target perception object, the distance between the target perception object and the vehicle in the last frame of the multi-frame image where the target perception object is located is calculated, and the M target perception objects are screened according to the M distances to obtain N target perception objects, where N is less than or equal to M.

[0140] Specifically, if the movement direction of the target perception object is longitudinal movement, and the lateral distance between the target perception object and the vehicle is greater than a first threshold, the first threshold is set according to actual needs, the target perception object and the vehicle are moving in the same direction, but the lateral distance is relatively far, and the target perception object can be discarded; if the movement direction of the target perception object is longitudinal movement, and the lateral distance between the target perception object and the vehicle is less than or equal to the first threshold, the target perception object and the vehicle are moving in the same direction, but the lateral distance is relatively close, there is a possibility of a collision, and the target perception object is retained.

[0141] Specifically, if the movement direction of the target perception object is lateral movement, and the longitudinal distance between the target perception object and the vehicle is greater than the second threshold, the second threshold is set according to actual needs, the target perception object and the vehicle are moving in different directions, but the longitudinal distance is relatively far, and the target perception object can be discarded; if the movement direction of the target perception object is lateral movement, and the longitudinal distance between the target perception object and the vehicle is less than or equal to the second threshold, the target perception object and the vehicle are not moving in the same direction, but the longitudinal distance is relatively close, and there is a possibility of collision, and the target perception object is retained.

[0142] By screening multiple target perception objects in the above manner, N target perception objects are obtained.

[0143] A decision model is used to make a decision, and a plurality of training samples are used in advance to train the decision model to obtain a trained decision model, wherein the training samples are extracted from sample videos.

[0144] The speed, direction, category, and distance between each of the N target perception objects and the vehicle are input into the decision model to obtain the vehicle's braking decision result. The decision model outputs the probability corresponding to each of the N target perception objects based on their speed, direction, category, and distance between the perception objects and the vehicle, and:

[0145] When the probability corresponding to the existence of at least one target perception object among N target perception objects is greater than the preset probability, it means that the vehicle will collide with the target perception object if it continues to move forward, and it is necessary to stop or slow down. Then, it is determined that the braking decision result of the vehicle includes a second indication, and the second indication is used to instruct the vehicle to stop or slow down.

[0146] If the probabilities corresponding to each of the N target perception objects are all less than a preset probability, indicating that the vehicle will not collide with the target perception objects if it continues to move forward, and thus can continue driving, the vehicle's braking decision result is determined to include a third instruction, which is used to instruct the vehicle to continue driving. Alternatively, if the probabilities of some target perception objects are less than the preset probability and the probabilities of some target perception objects are equal to the preset probability, in which case the vehicle will not collide with the target perception objects if it continues to move forward, and thus can continue driving, the vehicle's braking decision result is determined to include a third instruction, which is used to instruct the vehicle to continue driving.

[0147] In one embodiment of the present application, the decision model includes multiple base models, each base model includes multiple decision units, the sum of the weights of the decision units in the base model is 1, and the sum of the weights of each base model is 1;

[0148] The decision model is trained according to the following process:

[0149] Acquire multiple training samples, wherein the training samples include the category of the perceived object, the movement speed of the perceived object, the movement direction of the perceived object, the distance between the perceived object and the sample vehicle, and labeling results of the training samples; the labeling results are used to indicate the braking decision results of the sample video;

[0150] In each round of training, a training sample is input to each decision unit, and the probability of the perception object included in the training sample output by each decision unit is obtained;

[0151] For each of the base models, performing weighted calculation according to the weight of each of the decision units in the base model and the probability of the output of each of the decision units in the base model to obtain the probability of the output of the base model;

[0152] Performing weighted calculation according to the weight of each base model and the probability output by the base model to obtain a total probability;

[0153] Determine a sample decision result based on the total probability;

[0154] According to the difference between the sample decision result and the labeling result corresponding to the training sample, the weight of each base model and the weight of each decision unit are adjusted.

[0155] The above decision model includes multiple base models, each base model includes multiple decision units, such as 3 decision units, the sum of the weights of each decision unit in the base model is 1, and the sum of the weights of multiple base models is 1.

[0156] In the above process, multiple training samples are obtained, and the training samples include the category of the perceived object, the movement speed of the perceived object, the movement direction of the perceived object, the distance between the perceived object and the vehicle, and the annotation results of the sample video. The annotation results are used to indicate the braking decision results of the sample video. The above-mentioned category of the perceived object, the movement speed of the perceived object, the movement direction of the perceived object, and the distance between the perceived object and the vehicle are extracted from multiple or one sample videos. The annotation results can be the annotation results obtained by manually annotating the sample video. The annotation results are used to indicate the braking decision results of the sample video. The annotation result is triggered braking, and triggered braking is used to indicate that the braking decision results of the sample video include a second indication, and the second indication is used to instruct the sample vehicle to stop or slow down; the annotation result is not triggered braking, and not triggered braking is used to indicate that the braking decision results of the sample video include a third indication, and the third indication is used to instruct the sample vehicle to continue driving.

[0157] During each round of training, a training sample is input to each decision unit to obtain the probability of the perceived object included in the training sample output by each decision unit. Optionally, the training sample input to each decision unit can be the same training sample.

[0158] For each base model, a weighted calculation is performed according to the weight of each decision unit in the base model and the probability of each decision unit output in the base model to obtain the probability of the base model output.

[0159] A weighted calculation is performed based on the weight of each base model and the probability of the base model output to obtain the total probability. Further, the sample decision result is determined based on the total probability. Specifically, if the total probability is greater than the preset probability, it is determined that the sample decision result includes the second indication; if the total probability is less than the preset probability, it is determined that the sample decision result includes the third indication.

[0160] According to the difference between the sample decision result and the annotation result corresponding to the training sample, the weight of each base model and the weight of each decision unit in each base model are adjusted according to the difference. For example, the category of the perceived object, the movement speed of the perceived object, the movement direction of the perceived object, and the distance between the perceived object and the vehicle are extracted from the sample video a and input into the decision model. The output result of the decision model includes a second indication, which is used to instruct the sample vehicle to stop or slow down. The annotation result of the sample video a is that braking is not triggered, that is, braking does not trigger the third indication. The decision model needs to be trained further, that is, the weight is adjusted so that the difference between the braking decision result output by the decision model and the annotation result corresponding to the training sample is relatively small.

[0161] The pre-trained decision model can output decision results so that the vehicle can make decisions quickly, reducing emergency braking caused by false triggering.

[0162] Figure 7 FIG1 shows a structural diagram of a vehicle control device provided in an embodiment of the present application. Figure 7 As shown, a vehicle control device 700 is applied to a vehicle and includes:

[0163] The acquisition module 701 is used to acquire multiple frames of images in the driving direction of the vehicle during the driving process of the vehicle;

[0164] A decision module 702 is configured to, if it is determined from the multiple frames of images that a target perception object exists, determine a braking decision result for the vehicle based on perception information of the target perception object, where the perception information includes a speed and a direction of movement of the target perception object;

[0165] The control module 703 is used to control the vehicle to perform a preset operation according to the braking decision result of the vehicle, and the preset operation includes one of the following: stopping, decelerating and continuing to drive.

[0166] In one embodiment of the present application, the vehicle control device further includes: a detection module, a first screening module, a second screening module, and a determination module;

[0167] a detection module, configured to perform target detection on each of the multiple image frames to obtain a first detection result for each of the image frames, wherein the first detection result includes at least one detected perceived object and a confidence level of the perceived object, or the first detection result does not include the perceived object;

[0168] A first screening module is configured to delete first detection results that do not include the perceived object from the first detection results of the multiple frames of images, to obtain J second detection results, where J is a positive integer;

[0169] a second screening module, configured to screen the J second detection results according to the confidence level of the perceived object included in each of the J second detection results, to obtain I second detection results, where I is a positive integer and is less than or equal to J;

[0170] A determination module is used to determine the target perception object based on I second detection results.

[0171] In one embodiment of the present application, the determination module includes a first determination submodule;

[0172] The first determination submodule is used to, for each image corresponding to the I second detection results, determine the perceived object as a candidate perceived object when it is identified that the center point of the perceived object in the image is located in a preset area of ​​the image; if the number of times the same candidate perceived object appears in the images corresponding to the I second detection results is greater than a preset number, determine the candidate perceived object as the target perceived object.

[0173] In one embodiment of the present application, the decision module includes a second determination sub-unit, a calculation module, a third determination sub-module and a decision sub-module;

[0174] a second determining subunit, configured to determine a motion trajectory of the target perception object based on first coordinate information of the target perception object in each frame of image;

[0175] a calculation module, configured to obtain a motion speed of the target perception object based on a motion trajectory of the target perception object and a duration corresponding to a plurality of frames of images where the target perception object is located;

[0176] A third determining submodule is configured to determine a moving direction of the target perception object according to a moving trajectory of the target perception object;

[0177] The decision submodule is used to determine a braking decision result of the vehicle according to the movement speed and the movement direction of the target perception object.

[0178] In one embodiment of the present application, the third determination submodule is specifically used to generate a first vector of the target perception object based on the motion trajectory of the target perception object, and the first vector is used to characterize the running direction of the target perception object; based on the angle between the first vector and the second vector, the motion direction of the target perception object is determined, and the second vector is determined according to the motion trajectory of the vehicle.

[0179] In one embodiment of the present application, the decision submodule includes a first calculation subunit and a first decision subunit;

[0180] a first calculating subunit, configured to multiply the movement speed of the target perception object by a preset duration to obtain a first predicted position point of the target perception object;

[0181] The first decision subunit is configured to determine that the braking decision result includes a second indication if the angle between the first vector and the second vector is less than a preset angle and the distance between the first predicted position point and the second predicted position point is less than a preset distance, wherein the second predicted position point is determined based on the movement speed of the vehicle and a preset duration, and the second indication is used to instruct the vehicle to stop or decelerate.

[0182] In one embodiment of the present application, the decision submodule includes a second calculation subunit and a second decision subunit;

[0183] a second calculation subunit, configured to multiply the movement speed of the target perception object by a preset duration to obtain a first predicted position point of the target perception object;

[0184] The second decision subunit is used to determine that the braking decision result includes a second indication if the angle between the first vector and the second vector is greater than or equal to a preset angle and the first predicted position point is within a first area. The second indication is used to instruct the vehicle to stop or decelerate, wherein the first area is determined based on the vehicle's body width, body length, movement speed and the preset duration.

[0185] In one embodiment of the present application, the number of target perception objects is M, where M is a positive integer; the decision submodule includes a third calculation subunit and a third decision submodule;

[0186] a third calculation subunit, configured to respectively calculate the distances between the M target perception objects and the vehicle, and screen the M target perception objects according to the M distances to obtain N target perception objects, where N is a positive integer less than or equal to M;

[0187] The third decision submodule is configured to input the motion speeds, motion directions, categories of the N target perception objects and the distances between the target perception objects and the vehicle into a decision model to obtain a braking decision result for the vehicle, wherein the decision model is trained using a plurality of training samples extracted from sample videos, and the decision model outputs a probability corresponding to each of the N target perception objects based on the motion speeds, motion directions, categories and the distances between the perception objects and the vehicle, and:

[0188] When the probability corresponding to the existence of at least one target perception object among the N target perception objects is greater than a preset probability, determining that the braking decision result of the vehicle includes a second instruction, where the second instruction is used to instruct the vehicle to stop or decelerate;

[0189] When the probabilities corresponding to the N target perception objects are all less than the preset probability, it is determined that the braking decision result of the vehicle includes a third indication, and the third indication is used to instruct the vehicle to continue driving.

[0190] In one embodiment of the present application, the decision model includes multiple base models, each base model includes multiple decision units, the sum of the weights of the decision units in the base model is 1, and the sum of the weights of each base model is 1; the device also includes a training module;

[0191] A training module is used to obtain multiple training samples, wherein the training samples include the category of the perceived object, the movement speed of the perceived object, the movement direction of the perceived object, the distance between the perceived object and the sample vehicle, and the annotation results of the training samples; the annotation results are used to indicate the braking decision results of the sample video; in each round of training, a training sample is input into each decision unit to obtain the probability of the perceived object included in the training sample output by each decision unit; for each base model, a weighted calculation is performed according to the weight of each decision unit in the base model and the probability output by each decision unit in the base model to obtain the probability output by the base model; a weighted calculation is performed according to the weight of each base model and the probability output by the base model to obtain the total probability; a sample decision result is determined according to the total probability; and the weight of each base model and the weight of each decision unit are adjusted according to the difference between the sample decision result and the annotation result corresponding to the training sample.

[0192] The vehicle control device provided in the embodiment of the present application can implement the various processes implemented in the aforementioned vehicle control method embodiment and achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0193] Figure 8 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown.

[0194] The electronic device may include a processor 801 and a memory 802 storing computer program instructions.

[0195] Specifically, the processor 801 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0196] The memory 802 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 802 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 802 may include removable or non-removable (or fixed) media. Where appropriate, the memory 802 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 802 is a non-volatile solid-state memory.

[0197] The memory may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Thus, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to the first aspect or the second aspect of the present disclosure.

[0198] The processor 801 implements any one of the information auditing methods in the above embodiments by reading and executing computer program instructions stored in the memory 802 .

[0199] In one example, the electronic device may further include a communication interface 803 and a bus 810. Figure 8 As shown, the processor 801, the memory 802, and the communication interface 803 are connected via a bus 810 and communicate with each other.

[0200] The communication interface 803 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0201] Bus 810 comprises hardware, software or both, and the parts of information audit method or verification equipment are coupled to each other.For example, and not limitation, bus can comprise accelerated graphics port (AGP) or other graphics bus, enhanced industry standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industry standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more above these combinations.In suitable cases, bus 810 can comprise one or more buses.Although the present application embodiment describes and shows specific bus, the application considers any suitable bus or interconnection.

[0202] In addition, an embodiment of the present application may provide a vehicle, which includes the electronic device in the above embodiment.

[0203] In addition, in conjunction with the vehicle control method in the above embodiments, the present application may provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the vehicle control methods in the above embodiments is implemented.

[0204] In addition, the embodiments of the present application may be implemented by providing a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device implements any one of the vehicle control methods in the above embodiments.

[0205] It should be understood that the present application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described as examples. However, the method process of the present application is not limited to the specific steps described. Those skilled in the art can make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present application.

[0206] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium that can store or transmit information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0207] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps. In other words, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0208] Aspects of the present disclosure have been described above with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit. It is also understood that each box in the block diagram and / or flowchart and the combination of the boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0209] The above description is only a specific embodiment of the present application. Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the scope of protection of the present application.

Claims

1. A vehicle control method, characterized in that: The method is applied to a vehicle, and comprises: During the driving of the vehicle, collecting multiple frames of images in the driving direction of the vehicle; If it is determined according to the multiple frames of images that a target perception object exists, determining a braking decision result of the vehicle based on perception information of the target perception object, the perception information including a moving speed and a moving direction of the target perception object; According to the braking decision result of the vehicle, the vehicle is controlled to perform a preset operation, where the preset operation includes one of the following: parking, decelerating and continuing to drive.

2. The method according to claim 1, characterized in that After acquiring the multiple frames of images in the direction of travel of the vehicle, if it is determined based on the multiple frames of images that a target perception object exists, and before determining a braking decision result of the vehicle based on perception information of the target perception object, the method further includes: Performing target detection on each of the multiple frames of images to obtain a first detection result for each of the frames of images, where the first detection result includes at least one detected perceived object and a confidence level of the perceived object, or the first detection result does not include the perceived object; Deleting the first detection results that do not include the perceived object from the first detection results of the multiple frames of images to obtain J second detection results, where J is a positive integer; Filter the J second detection results according to the confidence level of the perceived object included in each of the J second detection results to obtain I second detection results, where I is a positive integer and is less than or equal to J; Determine the target perception object based on I of the second detection results.

3. The method according to claim 2, characterized in that The determining the target perception object according to the one second detection result includes: For an image corresponding to each of the one second detection results, if a center point of a perceived object in the image is identified to be located in a preset area of ​​the image, selecting the perceived object as a candidate perceived object; If the same candidate perception object appears in the images corresponding to I of the second detection results more than a preset number of times, the candidate perception object is determined as the target perception object.

4. The method according to claim 1, wherein The determining of the braking decision result of the vehicle according to the perception information of the target perception object includes: Determining a motion trajectory of the target perception object according to first coordinate information of the target perception object in each frame of image; Obtaining a motion speed of the target perception object according to a motion trajectory of the target perception object and a duration corresponding to a plurality of frames of images where the target perception object is located; Determining a moving direction of the target perception object according to the moving trajectory of the target perception object; A braking decision result of the vehicle is determined according to the moving speed of the target perception object and the moving direction of the target perception object.

5. The method according to claim 4, characterized in that Determining the movement direction of the target perception object according to the movement trajectory of the target perception object includes: generating a first vector of the target perception object according to the motion trajectory of the target perception object, wherein the first vector is used to represent the running direction of the target perception object; The moving direction of the target perception object is determined according to the angle between the first vector and the second vector, and the second vector is determined according to the moving trajectory of the vehicle.

6. The method according to claim 5, characterized in that The determining of the braking decision result of the vehicle according to the movement speed and the movement direction of the target perception object includes: Multiplying the movement speed of the target perception object by a preset time length to obtain a first predicted position point of the target perception object; If the angle between the first vector and the second vector is less than a preset angle, and the distance between the first predicted position point and the second predicted position point is less than a preset distance, it is determined that the braking decision result includes a second indication, the second predicted position point is determined based on the movement speed of the vehicle and a preset duration, and the second indication is used to instruct the vehicle to stop or decelerate.

7. The method according to claim 5, characterized in that The determining of the braking decision result of the vehicle according to the movement speed and the movement direction of the target perception object includes: Multiplying the movement speed of the target perception object by a preset time length to obtain a first predicted position point of the target perception object; If the angle between the first vector and the second vector is greater than or equal to a preset angle, and the first predicted position point is within a first area, it is determined that the braking decision result includes a second indication, and the second indication is used to instruct the vehicle to stop or decelerate, wherein the first area is determined based on the vehicle's body width, body length, movement speed and the preset duration.

8. The method according to claim 4, characterized in that The number of target perception objects is M, where M is a positive integer; The determining of the braking decision result of the vehicle according to the movement speed and the movement direction of the target perception object includes: Calculating the distances between the M target perception objects and the vehicle respectively, and screening the M target perception objects according to the M distances to obtain N target perception objects, where N is a positive integer less than or equal to M; The motion speeds, motion directions, categories of the N target perception objects, and the distances between the target perception objects and the vehicle are input into a decision model to obtain a braking decision result of the vehicle, wherein the decision model is trained using multiple training samples extracted from sample videos, and the decision model outputs a probability corresponding to each of the N target perception objects based on the motion speeds, motion directions, categories, and the distances between the perception objects and the vehicle, and: When the probability corresponding to the existence of at least one target perception object among the N target perception objects is greater than a preset probability, determining that the braking decision result of the vehicle includes a second instruction, where the second instruction is used to instruct the vehicle to stop or decelerate; When the probabilities corresponding to the N target perception objects are all less than the preset probability, it is determined that the braking decision result of the vehicle includes a third indication, and the third indication is used to instruct the vehicle to continue driving.

9. The method according to claim 8, characterized in that The decision model includes a plurality of base models, each base model includes a plurality of decision units, the sum of the weights of the decision units in the base model is 1, and the sum of the weights of each base model is 1; The decision model is trained according to the following process: Acquire multiple training samples, where the training samples include the category of the perceived object, the movement speed of the perceived object, the movement direction of the perceived object, the distance between the perceived object and the sample vehicle, and the labeling results of the training samples; The labeling result is used to indicate the braking decision result of the sample video; In each round of training, a training sample is input to each decision unit, and the probability of the perception object included in the training sample output by each decision unit is obtained; For each of the base models, performing weighted calculation according to the weight of each of the decision units in the base model and the probability of the output of each of the decision units in the base model to obtain the probability of the output of the base model; Performing weighted calculation according to the weight of each base model and the probability output by the base model to obtain a total probability; Determine a sample decision result based on the total probability; According to the difference between the sample decision result and the labeling result corresponding to the training sample, the weight of each base model and the weight of each decision unit are adjusted.

10. A vehicle control device, characterized in that: The device is applied to a vehicle and includes: An acquisition module, configured to acquire multiple frames of images in the direction of travel of the vehicle during the vehicle's travel; a decision module, configured to, if it is determined based on the multiple frames of image that a target perception object exists, determine a braking decision result for the vehicle based on perception information of the target perception object, the perception information including a moving speed and a moving direction of the target perception object; A control module is used to control the vehicle to perform a preset operation according to the braking decision result of the vehicle, and the preset operation includes one of the following: stopping, decelerating and continuing to drive.

11. An electronic device, characterized in that: include: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the vehicle control method according to any one of claims 1 to 9 is implemented.

12. A vehicle, characterized in that: Comprising the electronic device as claimed in claim 11.

Images