Vehicle driving planning methods, computer devices and storage media

Abstract

This invention discloses a vehicle driving planning method, computer device, and storage medium. The invention performs a first risk identification process on first image information to identify risk areas in the external scene, achieving preliminary risk detection. By determining the target area based on vehicle control information, areas closely related to the driving process are selected from the initially detected risk areas for further risk detection. A second onboard camera is then used to capture images with shooting control parameters matched to the target area, thereby obtaining richer image details for the target area. A second risk identification process is then performed on this second image information with richer image details, enabling the control module to achieve safer and more sensitive driving planning results with the same computing power, improving the utilization efficiency of computing power and ensuring road traffic safety. This invention has wide applications in the automotive technology field.

Description

Technical Field

[0001] This invention relates to the field of automotive technology, and in particular to an automotive driving planning method, computer device, and storage medium. Background Technology

[0002] Currently, some automotive technologies utilize driving planning functions. Specifically, cameras capture images of the external environment, and image recognition algorithms, such as those using artificial intelligence models, identify elements like roads, pedestrians, other vehicles, and obstacles to enable assisted or autonomous driving. However, during driving, the risks or impacts of different external targets—roads, pedestrians, other vehicles, and obstacles—generally vary. Current driving planning functions lack focus on specific targets, resulting in the need for more computing power to achieve the same level of safety and responsiveness, or, with the same computing power, achieving lower safety and responsiveness—in other words, lower computing power utilization efficiency. Summary of the Invention

[0003] In view of the technical problems of low computational efficiency in current vehicle driving planning technology, the purpose of this invention is to provide a vehicle driving planning method, computer device and storage medium.

[0004] On one hand, embodiments of the present invention include a vehicle driving planning method, the vehicle driving planning method comprising the following steps:

[0005] The first image information is obtained by capturing the scene outside the vehicle using the first vehicle-mounted camera;

[0006] A first risk identification process is performed on the first image information to determine at least one risk area in the first image information;

[0007] Obtain vehicle control information;

[0008] Based on the vehicle control information, a target area is determined; the target area is at least a portion of the risk areas among the risk areas.

[0009] Based on the target area, determine the shooting control parameters;

[0010] Using the aforementioned shooting control parameters, the second vehicle-mounted camera is controlled to capture images of the external scene to obtain second image information;

[0011] A second risk identification process is performed on the second image information, and driving planning is carried out based on the risk identification results of the second risk identification process.

[0012] Furthermore, the step of capturing images of the exterior scene using the first vehicle-mounted camera to obtain first image information includes:

[0013] Control the first vehicle-mounted camera to continuously capture images of the exterior scene using fixed shooting parameters;

[0014] Obtain the time series of images captured by the first vehicle-mounted camera; the image time series includes multiple pieces of the first image information sorted by capture time.

[0015] Furthermore, the acquisition of vehicle control information includes:

[0016] Obtain voice information from passengers in the vehicle;

[0017] The voice information is semantically recognized to obtain the vehicle control information.

[0018] Furthermore, the acquisition of vehicle control information includes:

[0019] The usage status of the vehicle-mounted equipment is detected to obtain equipment usage status information;

[0020] The vehicle control information is determined based on the device usage status information.

[0021] Furthermore, the acquisition of vehicle control information includes:

[0022] The interaction between this vehicle and its external environment is detected, and interaction recording information is obtained;

[0023] The vehicle control information is determined based on the interaction record information.

[0024] Further, determining the target area based on the vehicle control information includes:

[0025] The risk value of each of the aforementioned risk areas is determined through the first risk identification process.

[0026] The correlation between the vehicle control information and each of the risk areas is obtained respectively;

[0027] For any of the aforementioned risk areas, an influencing factor is determined based on the corresponding risk value and the correlation.

[0028] Based on each of the aforementioned influencing factors, at least a portion of the aforementioned risk areas are identified as the target areas.

[0029] Further, determining the shooting control parameters based on the target area includes:

[0030] The motion state of the target area is detected to obtain first motion state information;

[0031] The motion state of the vehicle is detected to obtain second motion state information;

[0032] The shooting control parameters are determined based on the first motion state information and the second motion state information.

[0033] Further, determining the shooting control parameters based on the first motion state information and the second motion state information includes:

[0034] Based on the first motion state information and the second motion state information, the relative motion state information between the vehicle and the target area is determined;

[0035] Obtain the current shooting parameters of the second vehicle-mounted camera;

[0036] The estimated shooting parameters are determined based on the relative motion state information;

[0037] Based on the current shooting parameters and the estimated shooting parameters, determine the duration of the shooting parameter change of the second vehicle-mounted camera;

[0038] The target shooting position is determined based on the relative motion state information and the duration of the shooting parameter changes;

[0039] The shooting control parameters are determined based on the target shooting position.

[0040] On the other hand, embodiments of the present invention also include a computer device, including a memory and a processor, the memory for storing at least one program, and the processor for loading at least one program to execute the vehicle driving planning method of the embodiments.

[0041] On the other hand, embodiments of the present invention also include a computer-readable storage medium storing a processor-executable program, which, when executed by a processor, is used to perform the vehicle driving planning method in the embodiments.

[0042] The beneficial effects of the present invention are as follows: The vehicle driving planning method in the embodiments obtains first image information by using a first vehicle-mounted camera to capture the external scene, enabling continuous environmental perception of the external scene. By performing a first risk identification process on the first image information, risk areas existing in the external scene can be discovered, achieving preliminary risk detection of the external scene. By determining the target area based on vehicle control information, areas closely related to the vehicle driving process can be selected from the risk areas discovered in the preliminary risk detection for further risk detection. A second vehicle-mounted camera is then invoked to capture images with shooting control parameters matched to the target area, thereby obtaining richer image details for the target area. A second risk identification process is then performed on the second image information with richer image details, enabling attention to specific high-risk targets. This allows the control module to achieve safer and more sensitive driving planning effects under the same computing power, improving the utilization efficiency of computing power and ensuring road traffic safety. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of a vehicle system to which the vehicle driving planning method can be applied in the embodiment.

[0044] Figure 2 This is a schematic diagram of the structure of the second vehicle-mounted camera in the embodiment;

[0045] Figure 3 This is a schematic diagram illustrating the steps of the vehicle driving planning method in the embodiment;

[0046] Figure 4 This is a flowchart illustrating the step of determining the shooting control parameters based on the first motion state information and the second motion state information in this embodiment. Detailed Implementation

[0047] In this embodiment, the vehicle driving planning method can be applied to... Figure 1 The vehicle system shown. (Refer to...) Figure 1 The vehicle system includes a control module, a first vehicle-mounted camera, and a second vehicle-mounted camera, all installed on the same vehicle. The control module is a device with data processing and control functions, and can utilize the vehicle's electronic control unit. The first and second vehicle-mounted cameras each have imaging capabilities. They can be separate cameras, integrated into a single device, or use the same camera component as both the first and second vehicle-mounted cameras. That is, when the control module needs to access the first vehicle-mounted camera, it invokes that camera component to perform the functions of the first camera; similarly, when the control module needs to access the second vehicle-mounted camera, it invokes that camera component to perform the functions of the second camera.

[0048] In this embodiment, the first vehicle-mounted camera is a fixed camera, and the second vehicle-mounted camera is a variable camera, as an example. The shooting parameters of the first vehicle-mounted camera, such as the angle of view, focal length, and aperture, are fixed. A component such as a car dashcam can be used as the first vehicle-mounted camera; refer to... Figure 2 The second vehicle-mounted camera is equipped with a rotation mechanism, a zoom mechanism, and an aperture mechanism. It can be mounted on the roof of a car via the rotation mechanism. The rotation mechanism adjusts the field of view, the zoom mechanism adjusts the zoom, and the aperture mechanism changes the aperture size. In other words, the second vehicle-mounted camera's shooting parameters, such as angle of view, focal length, and aperture, are variable. For example, by rotating, the second vehicle-mounted camera can adjust its field of view to the front, left, right, or rear of the car; by zooming, it can zoom in or out; and by changing the aperture size, it can adjust the depth of field, thereby achieving a blurred background effect to highlight a specific target.

[0049] In this embodiment, refer to Figure 3 The car driving planning method includes the following steps:

[0050] S1. Capture the scene outside the vehicle using the first vehicle-mounted camera to obtain first image information;

[0051] S2. Perform a first risk identification process on the first image information to determine at least one risk area in the first image information;

[0052] S3. Obtain vehicle control information;

[0053] S4. Determine the target area based on vehicle control information;

[0054] S5. Determine the shooting control parameters based on the target area;

[0055] S6. Using the shooting control parameters, control the second vehicle-mounted camera to shoot the scene outside the vehicle and obtain second image information;

[0056] S7. Perform a second risk identification process on the second image information, and make a driving plan based on the risk identification results of the second risk identification process.

[0057] In this embodiment, the control module executes steps S1-S7 and other steps in the vehicle driving planning method.

[0058] In step S1, the control module calls the first vehicle-mounted camera to capture images of the scene outside the vehicle. The images captured by the first vehicle-mounted camera are the first image information. Specifically, the first vehicle-mounted camera can capture images with fixed shooting parameters such as viewing angle, focal length, and aperture. Each frame of the image captured is the first image information. Multiple consecutive frames of the first image information, sorted by shooting time, form an image time sequence, which in turn forms a video stream.

[0059] In step S2, the control module can run a trained artificial intelligence model such as a convolutional neural network, inputting the first image information into the convolutional neural network for processing. The trained convolutional neural network has the ability to identify the regions where specific targets are located in the first image information and marks these regions as risk areas. The convolutional neural network can also analyze the risk areas and output the risk value of each risk area. For example, a convolutional neural network trained on training data such as images of people, obstacles, cars, and roads can identify specific targets in the first image information, such as pedestrians, obstacles (e.g., traffic cones), cars, intersections, zebra crossings, signs, and turning lanes. It then marks the areas of these specific targets in the first image information as risk regions. The convolutional neural network outputs the risk regions (or their coordinates in the first image information) and further performs image analysis on these risk regions, outputting their respective risk values. [For example, for any risk region, the smaller the distance between the risk region and the vehicle (specifically, the distance between the risk region and a feature point such as the center of the first image information), the greater the risk value of the risk region. For risk regions containing pedestrians, the posture of the pedestrians in the risk region can be detected; pedestrians looking down at their phones have a higher risk value, while pedestrians walking normally have a lower risk value. For risk regions containing turning lanes, the radius of curvature of the turning lane can be detected; the smaller the radius of curvature, the greater the risk value of the risk region.] This completes the first risk identification process, i.e., the identification of risk regions in the first image information.

[0060] In step S3, the control module can access the sensors installed on the vehicle to detect and obtain vehicle control information. This vehicle control information includes information related to the control of the vehicle, such as the extent to which the driver presses the accelerator pedal, brake pedal, or steering wheel; or the actions of passengers such as opening doors, adjusting window openings, adjusting seatbelt extensions, or moving in their seats; or various control commands issued to the control module by the driver, passengers, or other occupants via voice, touch, or other human-computer interaction methods; or information such as the vehicle reaching a specific navigation location, a specific driving trajectory, and the speed and acceleration generated during driving; or a combination of the above information.

[0061] In step S4, the control module selects at least one risk area as the target area from all the risk areas obtained in step S2 based on the vehicle control information. Since the vehicle control information represents information such as the driver's or passenger's control of the vehicle, by executing step S4, risk areas closely related to the vehicle's control can be selected as target areas.

[0062] In step S5, the control module determines the corresponding shooting control parameters based on the target area selected in step S4. The specific form of the shooting control parameters can be angle of view, focal length, aperture, etc. In step S6, the control module sends the shooting control parameters determined in step S5 to the second vehicle-mounted camera, and controls the second vehicle-mounted camera to adjust to the shooting parameters such as angle of view, focal length, and aperture corresponding to the shooting control parameters, and shoots the scene outside the vehicle to obtain second image information.

[0063] In step S7, the control module can run a trained artificial intelligence model such as a convolutional neural network, inputting the second image information into the convolutional neural network for processing. Similar to the principle of step S2, by executing step S7, the risk value of the content of the second image information on the driving process of the vehicle can be identified. This risk value can be used as the risk identification result of the second risk identification process, and the control module can perform driving planning based on the risk value of the second image information.

[0064] Specifically, when the control module performs driving planning, it can select between assisted driving and autonomous driving modes. When assisted driving mode is selected, the driver manually drives. The control module can output assisted driving guidance information based on the risk identification results of the second risk identification process. For example, if the second image information identifies an area containing the word "pedestrian" and the risk value of that area exceeds a threshold, the control module can generate a voice message saying "Please note pedestrians ahead" and play it through the vehicle's speakers to guide the driver to drive safely. When autonomous driving mode is selected, the control module can generate autonomous driving commands based on the risk identification results of the second risk identification process. These commands control the vehicle's powertrain, transmission, steering, and braking systems. For example, if the second image information identifies an area containing the word "pedestrian" and the risk value of that area exceeds a threshold, the control module can control the powertrain and transmission to reduce power output and control the braking system to brake, causing the vehicle to slow down and avoid pedestrians.

[0065] In this embodiment, the principle of executing steps S1-S7 is as follows: by using the first vehicle-mounted camera to capture the external scene to obtain first image information, continuous environmental perception of the external scene can be achieved. By performing a first risk identification process on the first image information, risk areas existing in the external scene can be discovered, realizing preliminary risk detection of the external scene. By determining the target area based on the vehicle control information, areas closely related to the driving process of the vehicle can be screened from the risk areas discovered in the preliminary risk detection for further risk detection. The second vehicle-mounted camera is then called to capture images with shooting control parameters matched to the target area, thereby obtaining richer image details for the target area. A second risk identification process is then performed on the second image information with richer image details, enabling attention to specific high-risk targets. This allows the control module to achieve safer and more sensitive driving planning effects under the same computing power, improving the utilization efficiency of computing power and ensuring road traffic safety.

[0066] In this embodiment, when the control module executes step S3, which is the step of acquiring vehicle control information, it can specifically perform the following steps:

[0067] S301A. Obtain voice information from passengers in the vehicle;

[0068] S302A. Performs semantic recognition on voice information to obtain vehicle control information.

[0069] Steps S301A-S302A are the first execution method of step S3.

[0070] The control module executes step S301A, acquiring the voice information of the occupants from the microphone and performing semantic recognition on the voice information using a voice recognition algorithm. For example, for the voice message "I am a new driver and unfamiliar with the traffic environment," the control module, through step S302A, may extract the semantic recognition result of "traffic" as the vehicle control information to be obtained in step S3. In subsequent step S4, the control module can, based on the vehicle control information containing "traffic," identify risk areas containing "car" from the risk areas of the first image information as target areas. In step S5, based on the positions of these target areas, it determines the corresponding viewing angle, focal length, and aperture, and generates corresponding shooting control parameters. In step S6, it controls the second vehicle-mounted camera to shoot according to these shooting control parameters, thereby capturing clearer images of the specific risk target "car," which are then used for the second risk identification process and driving planning in step S7.

[0071] By executing steps S301A-S302A, the specific risk targets that the occupants wish to pay special attention to can be determined based on their subjective desires, such as driving experience. The second vehicle-mounted camera is then used to capture close-up shots of these specific risk targets, thereby enabling driving planning decisions based on these targets and improving the efficiency of computing power utilization.

[0072] In this embodiment, when the control module executes step S3, which is the step of acquiring vehicle control information, it can specifically perform the following steps:

[0073] S301B. Detect the usage status of vehicle-mounted equipment and obtain equipment usage status information;

[0074] S302B. Determine vehicle control information based on equipment usage status information.

[0075] Steps S301B-S302B are the second execution method of step S3.

[0076] In step S301B, the control module can call sensors such as the window opening sensor, the seatbelt extension length sensor, and the seat pressure sensor to detect the usage status of in-vehicle devices such as windows, child locks, and seats. Specifically, the device usage status information of a window can indicate the opening degree of a certain window, the device usage status information of a child lock can indicate whether the child lock function of a certain door is activated, and the device usage status information of a seat can indicate whether a certain seat is being used.

[0077] In step S302B, the control module uses the device usage status information obtained in step S301B as the vehicle control information to be obtained in step S3. When executing step S4, the control module can determine the location of the target area from the first image information based on the device usage status information, and identify the risk areas located in those locations from the risk areas of the first image information as the target area. Then, in step S5, the control module determines the corresponding viewing angle, focal length, and aperture based on the location of the target area and generates corresponding shooting control parameters. In step S6, the control module controls the second vehicle-mounted camera to shoot according to these shooting control parameters, thereby capturing clearer images of the specific risk target "vehicle," which are then used for the second risk identification process and driving planning in step S7.

[0078] For example, if the window opening sensor detects that the right rear window opening has decreased by a preset range (e.g., increased by more than 50%), then in step S302B, this device usage status information is used as vehicle control information. In subsequent step S4, the control module can locate the risk area as the target area along the right rear direction in the first image information, and determine the corresponding focal length and aperture based on the distance between the target area and the center of the first image information (corresponding to the vehicle's position), thereby generating shooting control parameters. When the control module executes step S6, it controls the second vehicle-mounted camera to shoot according to these shooting control parameters, thereby enabling close-up shots of these target areas to obtain clearer images for the second risk identification process and driving planning.

[0079] For example, when the child lock controller detects that the child lock function on the right rear door is activated, in step S302B, this device usage status information is used as vehicle control information. In subsequent step S4, the control module can locate the risk area as the target area along the right rear direction in the first image information, and determine the corresponding focal length and aperture based on the distance between the target area and the center of the first image information (corresponding to the vehicle's position), thereby generating shooting control parameters. When the control module executes step S6, it controls the second vehicle-mounted camera to shoot according to these shooting control parameters, thereby enabling close-up shots of these target areas to obtain clearer images for the second risk identification process and driving planning.

[0080] For example, if the seat pressure sensor detects that the pressure on the right rear seat is decreasing and the pressure on the left rear seat is increasing, then in step S302B, this device usage status information is used as vehicle control information. In subsequent step S4, the control module can locate the risk area as the target area along the direction (right side) from the seat with increasing pressure to the seat with decreasing pressure in the first image information. The distance between the target area and the center of the first image information (corresponding to the vehicle's position) determines the corresponding focal length and aperture, thereby generating shooting control parameters. When the control module executes step S6, it controls the second vehicle-mounted camera to shoot according to these shooting control parameters, enabling close-up shots of these target areas to obtain clearer images for the second risk identification process and driving planning.

[0081] By executing steps S301B-S302B, the physical movements of occupants can be detected by monitoring the usage status of in-vehicle devices such as windows, child locks, and seats, thereby identifying high-risk target areas requiring special attention. For example, when occupants perceive risks such as crosswinds or large vehicles on their side, they typically close the window on that side or move away from it. These actions result in reduced window opening and decreased pressure on one seat while increasing pressure on the other. When children are present, the risk on the side where the child is located should be given special attention. Typically, the child lock on the door on the child's side will be activated. Therefore, the location of the target area can be determined by observing the direction of the door with the child lock activated. The device usage status information detected by steps S301B-S302B allows the location of high-risk target areas requiring special attention to be determined from the first image information. This determines the shooting control parameters for capturing the target area and controls the second in-vehicle camera to capture the image. Based on the execution steps S301B-S302B, steps S4-S7 are executed, which can automatically capture images of the target area of ​​the scene outside the vehicle when the people in the vehicle make instinctive physical movements to avoid risks. This is conducive to timely identification of driving risks in the scene outside the vehicle and ensures road traffic safety.

[0082] In this embodiment, when the control module executes step S3, which is the step of acquiring vehicle control information, it can specifically perform the following steps:

[0083] S301C. Detects the interaction between this vehicle and the external environment and obtains interaction recording information;

[0084] S302C. Determine vehicle control information based on interaction record information.

[0085] Steps S301C-S302C are the third execution method of step S3.

[0086] In step S301C, the interaction between the vehicle and the external environment includes collisions, overtaking, and other forms. The control module can call the acceleration sensor installed on the vehicle body to detect collisions; the control module can also call the side vehicle position sensor installed on the vehicle body to detect the position of the side vehicle and determine whether the vehicle has overtaken the side vehicle during driving, indicating an overtaking-type interaction.

[0087] In step S302C, the control module uses the device usage status information obtained in step S301C as the vehicle control information to be obtained in step S3. When executing step S4, the control module can, based on the vehicle control information containing "many vehicles," identify risk areas containing "vehicle" from the risk areas of the first image information as target areas. In step S5, it determines the corresponding viewing angle, focal length, and aperture based on the positions of these target areas and generates corresponding shooting control parameters. In step S6, it controls the second vehicle-mounted camera to shoot according to these shooting control parameters, thereby capturing clearer images of the specific risk target "vehicle," which are then used for the second risk identification process and driving planning in step S7.

[0088] For example, when the accelerometer detects that the vehicle's acceleration exceeds a threshold (e.g., 5g) and the direction of acceleration is directly in front of the vehicle, in step S302C, this interaction information is used as vehicle control information. In subsequent step S4, the control module can locate the risk area as the target area along the direction opposite to the acceleration (directly behind) in the first image information, and determine the corresponding focal length and aperture based on the distance between the target area and the center of the first image information (corresponding to the vehicle's position), thereby generating shooting control parameters. When the control module executes step S6, it controls the second vehicle-mounted camera to shoot according to these shooting control parameters, thereby enabling close-up shots of these target areas to obtain clearer images for the second risk identification process and driving planning.

[0089] For example, when the vehicle's position sensor detects that the vehicle has passed the vehicle on its right, in step S202C, this interaction information is used as vehicle control information. In subsequent step S4, the control module can locate the risk area as the target area along the right side of the first image information, and determine the corresponding focal length and aperture based on the distance between the target area and the center of the first image information (corresponding to the vehicle's position), thereby generating shooting control parameters. When the control module executes step S6, it controls the second vehicle-mounted camera to shoot according to these shooting control parameters, thereby enabling close-up shots of these target areas to obtain clearer images for the second risk identification process and driving planning.

[0090] By executing steps S301C-S302C, the interaction between the vehicle and its surroundings can be detected to determine the target area of ​​interest. For example, if the acceleration sensor detects excessive acceleration of the vehicle, it usually indicates a collision with a neighboring vehicle or obstacle. The opposite direction of the vehicle's acceleration is typically the direction of the neighboring vehicle or obstacle. Therefore, by executing steps S301C-S302C, the interaction between the vehicle and its surroundings detected can determine the location of the target area of ​​interest from the first image information. This allows for the determination of the shooting control parameters that can capture the target area, controlling the second vehicle-mounted camera to capture the image. Based on executing steps S301C-S302C, steps S4-S7 can be executed. In the event of a collision, especially when the collision originates from the side or rear of the vehicle—areas traditionally blind spots for dashcams—the second vehicle-mounted camera can be promptly used to capture the direction of the collision. This helps preserve evidence at the scene and prevents the loss of evidence due to hit-and-run incidents. For example, if the vehicle's position sensor detects that the vehicle has overtaken another vehicle, and since overtaking is riskier than normal driving, steps S301C-S302C are executed to detect the interaction between the vehicle and the outside world. This allows the location of the target area to be identified from the first image information, and the shooting control parameters for capturing the target area are determined to control the second vehicle-mounted camera to capture the image. Based on steps S301C-S302C, steps S4-S7 are executed. This allows attention to high-risk driving processes such as overtaking, and more computing power from the control module is allocated to perform a second risk identification process and driving planning for high-risk target areas. This helps ensure road traffic safety and improves the efficiency of computing power utilization.

[0091] In this embodiment, when the control module executes step S4, which is the step of determining the target area based on the vehicle control information, it can specifically perform the following steps:

[0092] S401. Through the first risk identification process, determine the risk value of each risk area;

[0093] S402. Obtain the correlation between vehicle control information and each risk area;

[0094] S403. For any risk area, determine the influencing factors based on the corresponding risk value and correlation.

[0095] S404. Based on each influencing factor, identify at least some of the risk areas as target areas.

[0096] In step S401, the control module obtains the risk value of each risk area obtained in step S2.

[0097] In step S402, the control module can use AI tools such as image-to-text tools to convert the content of each risk area into text with corresponding meanings, and calculate the correlation between the text and the corresponding vehicle control information to obtain the correlation between the vehicle control information and each risk area. For example, when multiple risk areas are detected in the first image information, the text corresponding to each risk area is obtained through image-to-text tools, such as "one vehicle", "multiple vehicles", "pedestrian", and "intersection". For the vehicle control information obtained in step S301A, which is "I am a novice driver and am not familiar with the environment with many cars", the risk area with the corresponding text "multiple vehicles" has the highest correlation with the vehicle control information, the risk area with the corresponding text "one vehicle" has the second highest correlation, and the risk areas with the corresponding text "pedestrian" and "intersection" have the lowest correlation with the vehicle control information.

[0098] In step S403, for the i-th risk area, let its risk value be risk. i The correlation between it and the vehicle control information detected in step S3 is denoted as correlation. i It can be based on its risk value. i correlation i The product of these factors determines its influence factor. i That is, factor i =risk i ×correlation i .

[0099] In step S404, a threshold can be set to include all risk areas with corresponding impact factors greater than the threshold as target areas, or the impact factors of each risk area can be sorted and the risk areas with the largest corresponding impact factors can be included as target areas.

[0100] In this embodiment, the principle of steps S401-S404 is as follows: the risk value of the risk area represents the objective impact of the target in the risk area on the driving risk of the vehicle, and the correlation between the risk area and the vehicle control information represents the impact of the target in the risk area on the occupants of the vehicle, that is, the subjective impact of the driving risk on the occupants. By executing steps S401-S404, the subjective and objective impacts of driving risk can be integrated, which is conducive to selecting several risk areas with the greatest impact as target areas for attention, mobilizing more computing power to perform the second risk identification process and driving planning for the target areas, ensuring road traffic safety, and improving the utilization efficiency of computing power.

[0101] In this embodiment, when the control module executes step S5, which is the step of determining the shooting control parameters based on the target area, it can specifically perform the following steps:

[0102] S501. Detect the motion state of the target area and obtain the first motion state information;

[0103] S502. Detect the motion state of the vehicle and obtain second motion state information;

[0104] S503. Determine the shooting control parameters based on the first motion state information and the second motion state information.

[0105] In step S501, the first motion state information represents the motion state of a target (e.g., a pedestrian, a car, etc.) in the target area, such as its speed. In step S502, the second motion state information represents the motion state of the vehicle itself, such as its speed. The first motion state information can be obtained by detecting the target area using sensors such as radar, and the second motion state information can be obtained by detecting the vehicle itself using sensors such as speed sensors and acceleration sensors installed on the vehicle.

[0106] When executing steps S501 and S502, it is not necessary to detect the individual values ​​of the first motion state information and the second motion state information separately; instead, only the difference between them, i.e., the relative motion state information between the vehicle and the target area, needs to be detected. Since the first image information in video stream form can be detected in step S1, where the coordinates of the vehicle in the first image information are fixed, the coordinate change (Δx) of the target area between two consecutive first image information images can be calculated by detecting the position of the target area in two or more consecutive first image information images. target Δy target The system obtains the time difference Δt between the two first images to acquire the relative motion state information (V) between the vehicle and the target area. x V y ),in Among them, V x V represents the x-component of the velocity of the target area relative to the vehicle. y The y-component represents the velocity of the target area relative to the vehicle.

[0107] In step S503, the relative motion state information (V) can be used as a basis. x V y Determine the shooting control parameters. Specifically, refer to... Figure 4The control module detects the current shooting parameters (θ0, d0, f0) of the second vehicle-mounted camera, where θ0 represents the current field of view of the second vehicle-mounted camera, d0 represents the current focal length of the second vehicle-mounted camera, and f0 represents the current aperture size of the second vehicle-mounted camera.

[0108] Next, refer to Figure 4 The control module uses relative motion state information (V) x V y The size and orientation of the image are used to obtain the estimated shooting parameters (θ) from the calibrated database. estimatr d estimate fe stimate ), including the estimated shooting parameters (θ) estimatr d estimate f estimate () indicates that the value determined through calibration experiments is suitable for capturing images with relative motion state information (V). x V y The parameters of the target area, such as the angle of view, focal length, and aperture.

[0109] Next, refer to Figure 4 The control module uses the current shooting parameters (θ0, d0, f0) and the estimated shooting parameters (θ) to determine the shooting parameters. estimatr d estimate f estimate The duration ΔT for the change in shooting parameters is retrieved from the calibrated database. Here, ΔT represents the time it takes for the second vehicle-mounted camera to switch from the current shooting parameters (θ0, d0, f0) to the estimated shooting parameters (θ0, d0, f0), as determined through calibration experiments. estimatr d estimate f estimate The time required.

[0110] Next, refer to Figure 4 The control module uses relative motion state information (V) x V y The target shooting position is determined by the shooting parameter change time ΔT. in Among them, the target shooting location It can reflect the position of the target area (relative to the first image information) after the second vehicle-mounted camera has adjusted its shooting parameters.

[0111] After obtaining the target shooting location Subsequently, in step S304, the control module retrieves the shooting control parameters (θ1, d1, f1) from the calibrated database, where the shooting control parameters (θ1, d1, f1) represent the suitable shooting position at the target shooting location determined through calibration experiments. The parameters of the target area, such as the angle of view, focal length, and aperture.

[0112] When performing step S503, a coefficient K can also be set, and the coefficient K can be multiplied with the shooting control parameters (θ1, d1, f1) to adjust the shooting control parameters (θ1, d1, f1) in order to reduce the influence of errors in the calculation process and enable the second vehicle-mounted camera to capture the target area.

[0113] In this embodiment, by executing steps S501-S503, suitable shooting control parameters can be generated when there is relative motion between the target area and the vehicle, thereby controlling the second vehicle-mounted camera to capture the target area, which is beneficial for capturing the second image information.

[0114] A computer program that executes the vehicle driving planning method in this embodiment can be written into a computer device or storage medium. When the computer program is read out and run, the vehicle driving planning method in this embodiment is executed, thereby achieving the same technical effect as the vehicle driving planning method in the embodiment.

[0115] It should be noted that, unless otherwise specified, when a feature is referred to as "fixed" or "connected" to another feature, it can be directly fixed or connected to the other feature, or indirectly fixed or connected to the other feature. Furthermore, the descriptions of "upper," "lower," "left," and "right" used in this disclosure are only relative to the relative positional relationships of the components of this disclosure in the accompanying drawings. The singular forms "a," "an," and "the" used in this disclosure are also intended to include the plural forms, unless the context clearly indicates otherwise. Moreover, unless otherwise defined, all technical and scientific terms used in this embodiment have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this embodiment specification is only for describing particular embodiments and is not intended to limit the invention. The term "and / or" as used in this embodiment includes any combination of one or more of the associated listed items.

[0116] It should be understood that although the terms first, second, third, etc., may be used to describe various elements in this disclosure, these elements should not be limited to these terms. These terms are only used to distinguish elements of the same type from each other. For example, a first element may also be referred to as a second element without departing from the scope of this disclosure, and similarly, a second element may also be referred to as a first element. The use of any and all instances or exemplary language (“e.g.,” “such as,” etc.) provided in this embodiment is intended only to better illustrate embodiments of the invention and, unless otherwise required, does not impose a limitation on the scope of the invention.

[0117] It should be recognized that embodiments of the present invention can be implemented or carried out by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable storage medium. The method can be implemented using standard programming techniques—including a non-transitory computer-readable storage medium configured with a computer program, wherein such a storage medium causes the computer to operate in a specific and predefined manner—according to the methods and drawings described in the specific embodiments. Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with the computer system. However, if desired, the program can be implemented in assembly or machine language. In any case, the language can be a compiled or interpreted language. Furthermore, for this purpose, the program can run on a programmed application-specific integrated circuit (ASIC).

[0118] Furthermore, the procedures described in this embodiment can be performed in any suitable order unless otherwise indicated by this embodiment or clearly contradicted by the context. The procedures (or variations and / or combinations thereof) described in this embodiment can be executed under the control of one or more computer systems configured with executable instructions, and can be implemented by hardware or a combination thereof as code (e.g., executable instructions, one or more computer programs, or one or more applications) that commonly executes on one or more processors. A computer program includes multiple instructions executable by one or more processors.

[0119] Furthermore, the method can be implemented in any suitable type of computing platform, including but not limited to personal computers, minicomputers, mainframes, workstations, networked or distributed computing environments, standalone or integrated computer platforms, or in communication with charged particle tools or other imaging devices, etc. Aspects of the invention can be implemented as machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, optical read and / or write storage medium, RAM, ROM, etc., such that it is readable by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the processes described herein. Furthermore, the machine-readable code, or portions thereof, can be transmitted via wired or wireless networks. The invention of this embodiment includes these and other different types of non-transitory computer-readable storage media when such media comprises instructions or programs that implement the steps above in conjunction with a microprocessor or other data processor. When programmed according to the methods and techniques of the invention, the invention also includes the computer itself.

[0120] A computer program can be applied to input data to perform the functions of this embodiment, thereby transforming the input data to generate output data stored in non-volatile memory. The output information can also be applied to one or more output devices, such as a display. In a preferred embodiment of the invention, the transformed data represents physical and tangible objects, including specific visual depictions of physical and tangible objects generated on the display.

[0121] The above are merely preferred embodiments of the present invention. The present invention is not limited to the above-described embodiments. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention, as long as they achieve the technical effects of the present invention by the same means, should be included within the scope of protection of the present invention. Within the scope of protection of the present invention, the technical solutions and / or implementation methods can have various modifications and variations.

Claims

1. A vehicle driving planning method, characterized in that, The vehicle driving planning method includes: The first image information is obtained by capturing the scene outside the vehicle using the first vehicle-mounted camera; A first risk identification process is performed on the first image information to determine at least one risk area in the first image information; Obtain vehicle control information; Based on the vehicle control information, a target area is determined; the target area is at least a portion of the risk areas among the risk areas. Based on the target area, determine the shooting control parameters; Using the aforementioned shooting control parameters, the second vehicle-mounted camera is controlled to capture images of the external scene to obtain second image information; A second risk identification process is performed on the second image information, and driving planning is carried out based on the risk identification results of the second risk identification process. The step of determining the shooting control parameters based on the target area includes: The motion state of the target area is detected to obtain first motion state information; The motion state of the vehicle is detected to obtain second motion state information; Based on the first motion state information and the second motion state information, the relative motion state information between the vehicle and the target area is determined; Obtain the current shooting parameters of the second vehicle-mounted camera; Based on the relative motion state information, the estimated shooting parameters are determined; Based on the current shooting parameters and the estimated shooting parameters, determine the duration of the shooting parameter change of the second vehicle-mounted camera; The target shooting position is determined based on the relative motion state information and the duration of the shooting parameter changes; The shooting control parameters are determined based on the target shooting position.

2. The vehicle driving planning method according to claim 1, characterized in that, The step of capturing images of the exterior scene using the first vehicle-mounted camera to obtain first image information includes: Control the first vehicle-mounted camera to continuously capture images of the exterior scene using fixed shooting parameters; Obtain the time series of images captured by the first vehicle-mounted camera; the image time series includes multiple pieces of the first image information sorted by capture time.

3. The vehicle driving planning method according to claim 1, characterized in that, The acquisition of vehicle control information includes: Obtain voice information from passengers in the vehicle; The voice information is semantically recognized to obtain the vehicle control information.

4. The vehicle driving planning method according to claim 1, characterized in that, The acquisition of vehicle control information includes: The usage status of the vehicle-mounted equipment is detected to obtain equipment usage status information; The vehicle control information is determined based on the device usage status information.

5. The vehicle driving planning method according to claim 1, characterized in that, The acquisition of vehicle control information includes: The interaction between this vehicle and its external environment is detected, and interaction recording information is obtained; The vehicle control information is determined based on the interaction record information.

6. The vehicle driving planning method according to claim 1, characterized in that, Determining the target area based on the vehicle control information includes: The risk value of each of the aforementioned risk areas is determined through the first risk identification process. The correlation between the vehicle control information and each of the risk areas is obtained respectively; For any of the aforementioned risk areas, an influencing factor is determined based on the corresponding risk value and the correlation. Based on each of the aforementioned influencing factors, at least a portion of the aforementioned risk areas are identified as the target areas.

7. A computer device, characterized in that, It includes a memory and a processor, the memory being used to store at least one program, and the processor being used to load at least one program to execute the vehicle driving planning method according to any one of claims 1-6.

8. A computer-readable storage medium storing a processor-executable program, characterized in that, The processor-executable program, when executed by the processor, is used to perform the vehicle driving planning method according to any one of claims 1-6.

Images