Vehicle control method, device and equipment, vehicle and product

Abstract

The invention provides a vehicle control method, device and equipment, a vehicle and a product, and the method comprises the steps: obtaining multiple frames of original images, the multiple frames of original images are obtained through the shooting of the vehicle by a visual sensor on the vehicle based on a target shooting frequency; determining the actual driving speed of the vehicle based on the target image of the target fusion frame number in the multiple frames of original images; the target shooting frequency and the target fusion frame number are correspondingly adjusted according to the change of the driving scene of the vehicle; and controlling the vehicle based on the actual driving speed of the vehicle. According to the embodiment of the invention, both the speed recognition accuracy and the speed recognition efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle control technology, and in particular to a vehicle control method, device, equipment, vehicle and product. Background Art

[0002] In the operation of autonomous vehicles, accurately grasping the actual driving speed of the vehicle is the basis for achieving safe and efficient control. Accurate identification of vehicle speed, as a key link in the autonomous driving technology system, is directly related to the vehicle's dynamic control ability and driving safety. Therefore, ensuring the accuracy of vehicle speed identification is crucial to improving the overall autonomous driving performance.

[0003] At present, speed recognition using image recognition technology has become one of the mainstream methods in the field of autonomous driving. This method mainly relies on the on-board camera to capture images of the front or surrounding environment of the vehicle in real time, and then use image processing and computer vision algorithms to analyze the movement of objects in the image sequence to infer the vehicle's speed.

[0004] Although it is currently possible to identify vehicle speed through image analysis, its accuracy still needs to be improved. Summary of the invention

[0005] Based on the above technical status, the present application proposes a vehicle control method, device, equipment, vehicle and product, which can improve the recognition accuracy of vehicle speed, thereby improving the control accuracy of autonomous driving vehicles and improving vehicle driving safety.

[0006] In order to achieve the above technical objectives, this application specifically proposes the following technical solutions:

[0007] In a first aspect, the present application proposes a vehicle control method, comprising: acquiring multiple frames of original images, wherein the multiple frames of original images are obtained by photographing the vehicle based on a target shooting frequency by a visual sensor on the vehicle; determining an actual driving speed of the vehicle based on a target image of a target fusion frame number in the multiple frames of original images; adjusting the target shooting frequency and the target fusion frame number accordingly according to changes in the driving scene of the vehicle; and controlling the vehicle based on the actual driving speed of the vehicle.

[0008] In some implementations, the driving scene includes a driving state; the driving state includes a stable driving state and an unstable driving state; the target shooting frequency and the target fusion frame number are determined by the following steps: when the vehicle driving state is a stable driving state, the target shooting frequency is set to a first shooting frequency, and the target fusion frame number is set to a first fusion frame number; when the vehicle driving state is an unstable driving state, the target shooting frequency is set to a second shooting frequency, and the target fusion frame number is set to a second fusion frame number; wherein, the first shooting frequency is less than the second shooting frequency, and the first fusion frame number is less than the second fusion frame number.

[0009] In some implementations, the driving scene includes a driving environment; the target shooting frequency and the target fusion frame number are determined by the following steps: based on the current driving environment of the vehicle and a mapping relationship between a preset driving environment and a shooting frequency, determining the target shooting frequency corresponding to the current driving environment of the vehicle; based on the current driving environment of the vehicle and a mapping relationship between a preset driving environment and a fusion frame number, determining the target fusion frame number corresponding to the current driving environment of the vehicle.

[0010] In some implementations, the multiple frames of original images include a target part of the vehicle, and determining the actual driving speed of the vehicle based on a target image corresponding to a target fusion frame number in the multiple frames of original images includes: performing edge detection on the target part of the vehicle in the target image to obtain edge features of the target part of the vehicle; determining the actual driving speed of the vehicle based on the edge features of the target part of the vehicle through a speed recognition model; wherein the speed recognition model is trained based on the edge features of the target part of the vehicle in multiple frames of sample images and its corresponding actual driving speed.

[0011] In some implementations, controlling the vehicle based on the actual driving speed of the vehicle includes: determining a fused driving speed of the vehicle based on the actual driving speed of the vehicle; and adjusting the actual driving speed of the vehicle to approach the desired target driving speed based on a speed error between the fused driving speed of the vehicle and a desired target driving speed.

[0012] In some implementations, determining the fused driving speed of the vehicle based on the actual driving speed of the vehicle includes: determining the actual driving speed of the vehicle as the fused driving speed of the vehicle; or, fusing the real-time vehicle speed obtained by a speed sensor on the vehicle with the actual driving speed to obtain the fused driving speed.

[0013] In some implementations, the controlling of the vehicle based on the actual driving speed of the vehicle includes: determining weights for adjusting the actual driving speed, actual driving direction, and actual energy consumption based on the current driving state and current driving environment of the vehicle; determining optimized driving speed, optimized driving direction, and optimized energy consumption based on the weights corresponding to the actual driving speed, actual driving direction, and actual energy consumption, as well as the desired target driving speed, target driving direction, and target energy consumption; and controlling the vehicle based on the optimized driving speed, optimized driving direction, and optimized energy consumption.

[0014] In some implementations, the weights for adjusting the actual driving speed, actual driving direction, and actual energy consumption are determined based on the current driving state and current driving environment of the vehicle, including: determining the weights for adjusting the actual driving speed, actual driving direction, and actual energy consumption based on the current driving state and current driving environment of the vehicle through a weight model; wherein the weight model is trained based on the historical driving state and historical driving environment of the vehicle, and the real weights corresponding to the driving speed, driving direction, and energy consumption of the vehicle.

[0015] The second aspect of the present application proposes a vehicle control device, comprising: an acquisition unit, used to acquire multiple frames of original images, wherein the multiple frames of original images are obtained by an image sensor on the vehicle shooting the vehicle based on a target shooting frequency, and the multiple frames of original images include a target part of the vehicle; a determination unit, used to determine the actual driving speed of the vehicle based on a target image of a target fusion frame number in the multiple frames of original images; the target shooting frequency and the target fusion frame number are adjusted accordingly according to changes in the driving scene of the vehicle; and a control unit, used to control the vehicle based on the actual driving speed of the vehicle.

[0016] A third aspect of the present application provides an electronic device, including a memory and a processor;

[0017] The memory is connected to the processor and is used to store programs;

[0018] The processor is used to implement the vehicle control method described in any one of the first aspect and the implementation methods of the first aspect by running the program in the memory.

[0019] The fourth aspect of the present application proposes a vehicle, comprising a vehicle body, a visual sensor installed on the vehicle body, and a controller, wherein the visual sensor is connected to the controller and is used to photograph the vehicle to obtain multiple frames of original images including target parts of the vehicle; the controller is configured to execute the vehicle control method as described in the first aspect and any one of the implementation methods of the first aspect.

[0020] In a fifth aspect, the present application proposes a storage medium having a computer program stored thereon. When the computer program is executed by a processor, the vehicle control method described in the first aspect and any one of the implementation methods of the first aspect is implemented.

[0021] In a sixth aspect, the present application proposes a computer program product, comprising computer program instructions, which, when executed by a processor, enable the processor to implement the vehicle control method described in the first aspect and any one of the implementation methods of the first aspect.

[0022] The vehicle control method, device, equipment, vehicle and product proposed in the present application obtain multiple frames of original images obtained by the visual sensor on the vehicle shooting the vehicle based on the target shooting frequency, and determine the actual driving speed of the vehicle based on the target image of the target fusion frame number in the multiple frames of original images; the target shooting frequency and the target fusion frame number are adjusted accordingly according to the changes in the driving scene of the vehicle; based on the actual driving speed of the vehicle, the vehicle is controlled. Since the vehicle may experience various driving scenes during driving, the actual driving speed of the vehicle is determined by using the target shooting frequency and the target fusion frame number adapted to different driving scenes, so that the target shooting frequency and the target fusion frame number can be adjusted accordingly according to the changes in the driving scene. When a lower target shooting frequency and target fusion frame number are used in some driving scenes, not only can the accurate recognition of the vehicle speed be guaranteed, but also the speed recognition efficiency can be improved because the number of original images and target images is relatively small; while in other driving scenes, a higher target shooting frequency and target fusion frame number are used. Since the number of original images and target images increases accordingly, it can provide more abundant vehicle driving condition information, thereby improving the accurate recognition of the vehicle speed and realizing precise control of the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0024] Figure 1 A schematic diagram of a vehicle control method according to an embodiment of the present application;

[0025] Figure 2 A flowchart of a specific implementation method for determining the actual driving speed of a vehicle provided in an embodiment of the present application;

[0026] Figure 3A flowchart of a specific implementation method of calculating speed based on an external reference object provided in an embodiment of the present application;

[0027] Figure 4 A flowchart of a specific implementation method of controlling a vehicle based on the actual driving speed of the vehicle provided in an embodiment of the present application;

[0028] Figure 5 A schematic diagram of the structure of a vehicle control device provided in an embodiment of the present application;

[0029] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The technical solution proposed in the embodiment of the present application is applicable to vehicle control application scenarios of autonomous driving vehicles. The adoption of the technical solution in the embodiment of the present application can improve the recognition accuracy of vehicle speed, thereby improving the control accuracy of the autonomous driving vehicle and enhancing the driving safety of the autonomous driving vehicle.

[0031] With the continuous evolution of autonomous driving technology, real-time and accurate vehicle status monitoring has become a core element to ensure driving safety and improve driving experience, especially the accurate identification of vehicle speed. Image recognition technology has become an important tool for speed identification in the field of autonomous driving due to its flexibility and accuracy. This technology uses the on-board camera to capture images of the vehicle in front and the surrounding environment in real time, and then analyzes the dynamic changes of objects in the image sequence through advanced image processing and computer vision algorithms to infer the actual driving speed of the vehicle.

[0032] In the application process of traditional image recognition technology, the shooting frequency and the number of image sequence frames are fixed. However, in different driving scenarios, such as urban congestion, highway driving, complex weather conditions, etc., the fixed shooting frequency and frame number settings may not be able to take into account the accuracy and efficiency of speed recognition. For example, in complex driving scenarios such as rugged mountain roads, traffic lights, fast lane changes, sharp turns, and sudden braking, a higher shooting frequency can capture more details and help improve the accuracy of speed recognition, but on flat roads or when waiting for parking, too high a shooting frequency will lead to unnecessary data redundancy, increase the processing burden, and reduce system efficiency.

[0033] In view of this, the embodiments of the present application provide a vehicle control method, device, equipment, vehicle and product, which are intended to dynamically adjust the shooting frequency of the vehicle-mounted camera and the number of frames used for image processing according to different driving scenarios. When the vehicle is in a complex driving scenario, the shooting frequency will be automatically increased to ensure sufficient image data for accurate speed identification; in simple driving scenarios such as low-speed driving or stationary waiting, the shooting frequency will be appropriately reduced to reduce the generation of useless data, thereby saving computing resources and improving overall efficiency. Through this mechanism of flexibly adjusting the shooting frequency and the number of frames used for image processing according to the complexity of the vehicle driving scene, not only can the accuracy of speed recognition be guaranteed, but also the resource utilization can be effectively optimized and the speed recognition efficiency can be improved.

[0034] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0035] The present application embodiment first proposes a vehicle control method, see Figure 1 As shown, the method includes:

[0036] S101, acquiring multiple frames of original images, where the multiple frames of original images are obtained by a visual sensor on a vehicle shooting the vehicle based on a target shooting frequency.

[0037] Various sensors are installed on the autonomous driving vehicle. Among the various sensors are visual sensors. The visual sensor may be a vehicle-mounted camera. The visual sensor corresponds to a target shooting frequency. The visual sensor may be installed on the vehicle at a position where the target part of the vehicle can be photographed, so that during the driving process of the vehicle, the target part of the vehicle is imaged according to the target shooting frequency, and multiple frames of initial images containing the target part of the vehicle are obtained. Among them, the target part of the vehicle may be a wheel.

[0038] After obtaining multiple frames of initial images, they need to be preprocessed to obtain multiple frames of original images. The preprocessing includes: at least one of denoising, grayscale processing and histogram equalization.

[0039] Denoising includes median filtering, mean filtering, Gaussian filtering, etc. Denoising the initial image can effectively remove the noise in the initial image, thereby improving the processing effect of subsequent processing steps (such as edge detection and corner detection).

[0040] Grayscale processing includes: weighted average method, maximum method and minimum method, etc. By converting the original color image into a grayscale image, the image can be made more concise and clear, and the subsequent edge detection effect can be improved.

[0041] Histogram equalization can be performed after grayscale processing to make the grayscale distribution of the image more uniform, making the image contrast higher. It can effectively enhance the details of the image and make the image clearer.

[0042] By preprocessing multiple frames of initial images using methods such as denoising, grayscale processing and histogram equalization, noise can be eliminated and image quality can be improved, and the accuracy and reliability of subsequent edge detection can be improved.

[0043] It is worth noting that when selecting a preprocessing method, it is necessary to consider the preprocessing effect and operation speed of each preprocessing method and find a suitable preprocessing method to achieve a balance between the preprocessing effect and the operation speed to achieve the best preprocessing effect.

[0044] In some embodiments, in order to reduce the pressure of data transmission and storage, a smart camera with edge computing capabilities can also be selected. After the smart camera pre-processes multiple frames of initial images, the pre-processed multiple frames of original images are transmitted to the cloud or other computing platforms for subsequent processing.

[0045] During the driving process of the vehicle, the target shooting frequency of the visual sensor is not fixed. The target shooting frequency can be adjusted accordingly according to the changes in the driving scene of the vehicle. That is, the target shooting frequency can be dynamically adjusted according to the different driving scenes in which the vehicle is located. The driving scene of the vehicle may include the driving state. The driving state includes a stable driving state and an unstable driving state. The stable driving state refers to the vehicle driving on a straight road, with low traffic volume and good road conditions; the unstable driving state includes sharp turns, sudden braking, acceleration exceeding the preset acceleration or deceleration exceeding the preset deceleration, bumpy roads, traffic congestion, rapid lane changes, encountering obstacles, traffic lights, and traffic-intensive areas. Among them, the stable driving state can be regarded as a simple driving scene, and the unstable driving state can be regarded as a complex driving scene.

[0046] The stable driving state and the unstable driving state can be judged based on various sensor data (such as laser radar data, millimeter wave radar data, acceleration sensor, visual sensor, angle sensor installed on the steering wheel, etc.). For example, if the change of the vehicle acceleration collected by the accelerometer is continuously within the preset range, and the vehicle surrounding environment collected by the laser radar data shows that the vehicle is driving on a straight road, with low traffic volume and good road conditions, then the vehicle is considered to be in a stable driving state; and if the vehicle surrounding environment collected by the laser radar, the current acceleration of the vehicle collected by the accelerometer and the steering wheel angle detected by the angle sensor detect that the vehicle is currently in a sharp turn, sudden braking, acceleration exceeding the preset acceleration or deceleration exceeding the preset deceleration, bumpy road, traffic congestion, rapid lane change, encountering obstacles, traffic lights, and traffic-intensive areas, then the vehicle is considered to be in an unstable driving state.

[0047] There are many ways to determine the target shooting frequency according to the driving scene of the vehicle, which can be as follows:

[0048] In some embodiments, determining the target shooting frequency according to the driving scene of the vehicle includes: determining the target shooting frequency according to the driving state of the vehicle. Specifically, when the driving state of the vehicle is a stable driving state, the target shooting frequency can be set to a first shooting frequency; and when the driving state of the vehicle is an unstable driving state, the target shooting frequency can be set to a second shooting frequency. The first shooting frequency is less than the second shooting frequency.

[0049] Since the vehicle moves relatively smoothly in a stable driving state, a lower first shooting frequency can be set, which can meet monitoring needs and save computing resources and storage space. On the contrary, in an unstable driving state, the vehicle may experience more complex dynamic changes. At this time, setting a higher second shooting frequency can ensure that emergencies that may affect driving safety are captured in a timely manner. This mechanism of dynamically adjusting the shooting frequency based on the driving scenario can effectively balance performance requirements and resource consumption, and improve overall operating efficiency and safety.

[0050] In some embodiments, determining the target shooting frequency according to the driving scene of the vehicle includes: dynamically adjusting the target shooting frequency according to the driving environment of the vehicle. Specifically, based on the current driving environment of the vehicle and a preset mapping relationship between the driving environment and the shooting frequency, the target shooting frequency corresponding to the current driving environment of the vehicle is determined.

[0051] In specific operations, a mapping table between driving environment and shooting frequency needs to be established in advance. This mapping table is based on the analysis of different driving environment characteristics, such as speed, traffic density, road type (such as urban roads, highways, rural roads, rugged sections, flat sections), weather conditions (clear, rainy, foggy), etc., and the impact of these characteristics on the demand for visual data collection frequency.

[0052] For example, at high speeds, high traffic density, or inclement weather conditions, a higher shooting frequency may be required to ensure that the vehicle can respond quickly to environmental changes.

[0053] When the vehicle is driving in different environments, the on-board system will monitor the current driving environment characteristics in real time and compare these characteristics with the preset mapping table to determine the target shooting frequency that best suits the current environment.

[0054] S102: Determine the actual driving speed of the vehicle based on the target image of the target fused frame number in the multiple frames of original images.

[0055] In this embodiment, the target shooting frequency and the target fusion frame number can be determined according to the driving scene of the vehicle. The target shooting frequency is used to control the image acquisition frequency of the vehicle by the visual sensor on the vehicle.

[0056] The specific implementation method of determining the target shooting frequency according to the driving scene of the vehicle can be specifically referred to the introduction of the relevant content in step S101, which will not be repeated here.

[0057] The target fusion frame number refers to the number of original images selected from multiple frames of original images for calculating the actual driving speed. The specific implementation method of determining the target fusion frame number according to the vehicle's driving scenario is as follows:

[0058] In some embodiments, determining the target fusion frame number according to the driving scene of the vehicle includes: determining the target fusion frame number according to the driving state of the vehicle. Specifically, when the driving state of the vehicle is a stable driving state, the target fusion frame number can be set to a first fusion frame number; and when the driving state of the vehicle is an unstable driving state, the target fusion frame number can be set to a second fusion frame number. The first fusion frame number is less than the second fusion frame number.

[0059] The setting of the target fusion frame number is closely related to the driving state of the vehicle. Specifically, when the vehicle is in a stable driving state, that is, when the vehicle is driving on a straight road, the traffic flow is low, and the road condition is good, the target fusion frame number can be set to a lower first fusion frame number. This is because in such scenarios, the vehicle speed is relatively stable, and too dense data points are not required to calculate the vehicle speed, thereby avoiding unnecessary calculation load.

[0060] On the contrary, when the vehicle is in an unstable driving state, such as frequent lane changes, frequent acceleration, frequent deceleration, sharp turns, complex traffic environments or severe weather conditions, the target fusion frame number can be set to a higher second fusion frame number. Obviously, the second fusion frame number is greater than the first fusion frame number. This is because in this case, the speed and driving state of the vehicle change greatly. By increasing the fusion frame number, the speed change details can be captured more accurately, improving the speed calculation accuracy in an unstable driving state.

[0061] In some embodiments, the target fusion frame number is determined according to the driving scene of the vehicle, including: based on the current driving environment of the vehicle and the mapping relationship between the preset driving environment and the fusion frame number, the target fusion frame number corresponding to the current driving environment of the vehicle is determined. This method comprehensively considers a variety of environmental variables, including but not limited to road type, weather conditions, traffic flow, etc., to ensure that the autonomous driving system can flexibly adjust and select the best target fusion frame number when facing various driving conditions, thereby optimizing the efficiency of system resource utilization while ensuring the accuracy of speed calculation, avoiding unnecessary data redundancy in a stable driving environment, and capturing more subtle speed fluctuations by increasing the fusion frame number in complex or rapidly changing scenes.

[0062] After determining the target fusion frame number, the original image of the target fusion frame number can be selected from multiple frames of original images as the target image, and then the actual driving speed is calculated based on the target image of the target fusion frame number to achieve precise control of the vehicle.

[0063] In some embodiments, the actual driving speed of the vehicle is determined based on a target image of a target fused frame number in multiple frames of original images, including: performing edge detection on a target part of the vehicle in the target image to obtain edge features of the target part of the vehicle; and determining the actual driving speed of the vehicle based on the edge features of the target part of the vehicle through a speed recognition model.

[0064] In this embodiment, in order to accurately calculate the actual driving speed of the vehicle, it is necessary to detect the edge features of the target part of the vehicle in the target image before determining the actual driving speed of the vehicle through the speed recognition model based on the edge features of the target part of the vehicle in the target image.

[0065] For example, if the target part of the vehicle is a wheel and the position change is an angle change, the wheel can be detected by edge detection algorithm or edge detection model to extract the edge features of the wheel with clear contour from the target image. Specifically, the following edge detection algorithms can be used: Canny algorithm, Sobel algorithm, Prewitt algorithm, Laplacian algorithm, etc.

[0066] In addition to edge detection algorithms, edge detection models based on deep learning can also be used. This edge detection model can be trained based on a large number of sample images and manually labeled real edge features, and can learn more complex and delicate edge feature expressions. Edge detection models, such as convolutional neural networks, have shown good detection performance in edge detection tasks due to their powerful feature extraction capabilities and adaptive learning mechanisms, especially when processing images with complex backgrounds and lighting changes, and can more accurately capture the details of the wheel edge.

[0067] After the wheel edge is detected by the above algorithm or model, the wheel edge features can be obtained, including but not limited to the edge position, shape and strength information. By accurately analyzing the changes in wheel edge features, the angle change of wheel rotation can be accurately calculated and finally converted into the vehicle's driving speed.

[0068] In this embodiment, the speed recognition model is trained based on edge features of the target part of the vehicle in multiple frames of sample images and its corresponding actual driving speed.

[0069] Figure 2 The following is a flow chart of a specific implementation method for determining the actual driving speed of a vehicle provided in an embodiment of the present application. Figure 2 As shown, the actual driving speed of the vehicle can be obtained by the following steps a1 to a6:

[0070] a1. Edge feature extraction

[0071] First, edge detection algorithms or edge detection models are applied to P frame sample images to extract edge features of P vehicle target parts. These edge features contain the shape, position and intensity information of the wheel profile, providing a basis for subsequent corner point detection and displacement analysis.

[0072] a2. Corner detection

[0073] Next, the corner detection algorithm is used to detect corners in the extracted edge features. Corner points are points in the image that are obvious and easy to track. Common corner detection algorithms include Kitchen-Rosenfeld, Harris, KLT (Kanade-Lucas-Tomasi) and SUSAN (Smallest Univalue Segment Assimilating Nucleus). The selection of corner points is crucial for subsequent displacement detection because they can provide stable reference points in the image sequence.

[0074] a3. Calculation of displacement and angle change

[0075] The displacement of the detected corner point between each two adjacent frames in the P frame sample image is measured, and combined with the wheel radius, the P-1 angle changes of the corner point in the P frame sample image are calculated. These angle changes reflect the rotation of the wheel in the image sequence and are the basis for calculating the wheel rotation speed.

[0076] a4. Accumulated angle change

[0077] The P-1 angle changes obtained above are accumulated to obtain the total angle change Δθ corresponding to the P frame sample image. This total reflects the rotation speed of the wheel from the first frame to the last frame.

[0078] a5. Calculate the rotation speed

[0079] The average rotation speed ω of the wheel can be obtained by dividing the total angle change Δθ by the total time Δt between the first and last frames of the P frame sample image. The average rotation speed is a measure of the rotation angle of the wheel per unit time.

[0080] Here, the rotation speed of the wheel ω=Δθ / Δt.

[0081] a6. Convert to vehicle speed

[0082] Finally, the actual driving speed v of the vehicle is obtained based on the product of the wheel rotation speed ω and the wheel radius r.

[0083] Since there is a conversion relationship between the rotation speed of the wheel and the vehicle speed, that is, the vehicle speed can be obtained by multiplying the rotation speed of the wheel by the wheel radius. Therefore, by multiplying the rotation speed of the wheel ω by the wheel radius r, the actual driving speed of the vehicle v = ω*r can be obtained.

[0084] In addition to calculating the actual speed of the vehicle through wheel edge features, in some embodiments, the actual speed of the vehicle can also be calculated based on changes in external reference objects around the vehicle.

[0085] Figure 3 This is a flow chart of a specific implementation method of calculating speed based on an external reference object provided in an embodiment of the present application. Figure 3 As shown, the speed calculation process based on the external reference object includes steps b1 to b6:

[0086] b1. Image acquisition

[0087] Continuous camera shooting: Use the camera installed on the vehicle to continuously capture multiple frames of initial images of the road ahead. These initial images contain fixed reference objects such as road markings, road signs and buildings.

[0088] b2. Preprocessing

[0089] The collected multiple frames of initial images are preprocessed to remove noise, adjust exposure and contrast to ensure image quality.

[0090] b3. Image Registration

[0091] Image registration technology is used to align multiple frames of original images to ensure the consistency of the same reference objects between different frames and facilitate feature comparison.

[0092] b4. Feature extraction

[0093] From the aligned multiple frames of original images, the features of external reference objects, such as the shape and position of road markings or the outline of buildings, are extracted.

[0094] b5. Motion analysis

[0095] Based on the extracted features, the position changes of the external reference objects in consecutive image frames are analyzed, which directly reflects the movement of the vehicle relative to the fixed reference objects.

[0096] b6. Speed ​​calculation

[0097] By calculating the position change of the external reference object between consecutive image frames, the actual driving speed of the vehicle can be obtained.

[0098] In some embodiments, after obtaining P-1 angular changes of the corner point in the P frame sample image through the above step a3, the time difference between each two consecutive target frames in the P frame target image can be further determined, and based on the angular change and the corresponding time difference between each two consecutive target frames, the instantaneous vehicle speed corresponding to each two consecutive target frames is obtained; then, the instantaneous vehicle speed corresponding to each two consecutive target frames is weighted averaged to obtain a more accurate true driving speed of the vehicle.

[0099] After obtaining the actual driving speed of the vehicle through steps a1 to a6, the speed recognition model can be trained based on the actual driving speed so that it can predict the actual driving speed of the vehicle based on the image data. Specifically, the following steps c1 to c3 are included:

[0100] c1. Predict vehicle speed

[0101] The collected multiple frames of sample images are input into the speed recognition model to be trained to obtain the predicted vehicle speed. The speed recognition model to be trained can be a recurrent neural network model such as a long short-term memory network (LSTM). LSTM is a special recurrent neural network that can effectively manage long-term dependencies, that is, it can remember information in a long time series and pass it to subsequent time steps. This feature can help the model capture the dynamic changes and correlations between image frames when processing data with time series characteristics, such as video frame sequences, so as to more accurately predict vehicle speed.

[0102] c2. Determine speed prediction loss

[0103] Based on the difference between the predicted vehicle speed output by the model and the actual driving speed calculated through steps a1 to a6, a speed prediction loss can be determined, wherein the speed prediction loss can be determined using a cross entropy loss function.

[0104] c3. Model training and convergence

[0105] Using the speed prediction loss calculated in step c2, the model parameters can be adjusted through the back propagation algorithm to gradually approach the optimal solution, that is, minimize the prediction error. This process is repeated until the preset convergence conditions are met, including but not limited to the speed prediction loss being less than the preset loss, the number of training iterations reaching the preset number, and the model accuracy reaching the preset accuracy.

[0106] After completing steps b1 to b3, a trained speed recognition model can be obtained. Afterwards, when facing new N frames of target images, it is only necessary to input the edge features of the vehicle target part in the N frames of target images into the speed recognition model to quickly and accurately obtain the actual driving speed of the vehicle.

[0107] Continue reading Figure 1 After step S102, the vehicle control method of this embodiment may further include step S103.

[0108] S103: Control the vehicle based on the actual driving speed of the vehicle.

[0109] After the actual driving speed of the vehicle is determined through the above steps S101 and S102, the vehicle can be controlled based on the actual driving speed of the vehicle. The control of the vehicle based on the actual driving speed of the vehicle includes: adjusting the actual driving speed of the vehicle to approach the expected target driving speed based on the speed error between the actual driving speed of the vehicle and the expected target driving speed.

[0110] This embodiment adjusts the vehicle's driving state by comparing the vehicle's actual driving speed with the desired target driving speed set by the automatic driving system to ensure that the actual driving speed is consistent with the target driving speed or within an acceptable error range. Specifically, this closed-loop control mechanism dynamically adjusts the vehicle's acceleration and deceleration based on the speed error, that is, the difference between the actual driving speed and the target driving speed, so that the vehicle's actual driving speed gradually approaches the target driving speed, achieving smooth and safe driving.

[0111] However, it is worth noting that although the speed recognition model based on image analysis can provide accurate speed information in most cases, due to factors such as environmental changes and model errors, cumulative errors in speed recognition may still occur after long-term operation. If this cumulative error is not corrected, it will directly affect the vehicle control accuracy. Therefore, in order to further improve the accuracy of speed recognition and the stability of vehicle control, this embodiment can also fuse the actual driving speed with the real-time vehicle speed measured by a speed sensor (such as an accelerometer) installed on the vehicle after controlling the vehicle for a period of time based on the actual driving speed to obtain a fused driving speed; and based on the speed error between the fused driving speed of the vehicle and the desired target driving speed, adjust the fused driving speed of the vehicle to approach the desired target driving speed.

[0112] As a type of speed sensor, the function of the accelerometer is to monitor the acceleration changes and motion state of the vehicle. The accelerometer can not only capture the instantaneous acceleration and deceleration information of the vehicle, but also indirectly reflect the driving speed of the vehicle. Combining the real-time driving speed of the accelerometer with the actual driving speed identified by the speed recognition model can effectively correct the potential errors caused by image recognition and improve the accuracy of speed recognition. The fused driving speed will serve as the basis for subsequent vehicle control to ensure that the driving state of the vehicle is more in line with the set target, and even in complex and changeable driving environments, it can maintain a high degree of driving accuracy and safety.

[0113] In summary, by combining image recognition technology and real-time sensor data, the autonomous driving system can achieve high-precision measurement and control of the vehicle's driving speed, which not only improves the accuracy and reliability of speed recognition, but also improves the accuracy of vehicle control and improves the safety of vehicle driving.

[0114] Among them, the specific implementation methods of fusing the real-time vehicle speed obtained by the speed sensor on the vehicle with the actual driving speed to obtain the fused driving speed may include multiple: for example, the real-time vehicle speed obtained by the speed sensor on the vehicle is weighted summed with the actual driving speed to obtain the fused driving speed; or the real-time vehicle speed and the actual driving speed are fused through algorithms such as Kalman filtering or extended Kalman filtering to obtain the fused driving speed.

[0115] Among them, when the real-time vehicle speed obtained by the speed sensor on the vehicle is weighted and summed with the actual driving speed, the selection of weights can be adjusted based on the trust or historical performance of the two data sources. For example, if the real-time driving speed comes from a high-precision speed sensor, the real-time driving speed can be given a higher weight; conversely, if the speed recognition model performs well within the preset time, the weight of the actual driving speed can be increased accordingly. For example, in the initial stage, the real-time vehicle speed and the actual driving speed can be set with corresponding weights based on manual experience; then, based on the degree of proximity between the actual driving speed recognized by the speed recognition model and the set target driving speed, the recognition accuracy of the speed recognition model in each recognition process within the preset time is determined; if the recognition accuracy of the speed recognition model in each recognition process is greater than the preset recognition accuracy for more than the preset number of times, the weight of the actual driving speed is increased; otherwise, the weight set initially is kept unchanged.

[0116] In addition, Kalman filtering is a probabilistic statistical method for estimating the state of a dynamic system, and is suitable for processing noisy measurement data. In the autonomous driving scenario, it can fuse the real-time driving speed and the actual driving speed, and gradually correct the speed estimation value through two stages of prediction and update, and finally obtain the fused driving speed, thereby achieving the stability and accuracy of speed estimation in a complex and changeable driving environment.

[0117] The fused driving speed obtained by the above method not only overcomes the limitations of a single data source, but also avoids the error accumulation that may occur in the speed recognition model during long-term operation. Based on this more accurate fused driving speed, the autonomous driving system can more finely adjust the vehicle's driving state, ensuring that the vehicle can maintain near the target driving speed under various driving conditions, improving overall driving safety and comfort.

[0118] To sum up, step S103 controls the vehicle based on the actual driving speed of the vehicle, which can be summarized as including the following steps: determining the vehicle's fusion driving speed based on the vehicle's actual driving speed; adjusting the vehicle's actual driving speed to approach the desired target driving speed based on the speed error between the vehicle's fusion driving speed and the desired target driving speed.

[0119] Wherein, determining the fused driving speed of the vehicle based on the actual driving speed of the vehicle includes: directly determining the actual driving speed of the vehicle as the fused driving speed of the vehicle; or fusing the real-time vehicle speed obtained by a speed sensor on the vehicle with the actual driving speed to obtain the fused driving speed.

[0120] In this embodiment, the current and voltage of the motor on the vehicle are adjusted according to the fused driving speed of the vehicle to control the actual driving speed of the vehicle to approach the desired target driving speed. Specifically, the precise control of the motor output can be achieved through a control algorithm, such as proportional integral control (PID). The PID control algorithm analyzes the speed error between the target driving speed and the fused driving speed, and then adjusts the output current and output voltage of the motor to achieve precise control of the vehicle speed and smoothly transition the actual driving speed of the vehicle to the desired target driving speed.

[0121] Specifically, the main principle of the PID control algorithm is to analyze, calculate and feedback the speed error, and then adjust the output signal to achieve the desired control target. When implementing PID control, the output current and voltage of the motor need to be controlled to adjust the speed of the vehicle.

[0122] The PID control algorithms in this embodiment include: an adaptive PID control algorithm, a nonlinear PID control algorithm, a robust PID control algorithm, a non-traditional PID control algorithm and a multivariable PID control algorithm.

[0123] Among them, the adaptive PID control algorithm automatically adjusts the PID control parameters by real-time monitoring and analysis of the vehicle's dynamic response, so that it can adapt to different working conditions and environments.

[0124] Nonlinear PID control algorithms, such as fuzzy PID control or neural network PID control, can better handle the nonlinear dynamic behavior of the vehicle.

[0125] The robust PID control algorithm can improve the robustness of the control system and enhance the control system's resistance to interference from uncertain factors such as friction, uneven road surface or wind.

[0126] Non-traditional control algorithms, such as model predictive control (MPC) or fuzzy control, can better handle the nonlinear and time-varying characteristics of vehicle speed control, as well as multi-objective optimization problems.

[0127] Multivariable PID control can expand PID control to multivariable PID control, taking into account multiple degrees of freedom, including multiple control objectives such as speed and direction, and achieving multi-objective optimization through appropriate adjustment strategies.

[0128] The above embodiment introduces a technical solution for controlling a vehicle based on the actual driving speed of the vehicle. However, in actual vehicle control, it also involves adjusting multiple target parameters such as the vehicle's driving direction and energy consumption. Therefore, this embodiment also proposes a multi-objective optimization solution to find a balance point between multiple target parameters and coordinately control multiple target parameters to achieve the best overall control performance. The following is a detailed introduction to multi-objective optimization control in conjunction with the accompanying drawings:

[0129] Figure 4 FIG. 1 is a flow chart of another implementation method of controlling a vehicle based on the actual driving speed of the vehicle provided in an embodiment of the present application. Figure 4 As shown, based on the actual driving speed of the vehicle, the vehicle is controlled, including the following steps S401, S402 and S403:

[0130] S401. Determine weights for adjusting actual driving speed, actual driving direction, and actual energy consumption based on the current driving state and current driving environment of the vehicle.

[0131] Before step S401, the actual driving direction and actual energy consumption of the vehicle during the current driving process need to be obtained. The actual driving direction can be obtained by collecting angles based on the angle sensor installed on the steering wheel of the vehicle; the actual energy consumption can be calculated based on the actual mileage of the vehicle and the energy consumption per 100 kilometers, that is, the actual energy consumption = actual mileage / energy consumption per 100 kilometers.

[0132] In some embodiments, the weights corresponding to the actual driving speed, the actual driving direction, and the actual energy consumption can be determined based on a weight model. Specifically, the weight model is used to determine the weights for adjusting the actual driving speed, the actual driving direction, and the actual energy consumption based on the current driving state and the current driving environment of the vehicle. The weight model is obtained by training based on the historical driving state and the historical driving environment of the vehicle, as well as the real weights corresponding to the driving speed, driving direction, and energy consumption of the vehicle.

[0133] Among them, the actual weights corresponding to the vehicle's driving speed, driving direction and energy consumption can be determined by manual labeling.

[0134] For example, on urban roads, due to heavy traffic and frequent stops, emphasis should be placed on low energy consumption and smooth driving. At this time, the weight corresponding to energy consumption should be greater than the weights of the other two.

[0135] On highways, since vehicles travel at high speeds, emphasis should be placed on speed and direction control, and the weights corresponding to speed and direction should be greater than the weight corresponding to energy consumption.

[0136] For example, on mountain roads or rugged sections, speed control and safety performance should be balanced, and the weights of speed variables and direction variables should be increased.

[0137] In some embodiments, the weights corresponding to the actual driving speed, actual driving direction and actual energy consumption can also be determined based on the mapping relationship between the vehicle's driving state, driving environment and the weights corresponding to the actual driving speed, actual driving direction and actual energy consumption.

[0138] S402: Determine the optimized driving speed, optimized driving direction and optimized energy consumption based on the weights corresponding to the actual driving speed, the actual driving direction and the actual energy consumption, as well as the expected target driving speed, the target driving direction and the target energy consumption.

[0139] There are multiple implementations of step S402, which are as follows:

[0140] In some embodiments, step S402 specifically includes: based on the weights corresponding to the actual driving speed, the actual driving direction and the actual energy consumption, weighted summing the speed variable, the driving direction variable and the energy consumption variable to be solved to obtain the objective function; and solving the optimal solution of the objective function by an optimization algorithm, the optimal solution includes optimizing the driving speed, optimizing the driving direction and optimizing the energy consumption. The objective function can be expressed as:

[0141] f(x)=w1·v+w2·d+w3·e;

[0142] Where f(x) is the objective function; v is the driving speed variable; d is the driving direction variable; e is the energy consumption variable; w1, w2 and w3 are their corresponding weights respectively.

[0143] Among them, optimization algorithms may include gradient descent, genetic algorithm, etc.

[0144] The optimal solution of the objective function is solved by the optimization algorithm, that is, to find a set (v*, d*, e*) so that f(x) reaches the minimum or maximum. Among them, v*, d*, e* represent the optimized driving speed, optimized driving direction and optimized energy consumption after optimization.

[0145] In some embodiments, step S402 specifically includes: multiplying the weights corresponding to the actual driving speed, the actual driving direction and the actual energy consumption by the actual driving speed, the actual driving direction and the actual energy consumption respectively to obtain the adjusted driving speed, the adjusted driving direction and the adjusted energy consumption.

[0146] For example, the weight corresponding to the actual driving speed is multiplied by the actual driving speed to obtain the adjusted driving speed; the weight corresponding to the actual driving direction is multiplied by the actual driving direction to obtain the adjusted driving direction; the weight corresponding to the actual energy consumption is multiplied by the actual energy consumption to obtain the adjusted energy consumption. This method regards the weight as an adjustment factor and directly acts on each target parameter, so that the adjusted parameters can reflect the relative importance of each target parameter in the control strategy.

[0147] S403: Control the vehicle based on optimizing the driving speed, optimizing the driving direction, and optimizing the energy consumption.

[0148] Specifically, the vehicle is controlled based on optimizing the driving speed, optimizing the driving direction and optimizing the energy consumption, including:

[0149] Based on the speed error between the fusion driving speed and the optimized driving speed of the vehicle, the fusion driving speed of the vehicle is gradually approached to the adjusted driving speed by adjusting the throttle or brake force;

[0150] Based on the direction error between the actual driving direction of the vehicle and the optimized driving direction, the steering angle of the vehicle is adjusted through path planning and the actual driving direction so that the vehicle travels in the optimized driving direction.

[0151] Based on the energy consumption error between the actual energy consumption and the optimized energy consumption of the vehicle, the acceleration, deceleration and braking are adjusted so that the actual energy consumption of the vehicle gradually approaches the optimized energy consumption to achieve energy-saving effects.

[0152] In some embodiments, the future driving path and speed can be predicted by integrating the historical driving data and current vehicle status, and more refined vehicle status management and control can be performed based on the prediction results. Specifically,

[0153] (1) Data collection and analysis

[0154] First, collect the vehicle's historical driving data, including but not limited to driving speed, driving direction, energy consumption, road type, weather conditions, etc. As well as the current vehicle status, covering the vehicle's instant speed, direction, energy consumption level, environmental perception data, etc. These data are the basis for predicting future paths and speeds.

[0155] (2) Driving behavior recognition

[0156] Through the trained driving behavior recognition model, historical driving behavior and current driving behavior can be identified based on various sensor data and camera data. This model is a machine learning model trained based on a large amount of manually annotated driving behavior data, which can accurately identify and classify different driving patterns and habits.

[0157] (3) Behavior prediction model

[0158] Based on historical driving data and current vehicle status, the behavior prediction model is used to predict the vehicle's next possible behavior, including acceleration, deceleration, lane change, parking, etc. This prediction is based on the analysis of the vehicle's past driving behavior and real-time evaluation of the current environment and vehicle status.

[0159] Based on the above prediction results, the driving environment and state of the vehicle in the future can be further determined, including the road conditions, traffic conditions, weather changes, etc. that may be encountered. Based on this prediction information, the weights of speed variables, driving direction variables, and energy consumption variables in the future driving process can be calculated in advance to guide the vehicle's optimal control strategy.

[0160] By weighting the speed variables, driving direction variables and energy consumption variables in the future driving process, the vehicle speed, driving direction and energy consumption can be pre-optimized to ensure that the vehicle can operate in the optimal state in the future driving process, avoid potential risks, improve energy efficiency, and ensure driving comfort and safety.

[0161] In summary, by combining historical driving data, current vehicle status, driving behavior recognition and prediction models, predictive control can be achieved, which not only improves the intelligence level of the vehicle, but also enhances driving safety and efficiency.

[0162] The above scheme can be summarized as follows: based on the vehicle's historical driving data and current vehicle status, predict the vehicle's future path and future speed; based on the predicted future path and future speed, determine the vehicle's future driving environment and future vehicle status; and based on the vehicle's future driving environment and future vehicle status, determine the weights of speed variables, driving direction variables, and energy consumption variables in the future driving process; and based on the weights of speed variables, driving direction variables, and energy consumption variables in the future driving process, optimize the vehicle speed, driving direction, and energy consumption in the future driving process, thereby achieving predictive control.

[0163] Among them, the behavior prediction model can be used to predict the vehicle's next driving behavior based on historical driving data and the current vehicle state to obtain the predicted driving behavior. Historical driving data includes historical driving behavior; the current vehicle state includes current driving behavior; both historical driving behavior and current driving behavior can be identified by a trained driving behavior recognition model based on various sensor data and camera data; the driving behavior recognition model is a machine learning model trained based on various sensor data and camera data, as well as manually labeled driving behavior data.

[0164] The predictive control of this embodiment can adopt a combination of model predictive control (MPC) and multi-objective optimization (MOO), perform multi-objective optimization under the predicted path and speed requirements, and adjust vehicle control parameters in advance; and continuously update the prediction model and optimization objectives based on real-time sensor data and vehicle status to ensure the real-time and accuracy of the control effect.

[0165] In addition, by integrating the energy management system (EMS) with the MOO algorithm and adding energy consumption optimization to the MOO algorithm, it is possible to optimize parameters such as motor power and energy recovery while optimizing speed and direction control, thereby maximizing energy utilization efficiency. During energy recovery, the vehicle speed and direction can be adjusted to achieve optimal energy recovery effects and reduce energy waste.

[0166] In some embodiments, the optimization results under different driving environments can also be recorded, and a strategy library for different driving environments can be constructed (such as rainy days, snowy days, nighttime, urban roads, and highways), so as to switch the control strategy according to the real-time perception results.

[0167] By integrating the data of LiDAR, millimeter-wave radar and camera, accurate environmental perception results are obtained, and the control strategy is dynamically adjusted according to the real-time environmental perception results to achieve adaptive optimization. For example, according to the perception results of multi-sensor fusion, the speed, direction and energy consumption control of the vehicle are adjusted in real time to ensure the best performance in various environments. For example, when the road surface is detected to be slippery on rainy days, the system reduces the speed and enhances stability; different strategy libraries are triggered according to the environmental perception results to achieve rapid switching of control strategies.

[0168] Corresponding to the above-mentioned model training method, the present application embodiment also proposes a vehicle control device, such as Figure 5 As shown, the device includes: an acquisition unit 501, used to acquire multiple frames of original images, where the multiple frames of original images are obtained by a visual sensor on the vehicle shooting the vehicle based on a target shooting frequency; a determination unit 502, used to determine the actual driving speed of the vehicle based on a target image of a target fusion frame number in the multiple frames of original images; the target shooting frequency and the target fusion frame number are adjusted accordingly according to changes in the driving scene in which the vehicle is located; and a control unit 503, used to control the vehicle based on the actual driving speed of the vehicle.

[0169] In some embodiments, the driving scene includes a driving state; the driving state includes a stable driving state and an unstable driving state; the target shooting frequency and the target fusion frame number are determined by the following steps: when the vehicle driving state is a stable driving state, the target shooting frequency is set to a first shooting frequency, and the target fusion frame number is set to a first fusion frame number; when the vehicle driving state is an unstable driving state, the target shooting frequency is set to a second shooting frequency, and the target fusion frame number is set to a second fusion frame number; wherein, the first shooting frequency is less than the second shooting frequency, and the first fusion frame number is less than the second fusion frame number.

[0170] In some embodiments, the driving scene includes a driving environment; the target shooting frequency and the target fusion frame number are determined by the following steps: based on the current driving environment of the vehicle and a mapping relationship between a preset driving environment and a shooting frequency, the target shooting frequency corresponding to the current driving environment of the vehicle is determined; based on the current driving environment of the vehicle and a mapping relationship between a preset driving environment and a fusion frame number, the target fusion frame number corresponding to the current driving environment of the vehicle is determined.

[0171] In some embodiments, the multiple frames of original images include a target part of a vehicle, and the determination unit 502 determines the actual driving speed of the vehicle based on the target image corresponding to the target fusion frame number in the multiple frames of original images, including: performing edge detection on the target part of the vehicle in the target image to obtain edge features of the target part of the vehicle; determining the actual driving speed of the vehicle based on the edge features of the target part of the vehicle through a speed recognition model; wherein the speed recognition model is trained based on the edge features of the target part of the vehicle in multiple frames of sample images and its corresponding actual driving speed.

[0172] In some embodiments, the control unit 503 controls the vehicle based on the actual driving speed of the vehicle, including: determining the fusion driving speed of the vehicle based on the actual driving speed of the vehicle; adjusting the actual driving speed of the vehicle to approach the desired target driving speed based on the speed error between the fusion driving speed of the vehicle and the desired target driving speed.

[0173] In some embodiments, the control unit 503 determines the fused driving speed of the vehicle based on the actual driving speed of the vehicle, including: determining the actual driving speed of the vehicle as the fused driving speed of the vehicle; or, fusing the real-time vehicle speed obtained by the speed sensor on the vehicle with the actual driving speed to obtain the fused driving speed.

[0174] In some embodiments, the control unit 503 controls the vehicle based on the actual driving speed of the vehicle, including: determining the weights for adjusting the actual driving speed, actual driving direction, and actual energy consumption based on the current driving state and current driving environment of the vehicle; determining the optimized driving speed, optimized driving direction, and optimized energy consumption based on the weights corresponding to the actual driving speed, actual driving direction, and actual energy consumption, as well as the expected target driving speed, target driving direction, and target energy consumption; and controlling the vehicle based on the optimized driving speed, optimized driving direction, and optimized energy consumption.

[0175] In some embodiments, the control unit 503 determines the weights for adjusting the actual driving speed, actual driving direction, and actual energy consumption based on the current driving state and current driving environment of the vehicle, including: determining the weights for adjusting the actual driving speed, actual driving direction, and actual energy consumption based on the current driving state and current driving environment of the vehicle through a weight model; wherein the weight model is trained based on the historical driving state and historical driving environment of the vehicle, as well as the real weights corresponding to the driving speed, driving direction, and energy consumption of the vehicle.

[0176] The vehicle control device provided in this embodiment belongs to the same application concept as the vehicle control method provided in the above embodiments of this application, can execute the vehicle control method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of executing the vehicle control method. For technical details not fully described in this embodiment, please refer to the specific processing content of the vehicle control method provided in the above embodiments of this application, and will not be repeated here.

[0177] The functions implemented by each unit in the above vehicle control device can be implemented by the same or different processors respectively, and the embodiments of the present application are not limited thereto.

[0178] It should be understood that the units in the above vehicle control device can be implemented in the form of a processor calling software. For example, the device includes a processor, the processor is connected to a memory, and instructions are stored in the memory. The processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of each unit of the device, wherein the processor can be a general-purpose processor, such as a CPU or a microprocessor, etc., and the memory can be a memory in the device or a memory outside the device. Alternatively, the unit in the device can be implemented in the form of a hardware circuit, and the functions of some or all units can be realized by designing the hardware circuit. The hardware circuit can be understood as one or more processors; for example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are realized by designing the logical relationship of the components in the circuit; for another example, in another implementation, the hardware circuit can be implemented by PLD, taking FPGA as an example, which can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by a configuration file, so as to realize the functions of some or all of the above units. All units of the above device can be implemented in the form of a processor calling software, or in the form of a hardware circuit, or in the form of a processor calling software, and the remaining part can be implemented in the form of a hardware circuit.

[0179] In an embodiment of the present application, a processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and run instructions, such as a CPU, a microprocessor, a GPU, or a DSP; in another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, DPU, etc.

[0180] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0181] In addition, all or part of the units in the above device can be integrated together, or can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a SOC. The SOC may include at least one processor for implementing any of the above methods or implementing the functions of each unit of the device. The type of the at least one processor may be different, for example, including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0182] An embodiment of the present application further provides a control device, which includes a processor and an interface circuit. The processor in the control device is connected to a data input component via the interface circuit of the control device.

[0183] The data input component specifically refers to a functional component capable of inputting or collecting voice data, such as a microphone, etc.

[0184] The above-mentioned interface circuit can be any interface circuit that can realize the data communication function, for example, it can be a USB interface circuit, a Type-C interface circuit, a serial port circuit, a PCIE circuit, etc.

[0185] The processor in the control device is also a circuit with signal processing capability, which executes the vehicle control method introduced in the above embodiment. The specific implementation of the processor can refer to the above processor implementation, and the embodiment of the present application is not strictly limited.

[0186] Another embodiment of the present application also provides an electronic device, see Figure 6 As shown, the device includes:

[0187] Memory 200 and processor 210;

[0188] The memory 200 is connected to the processor 210 and is used to store programs;

[0189] The processor 210 is used to implement the vehicle control method disclosed in any of the above embodiments by running the program stored in the memory 200.

[0190] Specifically, the electronic device may further include: a bus, a communication interface 220 , an input device 230 and an output device 240 .

[0191] The processor 210, the memory 200, the communication interface 220, the input device 230 and the output device 240 are connected to each other via a bus.

[0192] A bus may include a pathway that transfers information between components of a computer system.

[0193] The processor 210 may be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the scheme of the present invention. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0194] The processor 210 may include a main processor, and may also include a baseband chip, a modem, and the like.

[0195] The memory 200 stores a program for executing the technical solution of the present invention, and may also store an operating system and other key services. Specifically, the program may include a program code, and the program code includes a computer operation instruction. More specifically, the memory 200 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk storage, a flash, and the like.

[0196] The input device 230 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor.

[0197] Output device 240 may include a device that allows information to be output to a user, such as a display screen, a printer, a speaker, etc.

[0198] The communication interface 220 may include any transceiver or the like to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0199] The processor 210 executes the program stored in the memory 200 and calls other devices, which can be used to implement each step of any vehicle control method provided in the above embodiments of the present application.

[0200] An embodiment of the present application also provides a vehicle, including a vehicle body, a visual sensor installed on the vehicle body, and a controller, wherein the visual sensor is connected to the controller and is used to photograph the vehicle to obtain multiple frames of original images including target parts of the vehicle; the controller is configured to execute the vehicle control method introduced in any of the above embodiments.

[0201] An embodiment of the present application also proposes a chip, which includes a processor and a data interface. The processor reads and runs a program stored in a memory through the data interface to execute the vehicle control method introduced in any of the above embodiments. The specific processing process and its beneficial effects can be found in the introduction to the above-mentioned embodiment of the vehicle control method.

[0202] In addition to the above methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the vehicle control method described in any of the above embodiments of this specification.

[0203] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0204] In addition, an embodiment of the present application may also be a storage medium on which a computer program is stored, and the computer program is executed by a processor to execute the steps of the vehicle control method described in any of the above embodiments of this specification.

[0205] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0206] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0207] The steps in the methods of each embodiment of the present application can be adjusted in order, combined and deleted according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0208] The modules and sub-modules in the devices and terminals of the various embodiments of the present application can be combined, divided and deleted according to actual needs.

[0209] In the several embodiments provided in the present application, it should be understood that the disclosed terminals, devices and methods can be implemented in other ways. For example, the terminal embodiments described above are only schematic, for example, the division of modules or submodules is only a logical function division, and there may be other division methods in actual implementation, for example, multiple submodules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0210] The modules or submodules described as separate components may or may not be physically separated, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place, or they may be distributed on multiple network modules or submodules. Some or all of the modules or submodules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0211] In addition, each functional module or submodule in each embodiment of the present application may be integrated into one processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into one module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or in the form of software functional modules or submodules.

[0212] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0213] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly by hardware, software units executed by a processor, or a combination of the two. The software units may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0214] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0215] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A vehicle control method, characterized in that: include: Acquire multiple frames of original images, where the multiple frames of original images are obtained by a visual sensor on the vehicle photographing the vehicle based on a target photographing frequency; Based on the target image of the target fusion frame number in the multiple frames of original images, determining the actual driving speed of the vehicle; the target shooting frequency and the target fusion frame number are adjusted accordingly according to the change of the driving scene of the vehicle; The vehicle is controlled based on the actual driving speed of the vehicle.

2. The method according to claim 1, characterized in that The driving scenario includes a driving state; the driving state includes a stable driving state and an unstable driving state; The target shooting frequency and the target fusion frame number are determined by the following steps: When the vehicle driving state is a stable driving state, setting the target shooting frequency to a first shooting frequency, and setting the target fusion frame number to a first fusion frame number; When the driving state of the vehicle is an unstable driving state, setting the target shooting frequency to a second shooting frequency, and setting the target fusion frame number to a second fusion frame number; Among them, the first shooting frequency is smaller than the second shooting frequency, and the first fusion frame number is smaller than the second fusion frame number.

3. The method according to claim 1, characterized in that The driving scenario includes a driving environment; The target shooting frequency and the target fusion frame number are determined by the following steps: Determining a target shooting frequency corresponding to the current driving environment of the vehicle based on the current driving environment of the vehicle and a preset mapping relationship between the driving environment and the shooting frequency; Based on the current driving environment of the vehicle and a mapping relationship between a preset driving environment and a fusion frame number, the target fusion frame number corresponding to the current driving environment of the vehicle is determined.

4. The method according to any one of claims 1 to 3, characterized in that: The multiple frames of original images include a target part of the vehicle, and determining the actual driving speed of the vehicle based on the target image corresponding to the target fusion frame number in the multiple frames of original images includes: Performing edge detection on a target part of the vehicle in the target image to obtain edge features of the target part of the vehicle; Determining the actual driving speed of the vehicle based on the edge features of the target part of the vehicle through a speed recognition model; The speed recognition model is trained based on the edge features of the target part of the vehicle in multiple frames of sample images and its corresponding real driving speed.

5. The method according to any one of claims 1 to 3, characterized in that: The controlling of the vehicle based on the actual driving speed of the vehicle comprises: Determining a fused driving speed of the vehicle based on an actual driving speed of the vehicle; Based on a speed error between the fused driving speed of the vehicle and the desired target driving speed, the actual driving speed of the vehicle is adjusted to approach the desired target driving speed.

6. The method according to claim 5, characterized in that The determining, based on the actual driving speed of the vehicle, the fused driving speed of the vehicle comprises: Determine the actual driving speed of the vehicle as the fused driving speed of the vehicle; or, The real-time vehicle speed obtained by the speed sensor on the vehicle is fused with the actual driving speed to obtain the fused driving speed.

7. The method according to any one of claims 1 to 3, characterized in that: The controlling of the vehicle based on the actual driving speed of the vehicle comprises: Based on the current driving state and current driving environment of the vehicle, determining weights for adjusting the actual driving speed, the actual driving direction, and the actual energy consumption; Determine the optimized driving speed, the optimized driving direction and the optimized energy consumption based on the weights corresponding to the actual driving speed, the actual driving direction and the actual energy consumption, and the desired target driving speed, the target driving direction and the target energy consumption; The vehicle is controlled based on the optimized driving speed, the optimized driving direction and the optimized energy consumption.

8. The method according to claim 7, characterized in that The determining, based on the current driving state and current driving environment of the vehicle, weights for adjusting the actual driving speed, the actual driving direction, and the actual energy consumption respectively includes: Determining weights for adjusting the actual driving speed, the actual driving direction, and the actual energy consumption based on the current driving state and the current driving environment of the vehicle through a weight model; The weight model is obtained by training based on the historical driving state and historical driving environment of the vehicle, and the real weights corresponding to the driving speed, driving direction and energy consumption of the vehicle.

9. A vehicle control device, characterized in that: include: An acquisition unit, used to acquire multiple frames of original images, wherein the multiple frames of original images are obtained by an image sensor on the vehicle photographing the vehicle based on a target photographing frequency, and the multiple frames of original images include a target part of the vehicle; A determination unit, configured to determine the actual driving speed of the vehicle based on a target image of a target fusion frame number in the multiple frames of original images; the target shooting frequency and the target fusion frame number are adjusted accordingly according to changes in the driving scene of the vehicle; A control unit is used to control the vehicle based on the actual driving speed of the vehicle.

10. An electronic device, characterized in that: including memory and processor; The memory is connected to the processor and is used to store programs; The processor is configured to implement the method according to any one of claims 1 to 8 by running the program in the memory.

11. A vehicle, characterized in that: It includes a vehicle body, a visual sensor installed on the vehicle body, and a controller, wherein the visual sensor is connected to the controller and is used to photograph the vehicle to obtain multiple frames of original images including a target part of the vehicle; The controller is configured to perform the method according to any one of claims 1 to 8.

12. A computer program product, characterized in that The method comprises computer program instructions, which, when executed by a processor, cause the processor to implement the method as claimed in any one of claims 1 to 8.

Images