Data processing method and device, vehicle control system and vehicle

Abstract

The invention provides a data processing method and device, a vehicle control system and a vehicle. The method comprises the steps of obtaining to-be-processed data and operation information of target equipment; determining a target preprocessing strategy and a target algorithm model based on the operation information; preprocessing the to-be-processed data based on the target preprocessing strategy to obtain preprocessed data; carrying out computing power distribution on the target algorithm model based on the operation information, and processing the preprocessed data by utilizing the target algorithm model according to the corresponding computing power; by adopting the method, the preprocessing process and the computing power distribution process are coupled with the scene, so that the obtained preprocessed data can meet the detection requirements in different scenes, the detection precision in different scenes is improved, the model can meet the processing requirements of the preprocessed data as much as possible through computing power distribution, and the detection accuracy is improved. Therefore, on the basis of not changing the model parameters, the computing resources of the target equipment are fully utilized, and the data processing efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of automobiles, and particularly to a data processing method, apparatus, vehicle control system, and vehicle. Background Art

[0002] With the development of autonomous driving technology, intelligent vehicles are becoming increasingly intelligent. While intelligent vehicles bring convenience and driving experiences to people, the issue of vehicle safety has become more prominent. Currently, the sensor configurations and corresponding algorithm calculations of autonomous driving systems are prefabricated and customized designs for different types of vehicles, and their adaptation and debugging are carried out. Their requirements for computing power and system computing resources are basically fixed and do not change much with the actual driving scenario. However, for different driving scenarios, such as closed highways, parking lots, mountain roads, or urban traffic environments, as well as for different driving environments and weather conditions, the corresponding perception, planning, and control requirements for autonomous driving are different. Therefore, when processing sensor data, the resources of the in-vehicle system cannot be fully utilized, thereby affecting system performance. Summary of the Invention

[0003] In view of the above problems, this application provides a method that can automatically adjust the preprocessing strategy for data to be processed and allocate computing power to target algorithm models based on operating information, thereby meeting the data processing requirements in different scenarios and improving the efficiency and performance of data processing.

[0004] The embodiments of this application are implemented by adopting the following technical solutions:

[0005] In a first aspect, this application provides a data processing method, which includes: obtaining data to be processed and operating information of a target device; determining a target preprocessing strategy for the data to be processed from at least one preprocessing strategy based on the operating information, and determining at least one target algorithm model from multiple algorithm models based on the operating information; preprocessing the data to be processed based on the target preprocessing strategy to obtain preprocessed data, where the target preprocessing strategy is used to determine the data processing range of the data to be processed; allocating computing power to the at least one target algorithm model based on the operating information to obtain the computing power allocated to each target algorithm model, where the computing power is used to represent the amount of computing resources of the target device that the target algorithm model can call; and processing the preprocessed data using the target algorithm model according to the allocated computing power to obtain a processing result.

[0006] Second aspect, the present application provides a data processing device, the device comprising: an acquisition module, configured to acquire data to be processed and operation information of a target device; a selection module, configured to determine a target preprocessing strategy for the data to be processed from at least one preprocessing strategy based on the operation information, and determine at least one target algorithm model from a plurality of algorithm models based on the operation information; a preprocessing module, configured to preprocess the data to be processed based on the target preprocessing strategy to obtain preprocessed data, the target preprocessing strategy being used to determine the data processing range of the data to be processed; a computing power allocation module, configured to allocate computing power to the at least one target algorithm model based on the operation information to obtain the computing power allocated to each target algorithm model, the computing power being used to characterize the amount of computing resources of the target device that the target algorithm model can call; a data processing module, configured to process the preprocessed data using the target algorithm model according to the allocated computing power thereof to obtain a processing result.

[0007] In some embodiments, the target device is a vehicle, the operation information includes an operation speed and an operation direction, and the selection module is further configured to determine a driving scenario of the vehicle based on the operation speed and the operation direction of the vehicle; obtain a target preprocessing strategy corresponding to the driving scenario from a corresponding relationship between preset driving scenarios and preprocessing strategies, wherein the corresponding relationship includes a plurality of driving scenarios and preprocessing strategies corresponding to each driving scenario.

[0008] In some embodiments, the preprocessing module is configured to, if the driving scenario of the vehicle is a normal driving scenario, obtain data within a first preset distance range from the vehicle and within a precision range within a first preset precision threshold as preprocessed data, the normal driving scenario being a scenario where the vehicle is driving at a speed greater than a first preset vehicle speed in the forward direction.

[0009] In some embodiments, the preprocessing module is further configured to, if the driving scenario of the vehicle is a low-speed driving scenario, obtain data within a second preset distance range from the vehicle and within a precision range within a second preset precision threshold as preprocessed data, the low-speed driving scenario being a scenario where the vehicle is driving at a speed less than the first preset vehicle speed in the forward direction, the second preset distance range being less than the first preset distance range, and the second preset precision threshold range being greater than the first preset precision threshold range.

[0010] In some embodiments, the preprocessing module is further configured to, if the driving scenario of the vehicle is a reverse driving scenario, obtain data within a third preset distance range from the vehicle and within a third preset precision threshold range as preprocessing data from the data to be processed. The reverse driving scenario is a scenario where the vehicle is driving in the reverse direction. The distance range of the third preset distance range in the forward direction of the vehicle is less than the distance range of the first preset distance range in the forward direction of the vehicle, the distance range of the third preset distance range in the reverse direction of the vehicle is greater than the distance range of the first preset distance range in the reverse direction of the vehicle, and the third preset precision threshold range is greater than the first preset precision threshold range.

[0011] In some embodiments, the operation information further includes an operation scenario. The preprocessing module is further configured to, if the driving scenario of the vehicle is a high-speed driving scenario, obtain data within a fourth preset distance range from the vehicle and within a first preset precision threshold range as preprocessing data from the data to be processed. The high-speed driving scenario is a scenario where the operation scenario is a preset operation scenario and the vehicle is driving in the forward direction at a speed greater than the first preset vehicle speed. The distance range of the fourth preset distance range in the forward and reverse directions of the vehicle is greater than the distance range of the first preset distance range in the forward and reverse directions of the vehicle, and the distance range of the fourth preset distance range in the lateral direction of the vehicle is less than the distance range of the first preset distance range in the lateral direction of the vehicle.

[0012] In some embodiments, the computing power allocation module is further configured to determine the driving scenario of the target device based on the operation information; determine the activation degree of each target algorithm model based on the driving scenario, where the activation degree is used to represent the amount of data input to the target algorithm model in the driving scenario corresponding to the operation information; and allocate computing power to the target algorithm model based on the activation degree and the preset computing power of the target algorithm model.

[0013] In a third aspect, the present application further provides a vehicle control system, the system comprising: a data acquisition component for acquiring operation data, the operation data including the running speed and running direction of the vehicle, environmental data of the environment where the vehicle is located, and data to be processed; a processor for obtaining the data to be processed, and obtaining operation information based on the running speed, running direction of the vehicle, and environmental data of the environment where the vehicle is located; determining a target preprocessing strategy for the data to be processed from at least one preprocessing strategy based on the operation information, and determining at least one target algorithm model from a plurality of algorithm models based on the operation information; preprocessing the data to be processed based on the target preprocessing strategy to obtain preprocessed data, the target preprocessing strategy being used to determine the data processing range of the data to be processed; allocating computing power to the at least one target algorithm model based on the operation information to obtain the computing power allocated to each target algorithm model, the computing power being used to represent the amount of computing resources of the target device that the target algorithm model can call; and processing the preprocessed data using the target algorithm model according to the allocated computing power corresponding thereto to obtain a processing result.

[0014] In some embodiments, the data acquisition component includes a radar, a vehicle speed detector, and a camera; the radar is used to acquire point cloud data and use the point cloud data as the data to be processed; the vehicle speed detector is used to detect the running speed and running direction of the vehicle; and the camera is used to acquire environmental data of the environment where the vehicle is located.

[0015] In a fourth aspect, the present application further provides a vehicle, the vehicle comprising: a vehicle body and the above-mentioned vehicle control system.

[0016] The data processing method, apparatus, vehicle control system, and vehicle provided by the embodiments of the present application. The data processing method includes obtaining the data to be processed and the running information of the target device; determining the target preprocessing strategy for the data to be processed and determining at least one target algorithm model based on the running information, where the target preprocessing strategy is used to determine the data processing range of the data to be processed; preprocessing the data to be processed based on the target preprocessing strategy to obtain preprocessed data; allocating computing power to at least one target algorithm model based on the running information to obtain the computing power allocated to each target algorithm model, and using the target algorithm model to process the preprocessed data according to the corresponding allocated computing power to obtain a processing result. By adopting the data processing method provided by the embodiments of the present application, the preprocessing process of the data to be processed and the computing power allocation process of the algorithm model are coupled with the scenario. By adjusting the data processing range of the data to be processed, the preprocessed data can meet the detection requirements in different scenarios, improving the detection accuracy in different scenarios. And through computing power allocation, the target algorithm model can meet the processing requirements of the preprocessed data as much as possible, so as to make full use of the computing resources of the target device without changing the model parameters, improving the efficiency of model data processing, and thus improving the overall data processing ability.

[0017] These aspects or other aspects of the present application will be more clearly understood in the following description of the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those skilled in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0019] Figure 1 It is a flowchart of the data processing method provided by the embodiments of the present application.

[0020] Figure 2 It is a schematic diagram of point cloud voxelization sampling provided by the embodiments of the present application.

[0021] Figure 3 It is a comparison schematic diagram of image resolutions provided by the embodiments of the present application.

[0022] Figure 4 It is provided by the embodiments of the present application Figure 2 The flowchart of step S140 in

[0023] Figure 5 It is a flowchart of the training of the target algorithm model provided by the embodiments of the present application.

[0024] Figure 6It is a schematic diagram of the scenario of the vehicle-mounted front-end system provided by the embodiments of the present application.

[0025] Figure 7 It is a flowchart of the execution of the data processing method provided by the embodiments of the present application.

[0026] Figure 8 It is a schematic diagram of the data processing device provided by the embodiments of the present application.

[0027] Figure 9 It is a schematic diagram of the vehicle control system provided by the present application.

[0028] Figure 10 It is a schematic diagram of the vehicle provided by the present application. Detailed implementation manners

[0029] The following details the implementation manners of the present application. Examples of the implementation manners are shown in the accompanying drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions throughout. The implementation manners described below by referring to the accompanying drawings are exemplary only for explaining the present application and should not be construed as limiting the present application.

[0030] To enable those skilled in the art of the present technology to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts fall within the scope of protection of the present application.

[0031] A vehicle-mounted system refers to an automatic operation system that installs a mobile system on a motor vehicle to manage and control various devices on the vehicle. This system is also called a vehicle-mounted computer system, an autonomous driving vehicle, automotive electronic signal processing, an automotive navigation system, etc. The vehicle-mounted front-end is an information node of the system. Devices such as cameras (inside the vehicle), cameras (outside and inside), microphones, emergency alarm buttons, vehicle-mounted information displays, and vehicle-mounted hosts are installed inside the vehicle. As information collection devices for video images, audio data, emergency alarm information, location, etc., the collected information is transmitted to the central platform after being processed by the vehicle-mounted front-end; the data processing method provided by the present application is mainly applied to the data processing of the vehicle-mounted front-end.

[0032] As Figure 1 , Figure 1 shows a schematic flowchart of the data processing method provided by the embodiments of the present application. The data processing method includes:

[0033] S110. Obtain the data to be processed and the operation information of the target device.

[0034] Among them, the target device can be an automobile, such as a fuel vehicle, an electric vehicle or a hybrid vehicle, or a mobile device such as a mobile robot. In the embodiments of the present application, the target device is taken as an automobile for illustration.

[0035] In some embodiments, to obtain the data to be processed and the operation information of the target device, the wireless communication module of the automobile can be used to obtain them through the network; or they can be obtained through bus communication or other means.

[0036] The data to be processed of the target device can be image data, point cloud data, position data, driving data, etc. of the automobile; further, the data to be processed can be obtained by collecting the internal and external data of the automobile in real time through sensors. Exemplarily, image data is obtained through a camera, point cloud data is obtained through a lidar or millimeter wave radar, and driving data is obtained through an accelerator position sensor, etc.

[0037] The operation information of the target device can include the external operation environment information of the automobile, such as weather information (such as rainy day, sunny day, day or night, etc.) and driving environment information (such as highway, forest road, urban road, etc.), and can also include the driving information of the target device itself, such as speed (such as low speed, medium speed, high speed, etc.), operation direction (such as forward, backward, turning, climbing, etc.). The corresponding scenario can be determined through the operation information.

[0038] Further, the operation information of the target device can be obtained through external devices. For example, through a rain sensor, it can be determined that the operation weather scenario is a rainy day; through a camera, it can be determined that the operation road is a highway scenario, etc.

[0039] It should be noted that when multiple pieces of operation information are obtained, the obtained operation information can be automatically corrected according to the relationship between the multiple pieces of operation information, so as to improve the accuracy of the operation information; Exemplarily, through an acceleration sensor, the operation information is determined to be driving at medium speed, and at the same time, through a rain sensor, the operation information is determined to be a rainy day. If no correction is made, the obtained operation information is driving at medium speed on a rainy day; considering that the safe driving speed on a rainy day is usually low, driving at medium speed on a rainy day can be corrected to driving at high speed on a rainy day, so as to improve the accuracy of the operation information.

[0040] S120. Determine the target preprocessing strategy for the data to be processed from at least one preprocessing strategy based on the operation information, and determine at least one target algorithm model from multiple algorithm models based on the operation information.

[0041] Among them, preprocessing refers to operations such as data cleaning, feature selection, feature extraction, and feature transformation on the original data before data analysis, machine learning, or deep learning model training, so as to better meet the requirements of machine learning or deep learning algorithms.

[0042] In the embodiments of the present application, a preprocessing strategy refers to a set of a series of preprocessing operations; further, for each preprocessing strategy, it may include different preprocessing operations. For example, preprocessing strategy A includes downsampling and feature extraction, and preprocessing strategy B includes data cleaning and downsampling; it may also be different processing parameters including the same preprocessing operations. For example, both preprocessing strategies A and B include downsampling and voxel division, but the sampling frequencies and voxel division granularities of A and B are different.

[0043] Among them, multiple algorithm models refer to all the models required to process the data to be processed in each scenario, and the target algorithm model refers to the model required to process the data to be processed in a certain scenario; exemplarily, when processing the image data and point cloud data of a vehicle, multiple algorithm models may include an obstacle detection model, a lane detection model, a drivable area detection model, a parking space detection model, etc. When the scenario confirmed by the running information is a parking scenario, the target algorithm models are determined to be the obstacle detection model and the parking space detection model.

[0044] In some embodiments, the target device is a vehicle, and the running information includes the running speed and the running direction. Determining the target preprocessing strategy for the data to be processed from at least one preprocessing strategy includes:

[0045] S121. Determine the driving scenario of the vehicle based on the running speed and the running direction of the vehicle.

[0046] Exemplarily, the driving scenario may include low-speed forward, medium-speed forward, high-speed forward, reverse, etc.

[0047] It should be noted that the running information may also include other running information, such as road scenario, weather condition, etc. Determining the driving scenario of the vehicle based on the running speed and the running direction of the vehicle may also be determining the driving scenario of the vehicle based on the running speed, the running direction and other running information of the vehicle.

[0048] It can be understood that by adding other running information, the driving scenario can be made more specific and clear, such as high-speed forward in rainy days, low-speed forward on rural roads, etc.

[0049] Exemplarily, in the embodiments of the present application, when the running speed is greater than 60 km / h and the vehicle is moving forward, the driving scenario is determined to be a normal driving scenario; when the running speed is less than 60 km / h and the vehicle is moving forward, the driving scenario is determined to be a low-speed driving scenario; when the running speed is greater than 60 km / h, the vehicle is moving forward and the vehicle is driving on a highway, the driving scenario is determined to be a high-speed driving scenario; when the running direction is the reverse direction of the vehicle, the driving scenario is determined to be a reverse driving scenario.

[0050] S122. Obtain a target preprocessing strategy corresponding to the driving scenario from the correspondence between the preset driving scenarios and preprocessing strategies.

[0051] Among them, the correspondence includes multiple driving scenarios and the preprocessing strategies corresponding to each driving scenario.

[0052] It should be noted that in some embodiments, the process of determining at least one target algorithm model from multiple algorithm models based on the running information is similar to the processes of steps S121 - S122.

[0053] The only difference is that when determining at least one target algorithm model from multiple algorithm models based on the running information, it is necessary to obtain the target algorithm model corresponding to the driving scenario from the correspondence between the preset driving scenarios and algorithm models. Among them, the correspondence includes multiple driving scenarios and the target algorithm models corresponding to each driving scenario.

[0054] S130. Preprocess the data to be processed based on the target preprocessing strategy to obtain preprocessed data.

[0055] Among them, the target preprocessing strategy is used to determine the data processing range of the data to be processed.

[0056] It should be noted that in the embodiments of the present application, the data processing range includes a distance range and a precision threshold range; among them, the distance range is centered on the vehicle and includes a certain distance range in front of, behind, to the left, to the right, and even above and below the vehicle; the precision threshold range refers to the precision for sampling or dividing all the data to be processed within the distance range, which can be a specific precision value or a precision range.

[0057] Exemplarily, step S130 specifically includes:

[0058] If the driving scenario of the vehicle is a normal driving scenario, obtain the data within the first preset distance range from the vehicle and within the first preset precision threshold range as the preprocessed data. The normal driving scenario is a scenario where the vehicle drives at a speed greater than the first preset speed in the forward direction.

[0059] If the driving scenario of the vehicle is a low - speed driving scenario, obtain the data within the second preset distance range from the vehicle and within the second preset precision threshold range as the preprocessed data. The low - speed driving scenario is a scenario where the vehicle drives at a speed less than the first preset speed in the forward direction. The second preset distance range is less than the first preset distance range, and the second preset precision threshold range is greater than the first preset precision threshold range.

[0060] If the driving scenario of the vehicle is a reverse driving scenario, obtain the data within the third preset distance range from the vehicle and within the third preset accuracy threshold range as preprocessed data from the data to be processed. The reverse driving scenario is a scenario where the vehicle drives in the reverse direction. The third preset distance range is less than the first preset distance range in the forward direction of the vehicle, and the third preset accuracy threshold range is greater than the first preset accuracy threshold range.

[0061] The operation information also includes the operation scenario. If the driving scenario of the vehicle is a high-speed driving scenario, obtain the data within the fourth preset distance range from the vehicle and within the first preset accuracy threshold range as preprocessed data from the data to be processed. The high-speed driving scenario is a scenario where the operation scenario is a preset operation scenario and the vehicle drives at a speed greater than the first preset vehicle speed in the forward direction. The fourth preset distance range is greater than the first preset distance range in the forward and reverse directions of the vehicle, and the fourth preset distance range in the lateral direction of the vehicle is less than the first preset distance range in the lateral direction of the vehicle.

[0062] It should be noted that the reduction of the preset distance range means the reduction of the data collection range to be processed, that is, the reduction of the amount of data collected; while the increase of the preset accuracy threshold range means the increase of the accuracy of data collection or division to be processed, that is, the increase of the amount of data collected.

[0063] Through the method provided by the above embodiments, compared with the normal driving scenario, in the low-speed driving scenario, the collection distance range is reduced and the collection accuracy is increased; in the high-speed driving scenario, the lateral collection distance range is reduced and the front and rear collection distances are increased; in the reverse driving scenario, the collection distance in the forward direction is reduced and the collection accuracy is increased; thereby, the amount of preprocessed data in different driving scenarios can be kept stable or even reduced, so as to improve the detection effect without changing the model parameters and without increasing the computing power.

[0064] For the sake of easy understanding, in the embodiments of the present application, taking the data to be processed as the point cloud data collected by the lidar and the image data collected by the camera as an example, it should be noted that both the point cloud data and the image data contain depth information, and the depth information is the distance information between the point cloud or the image and the vehicle. Step S130 specifically includes:

[0065] If the forward vehicle speed speed is greater than or equal to 60 km / h, it is determined as a normal driving scenario. The first preset distance range is 200 meters in the front, 50 meters in the rear, and 50 meters on each side. That is, collect the data of the depth information within the first preset distance range in the image data and the point cloud data, and set the first preset accuracy threshold range as a fixed value, so that the computing power load required for the preprocessed data at this time is 100%.

[0066] If the forward vehicle speed speed is less than 60 km / h, it is judged as driving forward at a low speed. Through the calibration value rate_x1, the forward acquisition distance is reduced to rate_x1 * speed; through the calibration value rate_y1, the left and right acquisition distances are reduced to rate_y1 * speed, reducing the detection range, and then improving the image resolution and the voxel density of the point cloud, thereby improving the feature resolution and the detection performance.

[0067] If the forward vehicle speed speed is greater than 60 km / h and the running scenario is a highway, it is judged as a high-speed driving scenario. Through the calibration value rate_x2, the forward acquisition distance is increased to rate_x2 * speed; through the calibration value rate_y2, the lateral acquisition distance is reduced to rate_y2 * speed. By setting the proportional relationship between rate_x2 and rate_y2, the detection range remains unchanged (i.e., the detection range has the same spatial area or volume), and the actual computing power consumption remains unchanged. In addition, the acquisition accuracy can be reduced to reduce the overall computing power consumption.

[0068] If the running direction is the reverse direction, it is judged as a reverse driving scenario. The point cloud data and image data in the forward direction of the vehicle are discarded, and the voxel density of the point cloud and the pixel density of the image in the reverse direction of the vehicle are improved; further, more accurate and stable data can be obtained by fusing multiple frames of image data or point cloud data.

[0069] Among them, the point cloud voxel is the sampling result obtained by downsampling the point cloud data; further, voxel downsampling is to voxelize the three-dimensional space and then sample a point in each voxel. Usually, the center point or the point closest to the center can be used as the sampling point. During the sampling process, the size of the voxel needs to be set in advance, and the voxel density is controlled by controlling the size of the voxel.

[0070] Exemplarily, as Figure 2 shown, Figure 2 shows a schematic diagram of point cloud voxelization sampling. Taking a two-dimensional plane as an example, a grid in the figure represents a voxel, and all voxels constitute the data acquisition range. Figure 2 Both a and b in it have 10×16 voxels. Obviously, the data acquisition range of a is larger than that of b, while the acquisition accuracy of b is higher than that of a; although a and b can have the same data volume, the data focus is different. a focuses on the overall point cloud data, and b focuses on the point cloud data close to the vehicle. From a to b is the process of reducing the acquisition range and increasing the voxel density of the point cloud.

[0071] Similarly, for image data, downsampling is often achieved through pooling, upsampling is achieved through interpolation, and the image resolution is changed by adjusting the specific parameters of pooling or interpolation. As Figure 3 shown,Figure 3 A comparison schematic diagram of the image resolution is given. From Figure 3 a to b in

[0072] It should be noted that, in order to achieve a smooth switch between the change in the distance range and the change in the accuracy, in the embodiments of the present application, a functional relationship between the distance and the accuracy is established through a sliding mode algorithm. After confirming the change in the distance range, the change in the voxel density of the point cloud and the change in the image resolution (i.e., the change in the accuracy) are controlled through the sliding mode algorithm.

[0073] S140. Allocate computing power to at least one target algorithm model based on the running information, and obtain the computing power allocated to each target algorithm model.

[0074] Among them, the computing power is used to represent the amount of computing resources of the target device that the target algorithm model can call. Specifically, the greater the computing power, the more computing resources of the target device that the target algorithm model can call.

[0075] It should be noted that when processing image data and point cloud data, it is usually processed using a graphics processing unit (GPU). For an in-vehicle GPU, its computing power is fixed. Since the GPU needs to process multiple tasks simultaneously, it is necessary to allocate the computing power of the GPU to each task according to a fixed ratio.

[0076] It can be understood that since the total amount of computing power is fixed, when a certain processing process occupies more computing power, the computing power occupied by other processing processes will inevitably decrease. Further, in the embodiments of the present application, within the range of computing power that does not exceed the upper limit of the processing capacity of the target algorithm model, the more computing power (i.e., the computing resources of the target device) that the target algorithm model can call, the more data the target algorithm model can process simultaneously, and the faster the processing speed. That is, the higher the efficiency of the target algorithm model in processing data.

[0077] It should be noted that in different scenarios, the focus of data processing is different, resulting in different amounts of data that the target algorithm model needs to process, and then resulting in different computing power required by the target algorithm model. The processing performance of the target algorithm model in different scenarios can be dynamically adjusted through computing power allocation, thereby improving the overall efficiency of output processing; illustratively, in the two scenarios of driving on a highway and driving in the rain, the target algorithm model includes an obstacle detection model and a lane detection model, but when driving on a highway, the vehicle speed is faster and the road is wider, and the focus is on obstacle detection. The obstacle detection model needs to process image data and a large amount of point cloud data, and the lane detection model only needs to simply process image data. At this time, more computing power can be allocated to the obstacle detection model; when driving in the rain, the vehicle speed is slower and the road conditions are complex, and the focus is on lane detection. The point cloud data is disturbed by the rain, and the lane detection model and the obstacle detection model only need to process image data. At this time, the computing power can be evenly distributed to the obstacle detection model and the lane detection model, thereby improving the overall performance of data processing.

[0078] In some embodiments, Figure 4 , Figure 4 Given the example provided in this application Figure 1 Schematic diagram of the process of step S140, step S140 specifically includes:

[0079] S141. Determine a driving scenario of the target device based on the operation information.

[0080] Among them, step S141 is similar to step S121 in the above embodiment. For the specific description of step S141, please refer to the specific description of step S121 in the above embodiment.

[0081] S142. Determine the degree of activation of each target algorithm model based on the driving scenario.

[0082] Among them, the activation degree is used to characterize the amount of data input into the target algorithm model in the driving scenario, that is, the activation degree represents the amount of computing power required by the target algorithm model in the driving scenario.

[0083] It can be understood that the greater the degree of activation, the more data the target algorithm model needs to process, that is, the greater the computing power required by the target algorithm model.

[0084] It should be noted that the activation degree of each target algorithm model is determined based on the driving scenario. The activation degree of each target algorithm model can be confirmed separately according to the driving scenario, and the activation degree of each target algorithm model can be the same or different; or the activation degree of all target algorithm models can be confirmed simultaneously based on the driving scenario, so that each target algorithm model has the same activation degree.

[0085] In some embodiments, when the activation level of each target algorithm model is confirmed separately according to the driving scenario, the activation level can be directly expressed by the actual size of the data volume. The activation level of each target algorithm model, that is, the computing power, is directly confirmed by the input data volume of each target algorithm model.

[0086] In other embodiments, when the activation level of all target algorithm models is confirmed simultaneously according to the driving scenario, it can also be expressed as a percentage of the preset computing power, and dynamically adjusted according to the percentage based on the preset computing power of each target algorithm model.

[0087] S143. Allocate computing power to the target algorithm model based on the activation degree and the preset computing power of the target algorithm model.

[0088] It should be noted that, in some implementations, the specific setting method of the preset computing power of the target algorithm model may be to pre-set the corresponding preset computing power for all algorithm models, and after confirming the target algorithm model, confirm the preset computing power corresponding to the target algorithm model.

[0089] It is understandable that for each algorithm model, it has the same preset computing power in different scenarios.

[0090] In other implementations, the specific setting method of the preset computing power may also be to determine the target algorithm model under each scenario, and then pre-set the preset computing power of the target algorithm model under the scenario.

[0091] It can be understood that for each algorithm model, it can have different preset computing powers in different scenarios; for example, for algorithm model a, its preset computing power in scenario A is 50, and its preset computing power in scenario B can be 25.

[0092] It should be noted that when the operating information changes, the corresponding driving scenario, target algorithm model and the computing power allocation of the target algorithm model will change. In order to ensure smooth switching of computing power allocation when switching between different scenarios, in an embodiment of the present application, step S140 is implemented through a synovial algorithm.

[0093] Exemplarily, in the embodiment of the present application, it is assumed that there are N scenes, which are represented as S, S = {s_1, s_2, ..., s_N}, and for one of the scenes s_i, the preset computing power of its target algorithm model is w_i, w_i = {w1_i, w2_i, ..., wn_i}, where n represents the number of target algorithm models corresponding to the scene s_i;

[0094] For each scene, construct its corresponding switching function h(s_i), h(s_i)=w_i*x_i.

[0095] Among them, \(x_i\) is a parameter between 0 and 1, and \(x_i\) represents the activation degree of scenario \(s_i\) at this time. When \(x_i\) is close to 1, it means that scenario \(s_i\) requires a relatively high GPU computing power; when \(x_i\) is close to 0, it means that scenario \(s_i\) requires a relatively low GPU computing power. That is, \(x_i\) represents the activation degree of all target algorithm models under scenario \(s_i\).

[0096] Considering that the total amount of GPU resources is fixed, define the total amount of GPU resources as \(R\). For the computing power allocation functions under all scenarios, the following constraint conditions need to be satisfied:

[0097] \(\sum_{i = 1}^{N}h(s_i)=R\), where \(i = 1,2,\cdots,N\).

[0098] In some embodiments, the above constraint conditions can be solved by an optimization algorithm, such as the gradient descent algorithm or the Lagrange multiplier method, etc.

[0099] The method in the embodiment of the present application runs the information driving scenario, determines the activation degree through the driving scenario, and confirms the computing power allocation of the target algorithm model through the activation degree. Thus, the GPU resources can be dynamically allocated according to the requirements of different driving scenarios, improving the data processing performance of the target algorithm model; at the same time, by using the sliding mode algorithm, when switching between different scenarios, the switching of the computing power allocation of the target algorithm model is smoother, improving the stability of the processing process of the target algorithm model.

[0100] S150. Use the target algorithm model to process the preprocessed data according to the corresponding allocated computing power to obtain a processing result.

[0101] Among them, the processing of the preprocessed data by the target algorithm model may include obstacle detection, lane detection, drivable area detection, parking space inspection, etc.; further, the processing result can be used for point cloud detection, such as 3D composition, visual detection, etc.

[0102] Considering that the target algorithm model needs to process preprocessed data obtained by different preprocessing strategies, in order to ensure that the target algorithm model has a good processing effect on different preprocessed data, in some embodiments, before step 150, the target algorithm model also needs to be trained. The training process of the target algorithm model is as Figure 5 shown Figure 5 shows a schematic diagram of the training process of the target algorithm model, which specifically includes:

[0103] S151. Obtain data samples.

[0104] It should be noted that the data in the data sample and the data to be processed are of the same type; for example, if the data to be processed is image data, the data sample is an image data sample; if the data to be processed is point cloud data, the data sample is a point cloud data sample.

[0105] S152. Respectively preprocess the data sample by using each preprocessing strategy to obtain a preprocessed data sample corresponding to each preprocessing strategy.

[0106] It should be noted that since multiple data samples can be set, the number of data samples processed by each preprocessing strategy can be the same or different.

[0107] Furthermore, when the number of data samples processed by each preprocessing strategy is different, the number of data samples processed by each preprocessing strategy can be determined by the usage frequency of the preprocessing strategy, that is, the higher the usage frequency of the preprocessing strategy, the more data samples are processed by this preprocessing strategy, and the more preprocessed data samples are obtained.

[0108] S153. Train the target algorithm model based on the preprocessed data sample to obtain the target algorithm model.

[0109] It can be understood that through the training of the target algorithm model, the data processing range of the target algorithm model is improved, so that the target algorithm model has good processing effects on the preprocessed data samples obtained by different preprocessing strategies.

[0110] The method provided by the embodiment of the present application obtains the data to be processed and the running information of the target device; determines the target preprocessing strategy for the data to be processed and determines at least one target algorithm model based on the running information; preprocesses the data to be processed based on the target preprocessing strategy to obtain preprocessed data; performs computing power allocation on at least one target algorithm model based on the running information to obtain the computing power allocated to each target algorithm model, and uses the target algorithm model to process the preprocessed data according to the corresponding allocated computing power to obtain a processing result; by adopting the data processing method provided by the embodiment of the present application, the preprocessing process and the computing power allocation process of the data to be processed are coupled with the scenario, which not only enables the preprocessed data to meet the detection requirements in different scenarios and improves the detection accuracy in different scenarios, but also through the computing power allocation, enables the target algorithm model to meet the processing requirements of the preprocessed data as much as possible, so as to make full use of the computing resources of the target device without changing the model parameters and without increasing the computing power, improve the efficiency of model data processing, and thus improve the overall data processing ability.

[0111] For ease of understanding, please refer to Figure 6 , Figure 6Fig. 0 shows a schematic scenario diagram of the data processing method provided by the embodiments of the present application applied to an in-vehicle front-end system. The in-vehicle front-end system 200 includes a sensing unit 210, a control unit 220, and a detection unit 230.

[0112] Among them, the sensing unit 210 includes sensors and communication modules. The sensors include, but are not limited to, lidar, millimeter-wave radar, and cameras; the communication modules include, but are not limited to, wireless communication modules or bus communication modules.

[0113] The control unit 220, that is, the in-vehicle control chip, includes a preprocessing unit, a model unit, and a scheduling unit. Among them, the preprocessing unit, the model unit, and the scheduling unit are all stored in the control unit in the form of software or programs.

[0114] Furthermore, the sensors acquire information inside and outside the vehicle to obtain data to be processed and operation data, and transmit them to the scheduling unit and the preprocessing unit through the communication module. The scheduling unit confirms the driving scenario according to the operation information transmitted by the communication module, determines the preprocessing strategy of the preprocessing unit according to the driving scenario, and allocates computing power to the target algorithm model in the model unit; the preprocessing unit receives the data to be processed transmitted by the communication module, and preprocesses the data to be processed according to the confirmed preprocessing strategy to obtain preprocessed data; the model unit processes the preprocessed data and outputs the processing result to the detection unit 230; the detection unit 230 performs a series of detections according to the processing result, such as map detection, visual detection, etc.

[0115] Furthermore, in the above application scenario of the vehicle, considering the driving requirements of different drivers, it is also possible to add a judgment on the driving mode of the target device in the above application scenario. When it is confirmed that the driving mode of the target device is the adjustment mode, the data processing method provided by the above embodiments is used to process the data, so as to meet the driving requirements of different drivers.

[0116] Such as Figure 7 , Figure 7 Fig. 19 shows the execution flowchart of the data processing method provided by the embodiments of the present application, including:

[0117] During normal vehicle driving, determine whether the driving mode of the vehicle is an adjustment mode. If the driving mode is the adjustment mode, use the scheduling unit to confirm the driving scenario. When the driving scenario is a low-speed driving scenario, increase the point cloud voxel density and the image pixel density, and reduce the horizontal and vertical distances of the point cloud data and the image data (i.e., the front-back direction and the left-right direction of the vehicle). When the driving scenario is a high-speed driving scenario, reduce the point cloud voxel density and the image pixel density, increase the vertical distance of the point cloud data and the image data (i.e., the front-back direction of the vehicle), and reduce the horizontal distance of the point cloud data and the image data (i.e., the left-right direction of the vehicle). When the driving scenario is a reverse driving scenario, discard the forward data of the point cloud data and the image data, and increase the rearward point cloud voxel density and the image pixel density, so as to obtain preprocessed data.

[0118] After the preprocessed data is processed by the expert system scheduling model and the processing result is obtained, the scheduling unit simultaneously receives the processing result and makes a real-time judgment on the driving scenario in combination with the processing result, so as to ensure the accuracy of the driving scenario judgment.

[0119] In some embodiments, the present application further provides a data processing device, such as Figure 8 , Figure 8 shows a schematic diagram of the data processing device provided by the present application. The data processing device 300 includes:

[0120] An acquisition module 310, configured to acquire the data to be processed and the running information of the target device.

[0121] A selection module 320, configured to determine the target preprocessing strategy for the data to be processed from at least one preprocessing strategy based on the running information, and determine at least one target algorithm model from multiple algorithm models based on the running information.

[0122] A preprocessing module 330, configured to preprocess the data to be processed based on the target preprocessing strategy to obtain preprocessed data, and the target preprocessing strategy is used to determine the data processing range of the data to be processed.

[0123] A computing power allocation module 340, configured to allocate computing power to at least one target algorithm model based on the running information to obtain the computing power allocated to each target algorithm model, and the computing power is used to characterize the resource amount of the computing resources of the target device that the target algorithm model can call.

[0124] A data processing module 350, configured to process the preprocessed data using the target algorithm model according to the allocated computing power to obtain a processing result.

[0125] In some embodiments, the target device is a vehicle, the operation information includes an operation speed and an operation direction, and the selection module 320 is further configured to determine the driving scenario of the vehicle based on the operation speed and the operation direction of the vehicle; obtain a target preprocessing strategy corresponding to the driving scenario from the correspondence between the preset driving scenarios and the preprocessing strategies, where the correspondence includes a plurality of driving scenarios and the preprocessing strategies corresponding to each driving scenario.

[0126] In some embodiments, the preprocessing module 330 is configured to, if the driving scenario of the vehicle is a normal driving scenario, obtain data within a first preset distance range from the vehicle and within a first preset precision threshold range as preprocessing data from the data to be processed, where the normal driving scenario is a scenario in which the vehicle drives at a speed greater than the first preset vehicle speed in the forward direction.

[0127] In some embodiments, the preprocessing module 330 is further configured to, if the driving scenario of the vehicle is a low-speed driving scenario, obtain data within a second preset distance range from the vehicle and within a second preset precision threshold range as preprocessing data from the data to be processed, where the low-speed driving scenario is a scenario in which the vehicle drives at a speed less than the first preset vehicle speed in the forward direction, the second preset distance range is less than the first preset distance range, and the second preset precision threshold range is greater than the first preset precision threshold range.

[0128] In some embodiments, the preprocessing module 330 is further configured to, if the driving scenario of the vehicle is a reverse driving scenario, obtain data within a third preset distance range from the vehicle and within a third preset precision threshold range as preprocessing data from the data to be processed, where the reverse driving scenario is a scenario in which the vehicle drives in the reverse direction, the distance of the third preset distance range in the forward direction of the vehicle is less than the distance of the first preset distance range in the forward direction of the vehicle, the distance of the third preset distance range in the reverse direction of the vehicle is greater than the distance of the first preset distance range in the reverse direction of the vehicle, and the third preset precision threshold range is greater than the first preset precision threshold range.

[0129] In some embodiments, the operation information further includes an operation scenario, and the preprocessing module 330 is further configured to, if the driving scenario of the vehicle is a high-speed driving scenario, obtain data within a fourth preset distance range from the vehicle and within a first preset precision threshold range as preprocessing data from the data to be processed, where the high-speed driving scenario is a scenario in which the operation scenario is a preset operation scenario and the vehicle drives at a speed greater than the first preset vehicle speed in the forward direction, the distance of the fourth preset distance range in the forward and reverse directions of the vehicle is greater than the distance of the first preset distance range in the forward and reverse directions of the vehicle, and the distance of the fourth preset distance range in the lateral direction of the vehicle is less than the distance of the first preset distance range in the lateral direction of the vehicle.

[0130] In some embodiments, the computing power allocation module 340 is further configured to determine the driving scenario of the target device based on the operation information; determine the activation degree of each target algorithm model based on the driving scenario, where the activation degree is used to characterize the amount of data input to the target algorithm model in the driving scenario corresponding to the operation information; and allocate computing power to the target algorithm model based on the activation degree and the preset computing power of the target algorithm model.

[0131] In some embodiments, the present application further provides a vehicle control system, such as Figure 9 , Figure 9 FIG. shows a schematic diagram of the vehicle control system provided by the present application. The vehicle control system 400 includes a data acquisition component 410 and a processor 420.

[0132] Among them, the data acquisition component 410 is configured to acquire operation data, where the operation data includes the running speed and direction of the vehicle, the environmental data of the environment where the vehicle is located, and the data to be processed.

[0133] The processor 420 is configured to obtain the data to be processed, and obtain operation information based on the running speed, running direction of the vehicle, and the environmental data of the environment where the vehicle is located; determine the target preprocessing strategy for the data to be processed from at least one preprocessing strategy based on the operation information, and determine at least one target algorithm model from multiple algorithm models based on the operation information; preprocess the data to be processed based on the target preprocessing strategy to obtain preprocessed data, where the target preprocessing strategy is used to determine the data processing range of the data to be processed; allocate computing power to at least one target algorithm model based on the operation information to obtain the computing power allocated to each target algorithm model, where the computing power is used to characterize the amount of computing resources of the target device that the target algorithm model can call; and process the preprocessed data using the target algorithm model according to the allocated computing power to obtain a processing result.

[0134] In some embodiments, the data acquisition component 410 includes a radar, a vehicle speed detector, and a camera; the radar is configured to acquire point cloud data and use the point cloud data as the data to be processed; the vehicle speed detector is configured to detect the running speed and direction of the vehicle; and the camera is configured to acquire the environmental data of the environment where the vehicle is located.

[0135] In some embodiments, the present application further provides a vehicle, such as Figure 10 as shown, the vehicle 500 includes a vehicle body 510 and the vehicle control system provided by the present application (not shown in the figure).

[0136] Among them, the vehicle 500 can be an automobile powered by traditional energy such as gasoline and diesel, or a new energy vehicle such as a hybrid electric vehicle, a pure electric vehicle, or a fuel cell electric vehicle.

[0137] Since vehicle 500 adopts all the technical solutions of the above-mentioned embodiments, it has at least all the beneficial effects brought by the technical solutions of the above-mentioned embodiments, which will not be elaborated one by one here.

[0138] In some embodiments, the present application further provides an electronic device, which includes one or more processors in the vehicle control system in the foregoing embodiments; a memory; and one or more programs. One or more of the programs are stored in the memory and configured to be executed by one or more processors, and the one or more programs are configured to execute the data processing method provided by the embodiments of the present application.

[0139] In some embodiments, a computer-readable storage medium provided by the embodiments of the present application stores program codes, and the program codes can be called by a processor to execute the data processing method provided by the embodiments of the present application.

[0140] The above are only the preferred embodiments of the present application, and do not impose any formal restrictions on the present application. Although the present application has been disclosed above with the preferred embodiments, it is not intended to limit the present application. Any person skilled in the art can make some changes or modifications to the equivalent embodiments by using the technical content disclosed above without departing from the technical solution scope of the present application. However, as long as it does not depart from the content of the technical solution of the present application, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present application still fall within the scope of the technical solution of the present application.

Claims

1. A data processing method, characterized in that, Including: Obtain the data to be processed and operation information of the target device; Determine a target preprocessing strategy for the data to be processed from at least one preprocessing strategy based on the operation information, and determine at least one target algorithm model from multiple algorithm models based on the operation information; Preprocess the data to be processed based on the target preprocessing strategy to obtain preprocessed data, where the target preprocessing strategy is used to determine the data processing range of the data to be processed; Perform computing power allocation on the at least one target algorithm model based on the operation information to obtain the computing power allocated to each target algorithm model, where the computing power is used to represent the amount of computing resources of the target device that the target algorithm model can call; Use the target algorithm model to process the preprocessed data according to the allocated computing power to obtain a processing result.

2. The method according to claim 1, wherein The performing computing power allocation on the at least one target algorithm model based on the operation information to obtain the computing power allocated to each target algorithm model includes: Determine the driving scenario of the target device based on the operation information; Determine the activation degree of each target algorithm model based on the driving scenario, where the activation degree is used to represent the amount of data input to the target algorithm model in the driving scenario corresponding to the operation information; Allocate computing power to the target algorithm model based on the activation degree and the preset computing power of the target algorithm model.

3. The method according to claim 1, characterized in that, The target device is a vehicle, and the operation information includes the running speed and running direction. The determining a target preprocessing strategy for the data to be processed from at least one preprocessing strategy based on the operation information includes: Determine the driving scenario of the vehicle based on the running speed and running direction of the vehicle; Obtain a target preprocessing strategy corresponding to the driving scenario from the corresponding relationship between the preset driving scenarios and preprocessing strategies, where the corresponding relationship includes multiple driving scenarios and preprocessing strategies corresponding to each driving scenario.

4. The method according to claim 3, characterized in that, If the driving scenario of the vehicle is a normal driving scenario, the preprocessing the data to be processed based on the target preprocessing strategy to obtain preprocessed data includes: Obtain the data within a first preset distance range from the vehicle and within a precision range within a first preset precision threshold as preprocessed data. The normal driving scenario is a scenario where the vehicle drives at a speed greater than the first preset vehicle speed in the forward direction.

5. The method according to claim 4, characterized in that If the driving scenario of the vehicle is a low-speed driving scenario, the preprocessing the data to be processed based on the target preprocessing strategy to obtain preprocessed data includes: Obtain the data within a second preset distance range from the vehicle and within a precision range within a second preset precision threshold as preprocessed data. The low-speed driving scenario is a scenario where the vehicle drives at a speed less than the first preset vehicle speed in the forward direction. The second preset distance range is less than the first preset distance range, and the second preset precision threshold range is greater than the first preset precision threshold range.

6. The method according to claim 4, characterized in that If the driving scenario of the vehicle is a reverse driving scenario, preprocessing the to-be-processed data based on the target preprocessing strategy to obtain preprocessed data, including: Obtaining data within a third preset distance range from the vehicle and within a third preset accuracy threshold range as preprocessed data from the to-be-processed data. The reverse driving scenario is a scenario where the vehicle drives in the reverse direction. The third preset distance range is less than the first preset distance range in the forward direction of the vehicle, and the third preset accuracy threshold range is greater than the first preset accuracy threshold range.

7. The method according to claim 4, characterized in that, The operation information further includes an operation scenario. If the driving scenario of the vehicle is a high-speed driving scenario, preprocessing the to-be-processed data based on the target preprocessing strategy to obtain preprocessed data, including: Obtaining data within a fourth preset distance range from the vehicle and within a first preset accuracy threshold range as preprocessed data from the to-be-processed data. The high-speed driving scenario is a scenario where the operation scenario is a preset operation scenario and the vehicle drives at a speed greater than a first preset vehicle speed in the forward direction. The fourth preset distance range is greater than the first preset distance range in both the forward and reverse directions of the vehicle, and the fourth preset distance range is less than the first preset distance range in the lateral direction of the vehicle.

8. A data processing device, characterized in that An acquisition module, configured to acquire to-be-processed data and operation information of a target device; A selection module, configured to determine a target preprocessing strategy for the to-be-processed data from at least one preprocessing strategy based on the operation information, and determine at least one target algorithm model from multiple algorithm models based on the operation information; A preprocessing module, configured to preprocess the to-be-processed data based on the target preprocessing strategy to obtain preprocessed data, where the target preprocessing strategy is used to determine the data processing range of the to-be-processed data; A computing power allocation module, configured to allocate computing power to the at least one target algorithm model based on the operation information to obtain the computing power allocated to each target algorithm model, where the computing power is used to represent the resource amount of the computing resources of the target device that the target algorithm model can call; A data processing module, configured to process the preprocessed data using the target algorithm model according to the allocated computing power to obtain a processing result.

9. A vehicle control system, characterized in that, Including: A data acquisition component, configured to acquire operation data, where the operation data includes the running speed and running direction of the vehicle, the environmental data of the environment where the vehicle is located, and the to-be-processed data; A processor, configured to acquire the to-be-processed data, and obtain operation information based on the running speed, running direction of the vehicle, and the environmental data of the environment where the vehicle is located; Determine a target preprocessing strategy for the to-be-processed data from at least one preprocessing strategy based on the operation information, and determine at least one target algorithm model from multiple algorithm models based on the operation information; Preprocess the to-be-processed data based on the target preprocessing strategy to obtain preprocessed data, where the target preprocessing strategy is used to determine the data processing scope of the to-be-processed data; Allocate computing power to the at least one target algorithm model based on the operation information to obtain the computing power allocated to each target algorithm model, where the computing power is used to characterize the amount of computing resources of the target device that the target algorithm model can call; Process the preprocessed data using the target algorithm model according to the allocated computing power to obtain a processing result.

10. The system according to claim 9, wherein The data acquisition component includes a radar, a vehicle speed detector, and a camera; The radar is used to collect point cloud data and use the point cloud data as the to-be-processed data; The vehicle speed detector is used to detect the running speed and running direction of the vehicle; The camera is used to collect environmental data of the environment where the vehicle is located.

11. A vehicle, characterized in that, It includes a vehicle body and a vehicle control system according to any one of claims 9-10.

Images