A vehicle-mounted data processing method, device, equipment and medium

Abstract

The application provides a vehicle-mounted data processing method, device, equipment and medium, the method comprises the following steps: detecting a target through a plurality of vehicle-mounted sensors to obtain original vehicle-mounted data in different formats; preprocessing the original vehicle-mounted data in different formats to obtain vehicle-mounted data in the same format; effectively counting the vehicle-mounted data in the same format to obtain the vehicle-mounted data of effective sensing; performing parameter time-varying weighted fusion processing on the vehicle-mounted data of effective sensing to obtain fusion width information of the vehicle-mounted data; performing width information limiting processing on the fusion width information to obtain the fusion width information after limiting; and performing rationality judgment on the fusion width information after limiting to obtain the fusion width information of the vehicle-mounted data without exception. Through the vehicle-mounted data processing method disclosed by the application, the accuracy of the fusion width information of target recognition is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, in particular to a vehicle-mounted data processing method and device, equipment and medium. BACKGROUND

[0002] In the automatic driving technology, the perception and positioning of the target use a weighting scheme as a common technical solution, which involves fusing the width information according to this weighting scheme. This method is simple to implement and can be quickly developed and deployed with only some engineering experience. However, in the automatic driving mainly in urban areas, the types of observed targets increase, the number of targets is more intensive, and the observed targets not only travel in front of the vehicle for a long time in high-speed scenarios, but also frequently deviate at a large angle, appear at many traffic lights, and even appear in the case of a parked vehicle approaching the front vehicle. In these non-high-speed standard working conditions, the target recognition effect of each sensor is reduced to varying degrees, especially the vehicle-mounted camera. In such urban working conditions, the recognition distance of the vehicle-mounted camera for the target is shortened, the recognition stability is deteriorated, and the target information is greatly different from the actual situation. Among them, the most obvious difference is the reduction of the target width recognition performance. SUMMARY

[0003] In view of the above-mentioned shortcomings of the prior art, the present application provides a vehicle-mounted data processing method, device, equipment and medium to solve the problem of abnormal changes and insufficient precision of the width information in the traditional target perception algorithm due to inaccurate sensor measurement in the automatic driving scene mainly in urban areas, and the reduction of target width recognition performance.

[0004] The present application provides a vehicle-mounted data processing method, which comprises:

[0005] Identifying the target through multiple vehicle-mounted sensors to obtain original vehicle-mounted data in different formats;

[0006] Preprocessing the original vehicle-mounted data in different formats to obtain vehicle-mounted data in the same format;

[0007] Effectively counting the vehicle-mounted data in the same format to obtain the vehicle-mounted data of effective sensing;

[0008] Performing parameter time-varying weighted fusion processing on the vehicle-mounted data of effective sensing to obtain the fusion width information of the vehicle-mounted data;

[0009] Performing width information limiting processing on the fusion width information to obtain the fusion width information after limiting; and

[0010] Reasonably judging the fusion width information after limiting to obtain the fusion width information of the vehicle-mounted data without abnormalities.

[0011] In an embodiment of the present application, the obtaining of the original vehicle data in different formats comprises the following steps:

[0012] The target is identified by a visual sensor and a radar, and target data is obtained;

[0013] Vehicle driving information and lane line information are obtained by accessing a communication protocol interface; and

[0014] The target data is calibrated according to the vehicle driving information, and calibrated target data is obtained.

[0015] In an embodiment of the present application, the obtaining of the original vehicle data in different formats further comprises the following steps

[0016] Priori knowledge information of the output of a visual sensor is obtained, and vehicle dynamics model information is obtained; and

[0017] The calibrated target data is corrected according to the priori knowledge information and the vehicle dynamics model information, and visual original vehicle data is obtained.

[0018] In an embodiment of the present application, the obtaining of the original vehicle data in different formats further comprises the following steps:

[0019] Priori knowledge information of the output of a radar is obtained; and

[0020] The calibrated target data is filtered according to the lane line information of the vehicle and the priori knowledge information of the output of the radar, and radar original vehicle data is obtained.

[0021] In an embodiment of the present application, the preprocessing of the original vehicle data in different formats comprises the following steps:

[0022] The original vehicle data is obtained;

[0023] The original vehicle data is converted into the same format, and the vehicle data in the same format is obtained; and

[0024] The vehicle data in the same format is subjected to time synchronization processing, and the vehicle data with unified time is obtained.

[0025] In an embodiment of the present application, the parameter time-varying weighted fusion processing of the vehicle data of an effective sensor comprises the following steps:

[0026] The scene information in which the vehicle is located is obtained;

[0027] According to the scene information in which the vehicle is located, reference weight information of multiple sensors is obtained; and

[0028] According to the reference weight information of the plurality of sensors, the parameter time-varying weighted fusion processing is performed on the valid sensed vehicle-mounted data.

[0029] In an embodiment of the present application, the rationality judgment on the limited fusion width information comprises the following steps:

[0030] The limited fusion width information is obtained, and the normal fusion width information at the latest time is obtained;

[0031] The maximum value of the target width change value under different longitudinal distances is obtained;

[0032] It is judged whether the limited fusion width is greater than the maximum value of the target width change value under different longitudinal distances, and if the limited fusion width is greater than the maximum value of the target width change value under different longitudinal distances, it is determined to be abnormal, and the normal fusion width information at the latest time is called as the reference output.

[0033] The present application provides a kind of processing device of vehicle-mounted data, the device comprises:

[0034] Raw data acquisition module, for detecting target by multiple vehicle-mounted sensors, obtains different formats of raw vehicle-mounted data;

[0035] Data preprocessing module, for pre-processing different formats of the raw vehicle-mounted data, obtains vehicle-mounted data of the same format;

[0036] Data effective statistical module, for effective statistics of the vehicle-mounted data of the same format, obtains the vehicle-mounted data of valid sensing;

[0037] Weighted fusion processing module, for parameter time-varying weighted fusion processing of the vehicle-mounted data of valid sensing, obtains the fusion width information of the vehicle-mounted data;

[0038] Width information limiting module, for width information limiting processing of the fusion width information, obtains the limited fusion width information; and

[0039] Rationality judgment module, for rationality judgment on the limited fusion width information, obtains the fusion width information of the vehicle-mounted data without exception.

[0040] The present application provides an electronic device, the device comprises one or more processors;

[0041] Storage device, for storing one or more programs, when the one or more programs are executed by the one or more processors, so that the electronic device realizes the processing method of any one of the vehicle-mounted data.

[0042] The application provides a computer readable storage medium, and the computer readable storage medium stores computer instructions.

[0043] The application provides a vehicle-mounted data processing method, which improves the accuracy of width information fusion of vehicle-mounted data by performing parameter time-varying weighting fusion processing on vehicle-mounted data of the same format and effective sensors, and effectively avoids width information abnormalities by performing width information limiting and rationality judgment on the vehicle-mounted data after parameter time-varying weighting fusion processing, thereby improving the accuracy, stability and robustness of width information output, and providing reliable data input for subsequent automatic driving function modules.

[0044] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and are not limiting to the application. BRIEF DESCRIPTION OF DRAWINGS

[0045] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the application and, together with the specification, serve to explain the principles of the application. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained from these drawings without creative labor for those skilled in the art. In the drawings:

[0046] Figure 1 is a schematic diagram of an implementation environment of the vehicle-mounted data processing method according to an exemplary embodiment of the application;

[0047] Figure 2 is a flowchart of the vehicle-mounted data processing method according to an exemplary embodiment of the application;

[0048] Figure 3 is a flowchart of the acquisition method of visual original vehicle-mounted data according to an exemplary embodiment of the application.

[0049] Figure 4 is a flowchart of the acquisition method of radar original vehicle-mounted data according to an exemplary embodiment of the application.

[0050] Figure 5 is a flowchart of the mapping and deserialization method of first data according to an exemplary embodiment of the application Figure 5 is a flowchart of the preprocessing method of original vehicle-mounted data according to an exemplary embodiment of the application.

[0051] Figure 6is a flow chart of a parameter time-varying weighting fusion processing method of effective sensed vehicle data shown by an exemplary embodiment of the present application.

[0052] Figure 7 is a flow chart of a reasonableness judgment method of the fusion width information after limiting shown by an exemplary embodiment of the present application.

[0053] Figure 8 is a block diagram of a processing device of vehicle data shown by an exemplary embodiment of the present application.

[0054] Figure 9 A structural schematic diagram of a computer system of an electronic device suitable for implementing the embodiments of the present application is shown. DETAILED DESCRIPTION

[0055] The embodiments of the present application will be described hereinafter with reference to the drawings and preferred embodiments, and other advantages and effects of the present application can be easily understood by those skilled in the art from the contents disclosed in the present specification. The present application can be implemented or applied in other different specific embodiments, and each detail in the present specification can be modified or changed based on different views and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for explaining the present application, and are not intended to limit the protection scope of the present application.

[0056] It should be noted that the diagrams provided in the following embodiments only schematically illustrate the basic concept of the present application, and only the components related to the present application are shown in the diagrams, rather than the number, shape and size of the components when actually implemented. The type, number and ratio of the components when actually implemented can be arbitrarily changed, and the layout type of the components can be more complex.

[0057] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application, however, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details, and in other embodiments, the well-known structures and devices are shown in the form of block diagrams rather than in the form of details, to avoid making the embodiments of the present application difficult to understand.

[0058] First of all, it needs to be explained that data fusion refers to making full use of data resources obtained by different time and space multi-sensor, using computer technology to analyze, synthesize, dominate and use multi-sensor observation data obtained in time sequence under certain criteria, to obtain consistent interpretation and description of the measured object, and then realize corresponding decision and estimation, so that the system obtains more accurate, more complete and more reliable estimation and judgment than the information source obtained by single sensor. The main application fields of data fusion are: automatic driving, multi-source image composite, robot and intelligent instrument system, battlefield and unmanned aircraft, image analysis and understanding, target detection and tracking, automatic target recognition, etc.

[0059] Compared with the image data obtained by single sensor for target object detection, i.e. single-source remote sensing image data, the image data obtained by multiple sensors for target object detection, i.e. multi-source remote sensing image data, has multiple characteristics, including redundancy, complementarity, cooperation and information hierarchical structure characteristics. Among them, redundancy means that multi-source remote sensing image data has the same representation, description or interpretation result for the environment or target. Complementarity means that information comes from different degrees of freedom and is independent of each other. Cooperation means that different sensors have a dependent relationship with other information when observing and processing information. Information hierarchical structure characteristics means that multi-source remote sensing information processed by data fusion can appear at different information levels. These information abstraction levels include pixel layer, feature layer and decision layer. The hierarchical structure and parallel processing mechanism can also ensure the real-time performance of the system. Data fusion is to integrate the multi-band information of single sensor or the information provided by different types of sensors, eliminate the redundancy and contradiction between multi-sensor information, complement each other, improve the timeliness and reliability of remote sensing information extraction, and improve the efficiency and accuracy of data use.

[0060] Figure 1 is a schematic diagram of the implementation environment of the vehicle-mounted data processing method shown in an exemplary embodiment of the present application. As shown in Figure 1As shown, the plurality of sensors 110 identify and acquire raw input vehicle data of targets in various scenarios required by autonomous driving functions, and then the plurality of sensors 110 transmit the raw input vehicle data to the data processing module 120. Then the data processing module 120 processes the raw input vehicle data and obtains fusion width information close to the true target width, and then the data processing module 120 transmits the fusion width information to the autonomous driving system of the vehicle 130. The autonomous driving system of the vehicle 130 identifies the template of the scene around the vehicle 130 according to the fusion width information and realizes the autonomous driving of the vehicle 130. Among them, the data processing module 120 can be a Telematics Service Provider (TSP), and can also be a cloud server that provides basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN (Content Delivery Network), and big data and artificial intelligence platform, etc. Herein, no limitation is made. The transmission of data from the sensor 110 to the data processing module 120 can be operated through 3G (third generation mobile information technology), 4G (fourth generation mobile information technology), 5G (fifth generation mobile information technology) and the like. The embodiments of the present application also do not limit this, and can be set according to actual needs.

[0061] In some embodiments, in an urban-based autonomous driving scenario, the target width identification performance is reduced due to inaccurate sensor measurement, abnormal changes in width information and insufficient accuracy in traditional target perception algorithms, thereby reducing the safety of autonomous driving of the vehicle. To solve these problems, the embodiments of the present application respectively propose a vehicle data processing method, device, equipment and medium, which will be described in detail below.

[0062] Please refer to Figure 2 , Figure 2 is a flowchart of a vehicle data processing method according to an exemplary embodiment of the present application. In some embodiments, the method can be applied to Figure 1 the implementation environment shown and specifically executed by the data processing module 120 in the implementation environment. It should be understood that the method can also be applied to other exemplary implementation environments and specifically executed by devices in other implementation environments, and the present embodiment does not limit the implementation environment to which the method is applied.

[0063] Exemplarily, the data processing module 120 to which the vehicle data processing method disclosed in the embodiment is applicable can be installed with an SDK (Software Development Kit, a software development kit, a collection of development tools for the development of application software for a specific software package, software framework, operating system, etc.), and the method disclosed in the embodiment is specifically implemented as one or more functions provided by the SDK.

[0064] As shown in Figure 2 In an exemplary embodiment, the vehicle data processing method comprises at least steps S210 to S250, which are described in detail as follows:

[0065] In step S210, the target is detected by a plurality of vehicle sensors to obtain original vehicle data in different formats.

[0066] First of all, it should be noted that the plurality of vehicle sensors 110 include, for example, a front camera, a front millimeter wave radar, an angular radar, and a surround view camera. Among them, the front camera is, for example, an 800-wixel variable focal length camera, and the detection range is 0°-120° in the front direction. The front camera can detect all vehicle and pedestrian targets in the front direction, realize the regular identification of target vehicles, pedestrians, animals, and cyclists on a structured road, and can output target-level information in the form of target attributes. The front camera is configured with a Controller Area Network (CAN) communication protocol interface, and the CAN communication protocol interface is connected to the vehicle 130 to transmit the data obtained by the front camera to the vehicle 130. Figure 1 As shown in the vehicle 130 autonomous driving, the front camera obtains the speed and heading of the vehicle 130 and other driving information through a predefined signal list, and uses these driving information to calibrate the target attribute of the result of self-identification, so as to obtain the input vehicle data of the target detection of the front camera. Then the input vehicle data is parsed according to the pre-defined signal, and the data is parsed to the data buffer area for use. The road targets obtained after parsing still have a certain degree of false positives and false negatives, so it is necessary to use the prior knowledge of the camera to filter the target output, eliminate obviously unreasonable target information, and use the vehicle dynamics model to correct the output result to obtain the original vehicle data, so as to make it close to the true target value, so as to ensure the accuracy of the camera output original vehicle data. Among them, the original vehicle data of the target detection of the front camera includes the position, speed, length and width, tracking number, heading information and target type of the target. After the front camera obtains the original vehicle data, it is sent to the back end according to the predetermined protocol for preprocessing.

[0067] The front millimeter wave radar and the angular radar detect the target point cloud information in front and side front through Doppler effect, and obtain target information through point cloud clustering algorithm. The greater the speed difference between the measured target and the vehicle in the radial direction, the clearer the detected target, and the more accurate the output information. The point cloud is a mass of point set of target surface characteristics. The more dense the point cloud, the more details and information of the reflected image. Clustering is to divide a data set into different classes or clusters according to a certain standard, so that the similarity of data objects in the same cluster is as large as possible, and the difference of data objects not in the same cluster is as large as possible. The detection range of the front millimeter wave radar is 0°-120°, and the millimeter wave radar obtains the vehicle speed and heading information through, for example, a Controller Area Network (CAN) communication interface, and calibrates the output target attribute in real time. The angular radar is installed on both sides of the front bumper of the vehicle 130 shown in the figure, and the detection range is 0°-120° on both sides. Figure 1

[0068] When identifying the target by using the radar, the Radar Cross Section (RCS) is used, wherein the RCS of a target is equal to the ratio of the power reflected by the target in the direction of the radar receiving antenna per unit solid angle to the power density (per square meter) shot at the target. However, the road conditions on the road are complex, and the shielding is serious, and the interference is strong, so the false alarm often occurs only by the detection of the RCS in a single dimension. Therefore, after receiving the radar detection data, the radar detection data is first analyzed according to the pre-defined signal, the data is analyzed to the data buffer area for use, then the target attribute with large fluctuation range is removed through the prior knowledge of the radar output, and the target information in a long-term unstable state is filtered out to prevent false detection caused by interference. And through the intermediate interface, the lane line information output by the camera is obtained, the invalid target outside the vehicle lane is filtered out, and the calculation burden is reduced. Finally, the stable and accurate radar target information is output as the original vehicle-mounted data of the rear end, so as to ensure the accuracy of the radar output original vehicle-mounted data. The radar target information includes target position, speed, length, tracking number, etc. The front millimeter wave radar and the angular radar send the original vehicle-mounted data to the rear end for preprocessing according to the predetermined protocol after obtaining the original vehicle-mounted data.

[0069] The omnidirectional camera is a 2 million pixel camera, which is installed on the front bumper of the vehicle 130 shown in the figure. Figure 1 ​The main function of the vehicle 130 shown on both sides is to detect all vehicle and pedestrian targets within the range of 0°-120° on both sides of the vehicle 130 and transmit target level information to the backend. The surround view camera integrates a deep learning target recognition algorithm inside, which can cut and divide the information on the pixel layer, output the vehicle and pedestrian targets, and the surround view camera mainly makes up for the detection dead angle on both sides of the front camera. The output information includes target position, speed, length and width, tracking number, heading information and target type, etc. The output information is sent to the backend module through a predetermined protocol as the original vehicle data of the backend.

[0070] In step S220, the original vehicle data of different formats is preprocessed to obtain vehicle data of the same format.

[0071] The original vehicle data includes front camera output information, front millimeter wave radar output information, angle radar output information and surround view camera output information, etc. Different formats of original vehicle data are preprocessed. The original vehicle data is preprocessed, i.e. the data is converted to the same data type preset internally, realizing the separation of software and hardware. At the same time, the output data of all sensors is time-stamped with the time of the vehicle system for data time synchronization, and the reference origin of the position information of the targets detected by each sensor is uniformly converted to the midpoint of the front bumper of the vehicle, realizing the unification of all data.

[0072] In step S230, the vehicle data of the effective sensor is obtained by effectively counting the vehicle data of the same format.

[0073] In a plurality of sensor systems, the detection of the same target has many differences in the output characteristics of each sensor, and often some sensors cannot detect the target at a certain moment, while other sensors can detect it. Therefore, for this feature, the vehicle data of the same format is effectively counted. When counting the vehicle data, the output information of all sensors at the same time is counted and searched, and if the output is empty, i.e. the target position, speed and length and width in the output information are all 0, it means that the corresponding sensor does not detect the target. At this time, the sensor is labeled as undetected, and the subsequent fusion information directly skips this sensor, greatly reducing the data indexing time.

[0074] In step S240, the fusion width information of the vehicle data is obtained by performing parameter time-varying weighted fusion processing on the vehicle data of the effective sensor.

[0075] The parameter time-varying weighting processing of the effective vehicle-mounted data is mainly the dynamic weighting data fusion of the width information of each sensor. According to the detection effect of each sensor on the target, in principle, the width information mainly adopts the visual sensor, and due to the difference in installation position and pixels, the width information obtained by the front camera is mainly used as the reference information under normal circumstances.

[0076] When the current camera fails to detect the target, the width of the target obtained by the omnidirectional camera is used as the reference information for weighting. In the case of crossing the scene, the special-shaped target and the extreme close range, the recognition distance of the visual sensor is greatly reduced, and the target is often lost and the recognition effect is poor. At this time, the width information of the millimeter wave radar is used as the reference information. If the target is on the lateral direction of the vehicle, the width information of the angular radar is used as the reference information at this time. Among them, the cross target refers to the non-motor vehicle target such as the three-wheeled vehicle target or the bicycle target. And the parameter time-varying weighting is dynamic weighting, and in each scene, the optimal sensor with high reference weight is allocated to the width information as the reference information, and the remaining sensors are allocated with low weight, which improves the stability and accuracy of the fused width information.

[0077] In step S250, the width information limit processing is performed on the fused width information to obtain the limited fused width information.

[0078] In the extreme close range or special-shaped target scene, the visual sensor width information may output a value that exceeds the actual situation. At this time, the weighting parameter is low, and the final weighted fusion width may still be too large or too small. Therefore, according to the type of road target and the actual project function requirement, the maximum and minimum values of the target width are set. When the fused width information exceeds the maximum and minimum values of the target width, a predetermined default value is directly output, which effectively avoids the abnormality of the fused width.

[0079] In step S260, the limited fused width information is reasonably judged to obtain the fused width information of the vehicle-mounted data without abnormality.

[0080] In Figure 1In the case of the vehicle 130 shown following the front vehicle and then parking next to the front vehicle, the output of the width of the target by the visual sensor is initially normal, and as the target gradually fills the entire camera, the camera cannot normally identify the object and its type, so that the output width starts to make irregular changes. At this time, due to the close distance between the target vehicle and the vehicle, the millimeter wave radar sensor has too concentrated transmission power, all of which hits the target vehicle, resulting in that the output is all clutter at this time, without effective information. Therefore, in principle, only the camera information and the last normal historical information can be relied on. The rationality judgment of the fused width information after limiting is to use the characteristic that the camera can still normally output the target longitudinal distance at this time, and use the target longitudinal distance as an important input, and use the characteristic that the object width cannot change too much on the camera image under the same longitudinal distance, and add width information change judgment. Moreover, according to engineering experience, under different longitudinal distances, the target width has a corresponding maximum width change value, and if the value is exceeded, it is considered abnormal. Therefore, after determining the abnormality, the normal fusion data at the latest time is called, and the width information in the normal fusion data is taken as the reference output. The width information of the last frame of normal fusion data is an example of the normal fusion data at the latest time. When no abnormality is determined, the width fusion information after limiting is taken as the reference output, and the fusion width information close to the true width of the target and without abnormality is obtained.

[0081] Figure 3 is a flowchart of a method for acquiring visual sensor raw data according to an example embodiment of the present application. As shown in Figure 3 The method for acquiring visual sensor raw data includes at least steps S310 to S320, which are described in detail as follows.

[0082] In step S310, a target is identified by a visual sensor to obtain target data.

[0083] The visual sensor, such as a front camera and a surround camera, identifies targets in various scenes under automatic driving requirements and obtains target data in the form of target-level information.

[0084] In step S320, vehicle driving information is obtained by accessing a communication protocol interface.

[0085] The visual sensor, such as a camera, is configured with a Controller Area Network (CAN) communication protocol interface, and in Figure 1 When the vehicle 130 is automatically driven, the camera obtains driving information such as the speed and heading of the vehicle 130 through a predefined signal list.

[0086] In step S330, the target data is calibrated according to the vehicle driving information to obtain calibrated target data.

[0087] Step S340, acquiring priori knowledge information of the vision sensor output condition and acquiring vehicle dynamics model information.

[0088] Step S350, correcting the calibrated target data according to the priori knowledge information of the vision sensor output condition and the vehicle dynamics model information, and acquiring vision original vehicle-mounted data.

[0089] Because the calibrated target data still has a certain degree of false positives and false negatives, the target data is first filtered using the priori knowledge of the vision sensor output condition to eliminate obviously unreasonable targets, and relatively reasonable target data is obtained, and then the vehicle dynamics model is used to correct the relatively reasonable target data to make it close to the real target data, and finally the vision original vehicle-mounted data is obtained.

[0090] Figure 4 is a flowchart of a radar original vehicle-mounted data acquisition method shown by an exemplary embodiment of the present application. As shown in Figure 4 The radar original vehicle-mounted data acquisition method includes at least steps S410 to S450, which are described in detail as follows:

[0091] Step S410, identifying a target by a radar to acquire target data.

[0092] For example, by using the Doppler effect, the front millimeter wave radar and the angle radar identify targets in various scenarios under the demand of autonomous driving to acquire target data.

[0093] Step S420, acquiring vehicle driving information and lane line information by accessing a communication protocol interface.

[0094] The radar is configured with a Controller Area Network (CAN) communication protocol interface, and Figure 1 As shown in the vehicle 130 autonomous driving, the camera acquires driving information such as speed and heading of the vehicle 130 through a pre-defined signal list, and obtains lane line information output by the camera when the vehicle is driving. The lane line information obtained when the vehicle is driving is used to determine whether there is an invalid target on the lane.

[0095] Step S430, calibrating the target data according to the vehicle driving information to acquire calibrated target data.

[0096] Step S440, acquiring priori knowledge information of the radar output condition.

[0097] Step S450, filtering the calibrated target data according to the lane line information of the vehicle and the priori knowledge information of the radar output condition to acquire radar original vehicle-mounted data.

[0098] The prior knowledge of the radar output case eliminates the target data with large fluctuation range, filters the target data in long-term unstable state, and prevents false detection caused by interference. And through the interface, the lane line information output by the camera is obtained, the invalid target outside the vehicle lane is filtered, the operation burden is reduced, and finally the original vehicle data of the radar is obtained.

[0099] Figure 5 is a flowchart of the preprocessing method of the original vehicle data according to an example embodiment of the present application. As shown in Figure 5 The preprocessing method of the original vehicle data at least includes steps S510 to S530, which are described in detail as follows:

[0100] S510, obtaining original vehicle data.

[0101] The targets in each scene under the demand of automatic driving are identified by each sensor to obtain the original vehicle data.

[0102] S520, converting the original vehicle data into the same format to obtain vehicle data in the same format.

[0103] After receiving the original vehicle data obtained by each sensor, the original vehicle data is converted into the internally preset data format to obtain vehicle data in the same format, and the software and hardware are separated.

[0104] S530, performing time synchronization processing on the vehicle data in the same format to obtain vehicle data with unified time.

[0105] The vehicle data in the same format is time-stamped with the system time of the vehicle as the reference to synchronize the data time, and the midpoint of the front bumper of the vehicle is taken as the unified reference origin of the position information in the target data of each sensor to realize the unification of the data.

[0106] Figure 6 is a flowchart of the parameter time-varying weighted fusion processing method of the effective sensing vehicle data according to an example embodiment of the present application. As shown in Figure 6 The parameter time-varying weighted fusion processing method of the effective sensing vehicle data at least includes steps S610 to S640, which are described in detail as follows:

[0107] S610, obtaining effective sensing vehicle data.

[0108] S620, obtaining scene information of the vehicle.

[0109] Figure 1 The scene information of the vehicle 130 shown in the figure includes normal driving state, facing crossing scene, facing special-shaped target and extreme close range scene.

[0110] S630, obtaining reference weight information of the plurality of sensors according to the scene information in which the vehicle is located.

[0111] In the scene of normal driving state, the width information obtained by the front camera is taken as the reference information, and the target cannot be detected by the current camera, so the target width information obtained by the surround view camera is taken as the reference information. When facing the scene of crossing the scene, the special-shaped target and the extreme close distance, the recognition distance of the visual sensor is greatly reduced, and the target is often lost and the recognition effect is poor. At this time, the width information obtained by the millimeter wave radar is taken as the reference information. If the target is located on both sides of the vehicle, the target is in the high-density area of the angle radar energy, so the width information of the angle radar is taken as the reference information. Therefore, according to the scene information in which the vehicle is located, the optimal sensor which is assigned to the width information as the reference information is given a higher reference weight, and the remaining sensors are given a lower weight, so that the stability and accuracy of the fused width information are improved.

[0112] S640, performing parameter time-varying weighted fusion processing on the vehicle-mounted data of the effective sensors according to the reference weight information of the plurality of sensors.

[0113] Figure 7 The flowchart of the method for reasonably judging the limited fusion width information is shown in an exemplary embodiment of the present application. As shown in Figure 7 The method for reasonably judging the limited fusion width information at least includes steps S710 to S750, which are described in detail as follows:

[0114] S710, obtaining the limited fusion width information and obtaining the normal fusion width information at the latest time.

[0115] The normal fusion width information at the latest time is, for example, the width information of the last frame of normal data, and the normal fusion data at the latest time is, for example, the width information of the last frame of normal fusion data.

[0116] S720, obtaining the maximum value of the target width change value at different longitudinal distances.

[0117] In Figure 1In the case of the vehicle 130 shown following the front vehicle and then parking next to the front vehicle, the output of the width information of the target by the visual sensor is normal at the beginning, and as the target gradually fills the entire camera, the camera cannot normally identify the object and its type, so that the output width starts to make irregular changes. At this time, due to the close distance between the target vehicle and the vehicle, the millimeter wave radar sensor transmission power is too concentrated, all of which hits the target vehicle, resulting in all output being clutter without effective information. Therefore, in principle, only the camera information and the last normal fusion history data can be relied on. The rationality judgment of the limited fusion width information is to use the feature that the target longitudinal distance can still be normally output by the camera at this time as an important input, and use the feature that the object width cannot change greatly on the camera image under the same longitudinal distance to add width information change judgment. Moreover, according to engineering experience, under different longitudinal distances, the target width has a corresponding maximum width change value.

[0118] S730, determining whether the limited fusion width is greater than the maximum value of the target width change value under different longitudinal distances, if the limited fusion width is greater than the maximum value of the target width change value under different longitudinal distances, determining abnormal, executing step S740. If the limited fusion width is less than or equal to the maximum value of the target width change value under different longitudinal distances, executing step S750.

[0119] S740, calling the normal fusion width information at the latest time as the reference output.

[0120] S750, the limited fusion width information as the reference output.

[0121] Figure 8 is a block diagram of a vehicle data processing device according to an example embodiment of the present application. The device can be applied to Figure 1 the implementation environment shown, and is specifically configured in the data processing module 120. The device can also be applied to other example implementation environments, and is specifically configured in other devices, and the present embodiment does not limit the implementation environment to which the device is applied.

[0122] As Figure 8 shown, the example vehicle data processing device includes:

[0123] The original data acquisition module 810 is configured to identify a target through a plurality of vehicle-mounted sensors and acquire original vehicle-mounted data in different formats; the data preprocessing module 820 is configured to preprocess the original vehicle-mounted data in different formats and acquire vehicle-mounted data in a same format; the data effective statistics module 830 is configured to perform effective statistics on the vehicle-mounted data in the same format and acquire effective-sensed vehicle-mounted data; the weighted fusion processing module 840 is configured to perform parameter time-varying weighted fusion processing on the effective-sensed vehicle-mounted data and acquire fusion width information of the vehicle-mounted data; the width information limiting module 850 is configured to perform width information limiting processing on the fusion width information and acquire limiting-amplitude fusion width information; and the rationality judgment module 860 is configured to perform rationality judgment on the limiting-amplitude fusion width information and acquire fusion width information of the vehicle-mounted data close to a true target width.

[0124] The embodiment of the present application further provides an electronic device, including: one or more processors; a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the electronic device implements the vehicle-mounted data processing method provided in each of the above embodiments.

[0125] Figure 9 The structure schematic diagram of the computer system of the electronic device suitable for realizing the embodiment of the present application is shown. It should be noted that, Figure 9 The computer system 900 of the electronic device shown is only an example, and should not bring any limitation to the functions and use range of the embodiment of the present application.

[0126] As Figure 9 shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 902 or programs loaded from a storage part 908 to a random access memory (RAM) 903, for example, the method described in the above embodiment. In the RAM 903, various programs and data required for system operation are also stored. The CPU 901, the ROM 902 and the RAM 903 are connected to each other through a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0127] The following components are connected to the I / O interface 905: an input part 906 including a keyboard, a mouse, etc.; an output part 907 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), etc., and a speaker, etc.; a storage part 908 including a hard disk, etc.; and a communication part 909 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication part 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as necessary. A removable medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 910 as necessary, so that a computer program read out therefrom is installed in the storage part 908 as necessary.

[0128] In particular, according to embodiments of the present application, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing a computer program for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the central processing unit (CPU) 901, various functions defined in the system of the present application are executed.

[0129] It should be noted that the computer-readable medium in the embodiments of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium may, for example, be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, the computer-readable signal medium can include a data signal propagated in a baseband or as a carrier wave in a propagated data signal, in which the computer-readable computer program is carried. Such a propagated data signal can take on many forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate, or transmit the program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted in any suitable medium, including but not limited to wireless, wired, or the like, or any suitable combination thereof.

[0130] The flowcharts and block diagrams in the drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. Each block in the flowcharts or block diagrams can represent a module, a program segment, or a portion of code, which contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in a different order than that shown in the figures. For example, two blocks noted in succession can actually be executed substantially concurrently, or they can sometimes be executed in reverse order, depending on the functionality involved. It should also be noted that each block in the flowcharts or block diagrams, and combinations of blocks in the flowcharts or block diagrams, can be implemented by special-purpose hardware-based systems, which perform the specified functions or operations, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0131] The units described in the embodiments of the present application can be implemented in the form of software, or can be implemented in the form of hardware, and the described units can also be arranged in a processor. In some cases, the names of the units do not constitute a limitation on the units themselves.

[0132] Another aspect of the present application also provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor of a computer, so that the computer executes the vehicle data processing method as described above. The computer readable storage medium can be included in the electronic device described in the above embodiments, or can exist separately and not be assembled into the electronic device.

[0133] Another aspect of the present application also provides a computer program product or a computer program, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the vehicle data processing method provided in each of the above embodiments.

[0134] The above embodiments only exemplarily illustrate the principles and effects of the present application, and are not used to limit the present application. Any person skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes completed by those skilled in the art without departing from the spirit and technical thought of the present application should be covered by the claims of the present application.

Claims

1. A method for processing vehicle-mounted data, characterized in that, The method includes: Targets are identified using a variety of vehicle sensors, and raw vehicle data in different formats is acquired. The original vehicle data in different formats are preprocessed to obtain vehicle data in the same format; Effective statistics are performed on the vehicle data in the same format to obtain the vehicle data that is effectively sensed; The vehicle data obtained from effective sensing is subjected to time-varying weighted fusion processing to obtain the fusion width information of the vehicle data; The fused width information is subjected to width information limiting processing to obtain the limited fused width information; and The reasonableness of the fused width information after the amplitude limiting is judged, and the fused width information of the vehicle data without abnormalities is obtained; The reasonableness assessment of the fused width information after clipping includes the following steps: Obtain the fused width information after the amplitude limiting, and obtain the normal fused width information at the most recent moment; Obtain the maximum value of the target width variation under different vertical distances; and Determine whether the fused width after limiting is greater than the maximum value of the target width change value under different longitudinal distances. If the fused width after limiting is greater than the maximum value of the target width change value under different longitudinal distances, an anomaly is determined, and the most recent normal fused width information is used as the baseline output.

2. The method for processing vehicle data according to claim 1, characterized in that, Acquiring the raw vehicle data in different formats includes the following steps: Targets are identified and target data is acquired through visual sensors and radar. Vehicle driving information and lane line information can be obtained by accessing the communication protocol interface; Based on the vehicle driving information, the target data is calibrated to obtain the calibrated target data.

3. The method for processing vehicle data according to claim 2, characterized in that, Obtaining the raw vehicle data in different formats also includes the following steps: Acquire prior knowledge about the output of vision sensors and acquire vehicle dynamics model information; as well as Based on the prior knowledge information and the vehicle dynamics model information, the calibrated target data is corrected to obtain visual raw vehicle data.

4. The method for processing vehicle-mounted data according to claim 2, characterized in that, Acquiring the raw vehicle data in different formats also includes the following steps: To acquire prior knowledge about radar output; and Based on prior knowledge of the vehicle's lane information and radar output, the calibrated target data is filtered to obtain the original vehicle-mounted radar data.

5. The method for processing vehicle-mounted data according to claim 1, characterized in that, Preprocessing the raw vehicle data in different formats includes the following steps: Obtain the original vehicle data; The original vehicle data is converted to the same format to obtain vehicle data in the same format; and The vehicle data in the same format is time-synchronized to obtain vehicle data with consistent time.

6. The method for processing vehicle-mounted data according to claim 1, characterized in that, The time-varying weighted fusion processing of the effectively sensed vehicle data includes the following steps: Obtain information about the vehicle's location; Based on the vehicle's location scene information, obtain the baseline weight information of multiple sensors; and Based on the reference weight information of multiple sensors, the effectively sensed vehicle data is subjected to time-varying weighted fusion processing.

7. A vehicle-mounted data processing device, characterized in that, The device includes: The raw data acquisition module is used to identify targets through various vehicle sensors and acquire raw vehicle data in different formats. The data preprocessing module is used to preprocess the original vehicle data in different formats to obtain vehicle data in the same format. The data valid statistics module is used to perform valid statistics on the vehicle data of the same format to obtain the valid sensor data of the vehicle. The weighted fusion processing module is used to perform time-varying weighted fusion processing on the effectively sensed vehicle data to obtain the fusion width information of the vehicle data; A width information limiting module is used to perform width information limiting processing on the fused width information to obtain the limited fused width information; and The rationality judgment module is used to judge the rationality of the fused width information after amplitude limiting and obtain the fused width information of the vehicle data without anomalies. The rationality judgment of the fused width information after amplitude limiting includes the following steps: obtaining the fused width information after amplitude limiting and obtaining the normal fused width information at the most recent moment; obtaining the maximum value of the target width change value under different longitudinal distances; and judging whether the fused width after amplitude limiting is greater than the maximum value of the target width change value under different longitudinal distances. If the fused width after amplitude limiting is greater than the maximum value of the target width change value under different longitudinal distances, it is judged as abnormal, and the normal fused width information at the most recent moment is called as the baseline output.

8. An electronic device, characterized in that, The electronic device includes: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the method for processing vehicle data as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by the computer's processor, causes the computer to perform the vehicle data processing method as described in any one of claims 1 to 6.

Images