Real-time vehicle end data screening method, device and equipment, medium and vehicle

Abstract

The invention discloses a real-time vehicle end data screening method, device and equipment, a medium and a vehicle. The method comprises the following steps: receiving vehicle end surrounding environment data collected by a vehicle end sensor of a target automatic driving vehicle in real time, and when the vehicle end surrounding environment data accords with a preset image screening rule, taking the vehicle end surrounding environment data as initial screened data; and detecting the initially screened data according to a pre-established large neural network model and a pre-established small neural network model to obtain two detection results of the target shielded vehicle, and when the difference between the two detection results is greater than a preset difference threshold, taking the initially screened data as final screened data. According to the invention, the screening process of all data is completed on line on the vehicle end computing chip, the data does not need to be uploaded to the cloud end for screening, the transmission cost and the storage cost are reduced, manual labeling is not needed, the screening cost is reduced, the original multi-stage long-period data screening process is shortened to be close to real time, and the data timeliness is improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a real-time vehicle-side data screening method, device, equipment, medium and vehicle. Background Art

[0002] Autonomous driving vehicles rely on the surrounding environment data collected from sensors and lidar to achieve autonomous driving. Therefore, autonomous driving companies and car manufacturers around the world currently have large fleets equipped with sensors such as cameras, millimeter-wave radars and lidars to conduct road tests in various places, under various road conditions and in various climate conditions.

[0003] It is estimated that an autonomous taxi generates an average of 1TB of data per hour. If a fleet of more than 200 autonomous taxis is in operation, a massive amount of vehicle-side data reaching the PB level can be generated within a day. In order to effectively utilize the massive amount of vehicle-side surrounding environment data collected, it is necessary to clean and filter the collected vehicle-side surrounding environment data, so as to filter out the vehicle-side surrounding environment data that is truly conducive to the iteration of the autonomous driving algorithm.

[0004] Traditional vehicle-side data screening methods are mainly based on manual screening and some offline programs to screen vehicle-side data. The specific screening method is: first, the vehicle-side surrounding environment data is collected through vehicle-side sensors, and then all the collected vehicle-side surrounding environment data is transmitted to the cloud. Finally, fine-grained screening is performed by manual labelers or automatic screening is performed by offline software based on pre-set judgment rules to obtain vehicle-side surrounding environment data that is truly beneficial to the iteration of the autonomous driving algorithm.

[0005] The above screening method has the following problems:

[0006] 1. It often takes more than a week for the vehicle-side surrounding environment data to be collected, transmitted, uploaded to the cloud, and finally screened. The screening cycle is too long, resulting in low data timeliness;

[0007] 2. The transmission and storage costs of a large amount of unlabeled vehicle-side surrounding environment data are too high, and manual labeling is also extremely costly, resulting in excessively high screening costs. Summary of the invention

[0008] The present invention provides a real-time vehicle-side data screening method, device, equipment, medium and vehicle, which can improve data timeliness and reduce screening costs. The specific technical solution is as follows.

[0009] In a first aspect, an embodiment of the present invention provides a real-time vehicle-side data screening method, the method being applied to a vehicle-side computing chip, the method comprising:

[0010] Receiving vehicle-side surrounding environment data collected in real time by a vehicle-side sensor of a target autonomous driving vehicle, wherein the vehicle-side surrounding environment data includes a target obscured vehicle;

[0011] When the vehicle-side surrounding environment data meets the preset image screening rule, the vehicle-side surrounding environment data is used as initial screened data;

[0012] The initial filtered data are respectively detected according to a pre-established large neural network model and a small neural network model to obtain two detection results of the target blocked vehicle, and when the difference between the two detection results is greater than a preset difference threshold, the initial filtered data is used as the final filtered data;

[0013] Among them, both the large neural network model and the small neural network model are used to associate the obscured vehicle sample data with the corresponding vehicle category and vehicle position. The large neural network model is a neural network model with a number of convolution layers greater than a preset number of layers or a parameter amount greater than a preset parameter amount, and the small neural network model is a neural network model with a number of convolution layers not greater than the preset number of layers or a parameter amount not greater than the preset parameter amount.

[0014] Optionally, after the step of using the initially screened data as the final screened data, the real-time vehicle-side data screening method further includes:

[0015] The final screened data is uploaded to the cloud.

[0016] Optionally, when the vehicle-side surrounding environment data meets a preset image screening rule, the step of using the vehicle-side surrounding environment data as initial screened data includes:

[0017] When the vehicle-side surrounding environment data is data whose image edges are not blurred and whose image grayscale values ​​are within a preset grayscale value range, the vehicle-side surrounding environment data is used as initial filtered data.

[0018] Optionally, the step of detecting the initial filtered data according to the pre-established large neural network model and the pre-established small neural network model to obtain two detection results of the target obscured vehicle, and when the difference between the two detection results is greater than a preset difference threshold, using the initial filtered data as the final filtered data comprises:

[0019] Detecting the initially screened data according to a pre-established large neural network model to obtain a first position and a first category of the target obscured vehicle;

[0020] Detecting the initially filtered data according to a pre-established small neural network model to obtain a second position and a second category of the target obscured vehicle;

[0021] Performing loss calculation on the probability value of the first category and the probability value of the second category according to a preset loss function to obtain a loss value;

[0022] calculating the sum of squares of distances between the first position and the second position;

[0023] When at least one of the loss value is greater than a preset loss value and the sum of squares is greater than a preset sum of squares, the initial screened data is used as the final screened data.

[0024] Optionally, the training process of the large neural network model is:

[0025] Acquire vehicle surrounding environment sample data around the target autonomous driving vehicle, wherein the vehicle surrounding environment sample data includes blocked vehicle sample data and corresponding vehicle categories and vehicle positions;

[0026] The obscured vehicle sample data and the corresponding vehicle category and vehicle position are used as model training data to train the initial large neural network model. When the number of iterations reaches a preset number, the training is completed to obtain a large neural network model that associates the obscured vehicle sample data with the corresponding vehicle category and vehicle position.

[0027] Optionally, the large neural network model is an enlarged model corresponding to a lightweight convolutional neural network model or an open source visual neural network model.

[0028] Optionally, the method for determining the enlarged model corresponding to the lightweight convolutional neural network model is:

[0029] According to a preset model magnification method, the lightweight convolutional neural network model is magnified to a preset multiple to obtain an enlarged model corresponding to the lightweight convolutional neural network model.

[0030] Optionally, the open source visual neural network model is a deep residual network Resnet-50 model.

[0031] In a second aspect, an embodiment of the present invention provides a real-time vehicle-side data screening device, the device is applied to a vehicle-side computing chip, and the device includes:

[0032] A receiving module, used to receive vehicle-side surrounding environment data collected in real time by a vehicle-side sensor of a target autonomous driving vehicle, wherein the vehicle-side surrounding environment data includes a target obscured vehicle;

[0033] A rule screening module, used for using the vehicle-side surrounding environment data as initial screened data when the vehicle-side surrounding environment data meets the preset image screening rules;

[0034] A neural network screening module is used to detect the initial screened data according to a pre-established large neural network model and a pre-established small neural network model to obtain two detection results of the target blocked vehicle, and when the difference between the two detection results is greater than a preset difference threshold, the initial screened data is used as the final screened data;

[0035] Among them, both the large neural network model and the small neural network model are used to associate the obscured vehicle sample data with the corresponding vehicle category and vehicle position. The large neural network model is a neural network model with a number of convolution layers greater than a preset number of layers or a parameter amount greater than a preset parameter amount, and the small neural network model is a neural network model with a number of convolution layers not greater than the preset number of layers or a parameter amount not greater than the preset parameter amount.

[0036] Optionally, the above-mentioned real-time vehicle-side data screening device further includes:

[0037] The uploading module is used to upload the final screened data to the cloud after taking the initial screened data as the final screened data.

[0038] Optionally, the rule screening module is specifically used to:

[0039] When the vehicle-side surrounding environment data is data whose image edges are not blurred and whose image grayscale values ​​are within a preset grayscale value range, the vehicle-side surrounding environment data is used as initial filtered data.

[0040] Optionally, the neural network screening module includes:

[0041] A first detection submodule, configured to detect the initially screened data according to a pre-established large neural network model to obtain a first position and a first category of the target obscured vehicle;

[0042] A second detection submodule, configured to detect the initially screened data according to a pre-established small neural network model to obtain a second position and a second category of the target obscured vehicle;

[0043] A loss degree calculation submodule, used to perform loss degree calculation on the probability value of the first category and the probability value of the second category according to a preset loss degree function to obtain a loss degree value;

[0044] a square sum calculation submodule, used to calculate the square sum of the distance between the first position and the second position;

[0045] The final screening submodule is used to use the initial screened data as the final screened data when at least one of the loss value is greater than a preset loss value and the sum of squares is greater than a preset sum of squares is established.

[0046] Optionally, the device further includes a model training module, which is used to train the large neural network model, and the model training module includes:

[0047] An acquisition submodule, configured to acquire vehicle surrounding environment sample data around the target autonomous driving vehicle, wherein the vehicle surrounding environment sample data includes obscured vehicle sample data and corresponding vehicle categories and vehicle positions;

[0048] The training submodule is used to use the obscured vehicle sample data and the corresponding vehicle category and vehicle position as model training data to train the initial large neural network model. When the number of iterations reaches a preset number, the training is completed to obtain a large neural network model that associates the obscured vehicle sample data with the corresponding vehicle category and vehicle position.

[0049] Optionally, the large neural network model is an enlarged model corresponding to a lightweight convolutional neural network model or an open source visual neural network model.

[0050] Optionally, the device further includes a model determination module, the model determination module is used to determine the enlarged model corresponding to the lightweight convolutional neural network model, and the model determination module is specifically used to:

[0051] According to a preset model magnification method, the lightweight convolutional neural network model is magnified to a preset multiple to obtain an enlarged model corresponding to the lightweight convolutional neural network model.

[0052] Optionally, the open source visual neural network model is a deep residual network Resnet-50 model.

[0053] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a real-time vehicle-side data screening method as provided in any embodiment of the present invention.

[0054] In a fourth aspect, an embodiment of the present invention provides an electronic device, the electronic device comprising:

[0055] one or more processors;

[0056] The processor is coupled to a storage device, and the storage device is used to store one or more programs;

[0057] When the one or more programs are executed by the one or more processors, the electronic device implements the real-time vehicle-side data screening method provided by any embodiment of the present invention.

[0058] In a fifth aspect, an embodiment of the present invention provides a vehicle, which includes a real-time vehicle-side data screening device as provided in any embodiment of the present invention, or includes an electronic device as provided in any embodiment of the present invention.

[0059] It can be seen from the above content that the real-time vehicle-side data screening method, device, equipment, medium and vehicle provided by the embodiments of the present invention can receive the vehicle-side surrounding environment data collected in real time by the vehicle-side sensor of the target autonomous driving vehicle, wherein the vehicle-side surrounding environment data includes the target obscured vehicle; when the vehicle-side surrounding environment data meets the preset image screening rules, the vehicle-side surrounding environment data is used as the initial screened data; according to the pre-established large neural network model and small neural network model, the initial screened data are respectively detected to obtain two detection results of the target obscured vehicle, and when the difference between the two detection results is greater than the preset difference threshold, the initial screened data is used as the final screened data; wherein the large neural network model and the small neural network model are both used to associate the obscured vehicle sample data with the corresponding vehicle category and vehicle position, the large neural network model is a neural network model with a number of convolution layers greater than the preset number of layers or a parameter amount greater than the preset parameter amount, and the small neural network model is a neural network model with a number of convolution layers not greater than the preset number of layers or a parameter amount not greater than the preset parameter amount. In the present invention, firstly, a rule screening method is used to determine in real time whether the vehicle-side surrounding environment data is data that has no practical use value for the iteration of the autonomous driving algorithm. Then, if so, a neural network screening method is used to further determine in real time whether the vehicle-side surrounding environment data is high-value data that helps the autonomous driving algorithm to iterate, thereby achieving the purpose of real-time screening. The screening process of all data is completed online on the vehicle-side computing chip of the target autonomous driving vehicle. There is no need to upload to the cloud for screening, which greatly reduces the transmission cost and storage cost. There is no need for manual labeling, which reduces the screening cost, and shortens the original multi-stage and long-cycle data screening process to near real-time, improving the timeliness of the data. Of course, implementing any product or method of the present invention does not necessarily require all of the advantages described above to be achieved at the same time.

[0060] The innovative features of the embodiments of the present invention include:

[0061] 1. First, the vehicle-side surrounding environment data is determined in real time through rule screening to determine whether it is data that has no practical use value for the iteration of the autonomous driving algorithm. Then, if it is, the vehicle-side surrounding environment data is further determined in real time through neural network screening to determine whether it is high-value data that helps the autonomous driving algorithm to iterate, thereby achieving the purpose of real-time screening. The screening process of all data is completed online on the vehicle-side computing chip of the target autonomous driving vehicle, without uploading to the cloud for screening, which greatly reduces the transmission cost and storage cost, and does not require manual labeling, reducing the screening cost, and shortening the original multi-stage and long-cycle data screening process to near real-time, improving the timeliness of data.

[0062] 2. Since real-time screening is performed in the embodiment of the present invention, only when the vehicle-side surrounding environment data is not dirty data but high-value data can it be screened and retained as the final screened data and uploaded to the cloud. The final screened data obtained by screening is a very small part. It can be seen that in the embodiment of the present invention, only the final screened data needs to be transmitted to the cloud, and there is no need to transmit all vehicle-side data to the cloud, so that data transmission and storage are reduced to less than 40%, greatly reducing transmission costs and storage costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative work.

[0064] Figure 1 A schematic diagram of a flow chart of a real-time vehicle-side data screening method provided by an embodiment of the present invention;

[0065] Figure 2 A schematic diagram of the structure of a real-time vehicle-side data screening device provided by an embodiment of the present invention;

[0066] Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention;

[0067] Figure 4 A schematic structural diagram of a vehicle provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0069] It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of the present invention and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0070] The embodiment of the present invention discloses a real-time vehicle-side data screening method, device, equipment, medium and vehicle, which can improve the timeliness of data and reduce the screening cost. The embodiment of the present invention is described in detail below.

[0071] Figure 1 A flow chart of a real-time vehicle-side data screening method provided by an embodiment of the present invention. The method is applied to a vehicle-side computing chip. The method specifically includes the following steps.

[0072] S110: Receive vehicle-side surrounding environment data collected in real time by a vehicle-side sensor of a target autonomous driving vehicle, wherein the vehicle-side surrounding environment data includes a target obscured vehicle.

[0073] In order to iterate the autonomous driving algorithm, the vehicle-side sensors of the target autonomous driving vehicle need to collect the vehicle-side surrounding environment data around the target autonomous driving vehicle in real time, and transmit the collected vehicle-side surrounding environment data to the vehicle-side computing chip in real time. The vehicle-side computing chip receives the vehicle-side surrounding environment data collected in real time by the vehicle-side sensors of the target autonomous driving vehicle.

[0074] Among them, the vehicle-side sensor can be a vehicle-side image sensor or a vehicle-side lidar sensor, and the vehicle-side surrounding environment data is data of the surrounding environment of the target autonomous driving vehicle collected by the vehicle-side sensor, and the vehicle-side surrounding environment data includes the target obscured vehicle.

[0075] S120: When the vehicle-side surrounding environment data meets the preset image screening rules, the vehicle-side surrounding environment data is used as initial screened data.

[0076] After receiving the vehicle-side surrounding environment data collected in real time by the vehicle-side sensors of the target autonomous driving vehicle, the vehicle-side computing chip performs real-time data screening on the vehicle-side surrounding environment data.

[0077] In an embodiment of the present invention, the vehicle-side surrounding environment data is doubly screened. First, dirty data is eliminated through rule screening, and then high-value data is selected through neural network screening. Dirty data refers to data that has no practical use value for the iteration of the autonomous driving algorithm, and high-value data refers to data that is beneficial to the iteration of the autonomous driving algorithm.

[0078] Since the embodiment of the present invention performs screening in real time, it is only necessary to determine whether the vehicle-side surrounding environment data is dirty data or high-value data. When the vehicle-side surrounding environment data meets the preset image screening rules, it means that the vehicle-side surrounding environment data is not dirty data and is retained as the initial screened data to continue post-screening.

[0079] Specifically, step S120 may include:

[0080] When the vehicle-side surrounding environment data is data with unblurred image edges and image grayscale values ​​within a preset grayscale value range, the vehicle-side surrounding environment data is used as initial filtered data.

[0081] Since distorted data with blurred image edges, data with too dark images, and data with too bright images have no practical use value for the iteration of the autonomous driving algorithm, it is necessary to determine whether the vehicle-side surrounding environment data is data with unblurred image edges and image grayscale values ​​within a preset grayscale value range. If so, the vehicle-side surrounding environment data is used as the initial screened data, where the preset grayscale value range can be determined according to actual needs.

[0082] Among them, the step of judging whether the vehicle-side surrounding environment data is undistorted data with unblurred image edges can be: performing a second-order derivative of the vehicle-side surrounding environment data to obtain the derived data, calculating the variance of the image edge of the derived data, and when the variance is not less than the preset variance, determining that the vehicle-side surrounding environment data is undistorted data with unblurred image edges.

[0083] Therefore, whether the vehicle-side surrounding environment data is dirty data is determined through rule screening. Only when the vehicle-side surrounding environment data is not dirty data, the vehicle-side surrounding environment data will be used as the initial screened data for subsequent screening. In other words, all data that can be retained are not dirty data. Therefore, distorted data, data with too dark images, and data with too bright images are eliminated.

[0084] S130: The initial filtered data are respectively detected according to the pre-established large neural network model and the small neural network model to obtain two detection results of the target obscured vehicle. When the difference between the two detection results is greater than a preset difference threshold, the initial filtered data is used as the final filtered data, wherein the large neural network model and the small neural network model are both used to associate the obscured vehicle sample data with the corresponding vehicle category and vehicle position. The large neural network model is a neural network model with a number of convolution layers greater than a preset number of layers or a parameter amount greater than a preset parameter amount, and the small neural network model is a neural network model with a number of convolution layers not greater than a preset number of layers or a parameter amount not greater than a preset parameter amount.

[0085] After the rule screening, the vehicle-side computing chip continues to perform neural network screening, that is, the initial screened data is detected according to the pre-established large neural network model and small neural network model to obtain two detection results of the target obscured vehicle. When the difference between the two detection results is greater than the preset difference threshold, the initial screened data is used as the final screened data.

[0086] Wherein, step S130 may include:

[0087] The initially screened data is detected according to a pre-established large neural network model to obtain the first position and the first category of the target blocked vehicle;

[0088] The data after the initial screening is detected according to a pre-established small neural network model to obtain a second position and a second category of the target blocked vehicle;

[0089] Performing loss calculation on the probability value of the first category and the probability value of the second category according to a preset loss function to obtain a loss value;

[0090] Calculate the sum of the squares of the distances between the first position and the second position;

[0091] When at least one of the loss value is greater than the preset loss value and the sum of squares is greater than the preset sum of squares, the initial screened data is used as the final screened data.

[0092] In an embodiment of the present invention, the neural network screening method uses a total of two neural network models, one is a large neural network model, and the other is a small neural network model. If the difference between the two detection results of the target obscured vehicle obtained after detecting the data after initial screening through these two neural network models is large, it means that the data after initial screening is relatively complex, which is helpful for the iteration of the autonomous driving algorithm and needs to be screened out.

[0093] Specifically, the data after initial screening is detected according to a pre-established large neural network model to obtain the first position and the first category of the target obscured vehicle, and the data after initial screening is detected according to a pre-established small neural network model to obtain the second position and the second category of the target obscured vehicle.

[0094] The large neural network model and the small neural network model have the same function, which is to associate the occluded vehicle sample data with the corresponding vehicle category and vehicle position. The only difference between the two is the scale of the model. The large neural network model is a neural network model with a convolution layer number greater than the preset number of layers or a parameter amount greater than the preset parameter amount, and the small neural network model is a neural network model with a convolution layer number not greater than the preset number of layers or a parameter amount not greater than the preset parameter amount.

[0095] If the model architectures of the large neural network model and the small neural network model are different, the number of convolution layers is different. If the model architectures of the large neural network model and the small neural network model are the same, the number of parameters is different. For example, the preset number of layers can be 15 and the preset number of parameters can be 450,000.

[0096] Exemplarily, the large neural network model may be an enlarged model corresponding to a lightweight convolutional neural network model or an open source visual neural network model.

[0097] Exemplarily, the open source visual neural network model can be a deep residual network Resnet-50 model, and the corresponding magnification model of the lightweight convolutional neural network model can be a 1.25-fold lightweight convolutional neural network model, a 1.5-fold lightweight convolutional neural network model, or a 2-fold lightweight convolutional neural network model. Specifically, the lightweight convolutional neural network model can be a vehicle-side surrounding environment perception model.

[0098] Among them, the training process of the large neural network model is:

[0099] Acquire vehicle surrounding environment sample data around the target autonomous driving vehicle, wherein the vehicle surrounding environment sample data includes blocked vehicle sample data and corresponding vehicle categories and vehicle positions;

[0100] The occluded vehicle sample data and the corresponding vehicle category and vehicle position are used as model training data to train the initial large neural network model. When the number of iterations reaches a preset number, the training is completed to obtain a large neural network model that associates the occluded vehicle sample data with the corresponding vehicle category and vehicle position.

[0101] Since the large-scale neural network model is established to predict the category and position of the target obscured vehicle in the surrounding environment data around the target autonomous driving vehicle, it is necessary to obtain the vehicle surrounding environment sample data around the target autonomous driving vehicle when establishing the large-scale neural network model, wherein the vehicle surrounding environment sample data includes the obscured vehicle sample data and the corresponding vehicle category and vehicle position.

[0102] It is understandable that first an initial large neural network model needs to be built, and then trained to obtain a large neural network model.

[0103] Specifically, after obtaining the sample data of the vehicle surrounding environment around the target autonomous driving vehicle, the occluded vehicle sample data and the corresponding vehicle category and vehicle position are used as model training data to train the initial large neural network model. When the number of iterations reaches a preset number, the training is completed to obtain a large neural network model that associates the occluded vehicle sample data with the corresponding vehicle category and vehicle position.

[0104] It can be seen that by training the initial large neural network model in the above training method, a large neural network model can be obtained for associating the occluded vehicle sample data with the corresponding vehicle category and vehicle position.

[0105] Since the large neural network model and the small neural network model have the same function, but the scale of the model is different, the training process of the small neural network model is the same as that of the large neural network model, and will not be repeated here.

[0106] Among them, the method for determining the enlarged model corresponding to the lightweight convolutional neural network model can be:

[0107] According to the preset model magnification method, the lightweight convolutional neural network model is magnified to a preset multiple to obtain an enlarged model corresponding to the lightweight convolutional neural network model.

[0108] In order to filter out data that is helpful for the iteration of the autonomous driving algorithm, after obtaining the first position, the first category, the second position and the second category, it is necessary to perform loss calculation on the probability value of the first category and the probability value of the second category according to a preset loss function to obtain the loss value, and calculate the sum of the squares of the distances between the first position and the second position.

[0109] Specifically, the preset loss function is:

[0110]

[0111] Among them, q is the loss value, z i is the probability value of the first category, z jis the probability value of the second category, i is the first category, j is the second category, and T is the temperature coefficient.

[0112] Where T is the temperature coefficient. When T=1, the above formula is the standard Softmax function formula. The higher T is, the smoother the output probability distribution of the above formula is, the greater the entropy of the loss value distribution is, the information carried by the negative label will be relatively amplified, and the model training will pay more attention to the negative label, that is, pay more attention to the more complex image data. For example, T=0.5.

[0113] If the calculated loss value is large, it means that the first category is quite different from the second category. If the calculated sum of squares is large, it means that the first position is quite different from the second position. Whether the loss value is large or the sum of squares is large, it means that the initial screened data is relatively complex, which is helpful for the iteration of the autonomous driving algorithm and needs to be screened out. Therefore, when at least one of the loss value is greater than the preset loss value and the sum of squares is greater than the preset sum of squares, the initial screened data is used as the final screened data. For example, the preset loss value can be 0.7 and the preset sum of squares can be 11.

[0114] Therefore, the initial filtered data are detected by a large neural network model and a small neural network model respectively to obtain two detection results of the target obscured vehicle, and whether the difference between the two detection results is large is determined by calculating the loss value and the sum of squares of the distance based on the two detection results. When at least one of the loss value is greater than the preset loss value and the sum of squares is greater than the preset sum of squares, it means that the difference between the two detection results is large, that is, the difference between the two detection results is greater than the preset difference threshold, indicating that the initial filtered data is helpful for the iteration of the autonomous driving algorithm. The initial filtered data is used as the final filtered data, thereby filtering out the vehicle-side data that is truly helpful for the iteration of the autonomous driving algorithm.

[0115] It can be seen from the above content that the embodiments of the present invention can receive vehicle-side surrounding environment data collected in real time by the vehicle-side sensor of the target autonomous driving vehicle, wherein the vehicle-side surrounding environment data includes the target obscured vehicle; when the vehicle-side surrounding environment data meets the preset image screening rules, the vehicle-side surrounding environment data is used as the initial screened data; the initial screened data is respectively detected according to the pre-established large neural network model and small neural network model to obtain two detection results of the target obscured vehicle, and when the difference between the two detection results is greater than the preset difference threshold, the initial screened data is used as the final screened data; wherein the large neural network model and the small neural network model are both used to associate the obscured vehicle sample data with the corresponding vehicle category and vehicle position, the large neural network model is a neural network model with a number of convolution layers greater than the preset number of layers or a parameter amount greater than the preset parameter amount, and the small neural network model is a neural network model with a number of convolution layers not greater than the preset number of layers or a parameter amount not greater than the preset parameter amount. In the present invention, firstly, a rule screening method is used to determine in real time whether the vehicle-side surrounding environment data is data that has no practical use value for the iteration of the autonomous driving algorithm. Then, if yes, a neural network screening method is used to further determine in real time whether the vehicle-side surrounding environment data is high-value data that helps the autonomous driving algorithm to iterate, thereby achieving the purpose of real-time screening. The screening process of all data is completed online on the vehicle-side computing chip of the target autonomous driving vehicle, without uploading to the cloud for screening, which greatly reduces the transmission cost and storage cost, and does not require manual labeling, which reduces the screening cost, and shortens the original multi-stage and long-cycle data screening process to near real time, improving the timeliness of data.

[0116] After step S130, the real-time vehicle-side data screening method provided by the embodiment of the present invention may further include:

[0117] S140: Upload the final screened data to the cloud.

[0118] In order to facilitate the subsequent iteration of the autonomous driving algorithm, after obtaining the final filtered data, it is necessary to upload the final filtered data to the cloud.

[0119] Since real-time screening is performed in the embodiment of the present invention, only when the vehicle-side surrounding environment data is not dirty data but high-value data can it be screened and retained as the final screened data and uploaded to the cloud. The final screened data obtained by screening is a very small part. It can be seen that in the embodiment of the present invention, only the final screened data needs to be transmitted to the cloud, and there is no need to transmit all vehicle-side data to the cloud, so that data transmission and storage are reduced to less than 40%, greatly reducing transmission costs and storage costs.

[0120] Figure 2 A real-time vehicle-side data screening device provided by an embodiment of the present invention, see Figure 2 , the device is applied to a vehicle-side computing chip, and the device may include:

[0121] The receiving module 210 is used to receive vehicle-side surrounding environment data collected in real time by a vehicle-side sensor of a target autonomous driving vehicle, wherein the vehicle-side surrounding environment data includes a target obscured vehicle;

[0122] A rule screening module 220, configured to use the vehicle-side surrounding environment data as initial screened data when the vehicle-side surrounding environment data meets a preset image screening rule;

[0123] Neural network screening module 230, for

[0124] The initial filtered data are respectively detected according to a pre-established large neural network model and a small neural network model to obtain two detection results of the target blocked vehicle, and when the difference between the two detection results is greater than a preset difference threshold, the initial filtered data is used as the final filtered data;

[0125] Among them, both the large neural network model and the small neural network model are used to associate the obscured vehicle sample data with the corresponding vehicle category and vehicle position. The large neural network model is a neural network model with a number of convolution layers greater than a preset number of layers or a parameter amount greater than a preset parameter amount, and the small neural network model is a neural network model with a number of convolution layers not greater than the preset number of layers or a parameter amount not greater than the preset parameter amount.

[0126] The device provided by the embodiment of the present invention can receive vehicle-side surrounding environment data collected in real time by the vehicle-side sensor of the target autonomous driving vehicle, wherein the vehicle-side surrounding environment data includes the target obscured vehicle; when the vehicle-side surrounding environment data meets the preset image screening rules, the vehicle-side surrounding environment data is used as the initial screened data; the initial screened data is respectively detected according to the pre-established large neural network model and small neural network model to obtain two detection results of the target obscured vehicle, and when the difference between the two detection results is greater than the preset difference threshold, the initial screened data is used as the final screened data; wherein the large neural network model and the small neural network model are both used to associate the obscured vehicle sample data with the corresponding vehicle category and vehicle position, the large neural network model is a neural network model with a convolution layer number greater than the preset number of layers or a parameter amount greater than the preset parameter amount, and the small neural network model is a neural network model with a convolution layer number not greater than the preset number of layers or a parameter amount not greater than the preset parameter amount. In the present invention, firstly, a rule screening method is used to determine in real time whether the vehicle-side surrounding environment data is data that has no practical use value for the iteration of the autonomous driving algorithm. Then, if yes, a neural network screening method is used to further determine in real time whether the vehicle-side surrounding environment data is high-value data that helps the autonomous driving algorithm to iterate, thereby achieving the purpose of real-time screening. The screening process of all data is completed online on the vehicle-side computing chip of the target autonomous driving vehicle, without uploading to the cloud for screening, which greatly reduces the transmission cost and storage cost, and does not require manual labeling, which reduces the screening cost, and shortens the original multi-stage and long-cycle data screening process to near real time, improving the timeliness of data.

[0127] In one implementation, the real-time vehicle-side data screening device further includes:

[0128] The uploading module is used to upload the final screened data to the cloud after taking the initial screened data as the final screened data.

[0129] In one implementation, the rule screening module 220 may be specifically used to:

[0130] When the vehicle-side surrounding environment data is data whose image edges are not blurred and whose image grayscale values ​​are within a preset grayscale value range, the vehicle-side surrounding environment data is used as initial filtered data.

[0131] In one implementation, the neural network screening module 230 may include:

[0132] A first detection submodule, configured to detect the initially screened data according to a pre-established large neural network model to obtain a first position and a first category of the target obscured vehicle;

[0133] A second detection submodule, configured to detect the initially screened data according to a pre-established small neural network model to obtain a second position and a second category of the target obscured vehicle;

[0134] A loss degree calculation submodule, used to perform loss degree calculation on the probability value of the first category and the probability value of the second category according to a preset loss degree function to obtain a loss degree value;

[0135] a square sum calculation submodule, used to calculate the square sum of the distance between the first position and the second position;

[0136] The final screening submodule is used to use the initial screened data as the final screened data when at least one of the loss value is greater than a preset loss value and the sum of squares is greater than a preset sum of squares is established.

[0137] In one implementation, the device further includes a model training module, which is used to train the large neural network model. The model training module may include:

[0138] An acquisition submodule, configured to acquire vehicle surrounding environment sample data around the target autonomous driving vehicle, wherein the vehicle surrounding environment sample data includes obscured vehicle sample data and corresponding vehicle categories and vehicle positions;

[0139] The training submodule is used to use the obscured vehicle sample data and the corresponding vehicle category and vehicle position as model training data to train the initial large neural network model. When the number of iterations reaches a preset number, the training is completed to obtain a large neural network model that associates the obscured vehicle sample data with the corresponding vehicle category and vehicle position.

[0140] In one implementation, the large neural network model may be an enlarged model corresponding to a lightweight convolutional neural network model or an open source visual neural network model.

[0141] In one implementation, the device may further include a model determination module, the model determination module being used to determine an amplified model corresponding to the lightweight convolutional neural network model, the model determination module being specifically used to:

[0142] According to a preset model magnification method, the lightweight convolutional neural network model is magnified to a preset multiple to obtain an enlarged model corresponding to the lightweight convolutional neural network model.

[0143] In one implementation, the open source visual neural network model may be a deep residual network Resnet-50 model.

[0144] The above device embodiment corresponds to the method embodiment and has the same technical effect as the method embodiment. For specific description, please refer to the method embodiment. The device embodiment is obtained based on the method embodiment. For specific description, please refer to the method embodiment part, which will not be repeated here.

[0145] Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention is shown in FIG. Figure 3 As shown, the electronic device includes:

[0146] One or more processors 310;

[0147] The processor 310 is coupled to a storage device 320, and the storage device 320 is used to store one or more programs;

[0148] When the one or more programs are executed by the one or more processors 310, the electronic device implements the real-time vehicle-side data screening method provided by any embodiment of the present invention.

[0149] Based on the above embodiments, another embodiment of the present invention provides a vehicle, wherein the vehicle includes a real-time vehicle-side data screening device as provided in any embodiment of the present invention, or includes an electronic device as provided in any embodiment of the present invention.

[0150] Figure 4 A schematic diagram of a vehicle provided by an embodiment of the present invention. Figure 4 As shown, the vehicle includes a speed sensor 41, an ECU (Electronic Control Unit) 42, a GPS (Global Positioning System) positioning device 43, and a T-Box (Telematics Box) 44. The speed sensor 41 is used to measure the vehicle speed and use the vehicle speed as an empirical speed for model training; the GPS positioning device 43 is used to obtain the current geographic location of the vehicle; the T-Box 64 can communicate with the server as a gateway; and the ECU 42 can execute the above-mentioned real-time vehicle-side data screening method.

[0151] In addition, the vehicle may also include: a V2X (Vehicle-to-Everything) module 45, a radar 46 and a camera 47. The V2X module 45 is used to communicate with other vehicles, roadside equipment, etc.; the radar 46 or the camera 47 is used to sense the road environment information in the front and / or other directions to obtain original point cloud data; the radar 46 and / or the camera 47 can be configured at the front and / or rear of the vehicle body.

[0152] Based on the above method embodiments, another embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a real-time vehicle-side data screening method as provided in any embodiment of the present invention.

[0153] Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0154] Those skilled in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description of the embodiment, or can be changed accordingly and located in one or more devices different from the embodiment. The modules in the above embodiment can be combined into one module, or can be further divided into multiple sub-modules.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A real-time vehicle-side data screening method, characterized in that: The method is applied to a vehicle-side computing chip, and the method includes: Receiving vehicle-side surrounding environment data collected in real time by a vehicle-side sensor of a target autonomous driving vehicle, wherein the vehicle-side surrounding environment data includes a target obscured vehicle; When the vehicle-side surrounding environment data meets the preset image screening rule, the vehicle-side surrounding environment data is used as initial screened data; The initial filtered data are respectively detected according to a pre-established large neural network model and a small neural network model to obtain two detection results of the target blocked vehicle, and when the difference between the two detection results is greater than a preset difference threshold, the initial filtered data is used as the final filtered data; Among them, both the large neural network model and the small neural network model are used to associate the obscured vehicle sample data with the corresponding vehicle category and vehicle position. The large neural network model is a neural network model with a number of convolution layers greater than a preset number of layers or a parameter amount greater than a preset parameter amount, and the small neural network model is a neural network model with a number of convolution layers not greater than the preset number of layers or a parameter amount not greater than the preset parameter amount.

2. The method according to claim 1, characterized in that After the step of using the initial screened data as final screened data, the method further comprises: The final screened data is uploaded to the cloud.

3. The method according to claim 1, characterized in that When the vehicle-side surrounding environment data meets the preset image screening rule, the step of using the vehicle-side surrounding environment data as initial screened data includes: When the vehicle-side surrounding environment data is data whose image edges are not blurred and whose image grayscale values ​​are within a preset grayscale value range, the vehicle-side surrounding environment data is used as initial filtered data.

4. The method according to claim 1, characterized in that The step of detecting the initial filtered data according to the pre-established large neural network model and the small neural network model to obtain two detection results of the target blocked vehicle, and when the difference between the two detection results is greater than a preset difference threshold, using the initial filtered data as the final filtered data comprises: Detecting the initially screened data according to a pre-established large neural network model to obtain a first position and a first category of the target obscured vehicle; Detecting the initially filtered data according to a pre-established small neural network model to obtain a second position and a second category of the target obscured vehicle; Performing loss calculation on the probability value of the first category and the probability value of the second category according to a preset loss function to obtain a loss value; calculating the sum of squares of distances between the first position and the second position; When at least one of the loss value is greater than a preset loss value and the sum of squares is greater than a preset sum of squares, the initial screened data is used as the final screened data.

5. The method according to claim 4, characterized in that The training process of the large neural network model is: Acquire vehicle surrounding environment sample data around the target autonomous driving vehicle, wherein the vehicle surrounding environment sample data includes blocked vehicle sample data and corresponding vehicle categories and vehicle positions; The obscured vehicle sample data and the corresponding vehicle category and vehicle position are used as model training data to train the initial large neural network model. When the number of iterations reaches a preset number, the training is completed to obtain a large neural network model that associates the obscured vehicle sample data with the corresponding vehicle category and vehicle position.

6. The method according to claim 4, characterized in that The large neural network model is an enlarged model corresponding to the lightweight convolutional neural network model or an open source visual neural network model.

7. The method according to claim 6, characterized in that The method for determining the enlarged model corresponding to the lightweight convolutional neural network model is as follows: According to a preset model magnification method, the lightweight convolutional neural network model is magnified to a preset multiple to obtain an enlarged model corresponding to the lightweight convolutional neural network model.

8. The method according to claim 6, characterized in that The open source visual neural network model is a deep residual network Resnet-50 model.

9. A real-time vehicle-side data screening device, characterized in that: The device is applied to a vehicle-side computing chip, and the device includes: A receiving module, used to receive vehicle-side surrounding environment data collected in real time by a vehicle-side sensor of a target autonomous driving vehicle, wherein the vehicle-side surrounding environment data includes a target obscured vehicle; A rule screening module, used for using the vehicle-side surrounding environment data as initial screened data when the vehicle-side surrounding environment data meets the preset image screening rules; A neural network screening module is used to detect the initial screened data according to a pre-established large neural network model and a pre-established small neural network model to obtain two detection results of the target blocked vehicle, and when the difference between the two detection results is greater than a preset difference threshold, the initial screened data is used as the final screened data; Among them, both the large neural network model and the small neural network model are used to associate the obscured vehicle sample data with the corresponding vehicle category and vehicle position. The large neural network model is a neural network model with a number of convolution layers greater than a preset number of layers or a parameter amount greater than a preset parameter amount, and the small neural network model is a neural network model with a number of convolution layers not greater than the preset number of layers or a parameter amount not greater than the preset parameter amount.

10. The device according to claim 9, characterized in that The device also includes: The uploading module is used to upload the final screened data to the cloud after taking the initial screened data as the final screened data.

11. The device according to claim 9, characterized in that The rule screening module is specifically used for: When the vehicle-side surrounding environment data is data whose image edges are not blurred and whose image grayscale values ​​are within a preset grayscale value range, the vehicle-side surrounding environment data is used as initial filtered data.

12. The device according to claim 9, characterized in that The neural network screening module comprises: A first detection submodule, configured to detect the initially screened data according to a pre-established large neural network model to obtain a first position and a first category of the target obscured vehicle; A second detection submodule, configured to detect the initially screened data according to a pre-established small neural network model to obtain a second position and a second category of the target obscured vehicle; A loss degree calculation submodule, used to perform loss degree calculation on the probability value of the first category and the probability value of the second category according to a preset loss degree function to obtain a loss degree value; a square sum calculation submodule, used to calculate the square sum of the distance between the first position and the second position; The final screening submodule is used to use the initial screened data as the final screened data when at least one of the loss value is greater than a preset loss value and the sum of squares is greater than a preset sum of squares is established.

13. The device according to claim 12, characterized in that The device also includes a model training module, which is used to train the large neural network model, and the model training module includes: An acquisition submodule, configured to acquire vehicle surrounding environment sample data around the target autonomous driving vehicle, wherein the vehicle surrounding environment sample data includes blocked vehicle sample data and corresponding vehicle categories and vehicle positions; The training submodule is used to use the obscured vehicle sample data and the corresponding vehicle category and vehicle position as model training data to train the initial large neural network model. When the number of iterations reaches a preset number, the training is completed to obtain a large neural network model that associates the obscured vehicle sample data with the corresponding vehicle category and vehicle position.

14. The device according to claim 12, characterized in that The large neural network model is an enlarged model corresponding to the lightweight convolutional neural network model or an open source visual neural network model.

15. The device according to claim 14, characterized in that The device further includes a model determination module, which is used to determine an amplified model corresponding to the lightweight convolutional neural network model, and the model determination module is specifically used to: According to a preset model magnification method, the lightweight convolutional neural network model is magnified to a preset multiple to obtain an enlarged model corresponding to the lightweight convolutional neural network model.

16. The device according to claim 14, characterized in that The open source visual neural network model is a deep residual network Resnet-50 model.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

18. An electronic device, characterized in that: The electronic device comprises: one or more processors; The processor is coupled to a storage device, and the storage device is used to store one or more programs; When the one or more programs are executed by the one or more processors, the electronic device implements the method as described in any one of claims 1 to 8.

19. A vehicle, characterized in that: The vehicle comprises the apparatus as claimed in any one of claims 9 to 16, or comprises the electronic device as claimed in claim 18.

Images