Data generation method, system, server and storage medium

Abstract

The present application relates to the field of autonomous driving technology, and specifically provides a data generation method, system, server and storage medium, aiming to solve the problem of improving the efficiency and quality of data acquisition. To this end, the method provided by the present application includes obtaining the operating data of a first autonomous driving model on an intelligent device, the first autonomous driving model is used to determine a first autonomous driving decision based on environmental data of the environment in which the intelligent device is located, and the operating data includes environmental data; according to the environmental data in the operating data, the first attribute information of the target in the environment in which the intelligent device is located is obtained, and the target is labeled according to the first attribute information to obtain the labeling information of the target; according to the environmental data and the labeling information of the target, environmental labeling data is generated, and the environmental labeling data is used to train the first autonomous driving model. The above method can significantly improve the efficiency and quality of obtaining environmental labeling data, which is conducive to improving the efficiency and performance of iterative upgrades of the first autonomous driving model.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and specifically to a data generation method, system, server and storage medium. Background Art

[0002] When autonomous driving is performed on intelligent devices such as vehicles, autonomous driving models are usually used to achieve autonomous driving control. Currently, autonomous driving models are mainly based on deep learning methods. For deep learning methods, data, algorithms, and computing power are key elements, among which data is an important driver of deep learning methods. In the field of autonomous driving, a large amount of data in different driving scenarios is needed to train autonomous driving models. The quality and quantity of data determine the performance of autonomous driving models. Therefore, it is crucial to obtain large quantities of high-quality data.

[0003] However, the current conventional data acquisition method is mainly for professionals to go to different driving scenarios to obtain data, which is equivalent to a manual acquisition method. This method is very inefficient and cannot cover more driving scenarios. Based on this, using the data obtained by this method to train the autonomous driving model will seriously affect the efficiency and performance improvement of the iterative upgrade of the autonomous driving model.

[0004] Accordingly, the art needs a new technical solution to solve the above problems. Summary of the invention

[0005] In order to overcome the above-mentioned defects, the present application is proposed to solve or at least partially solve the following technical problems: improve the efficiency of acquiring autonomous driving model training data and the data quality, so as to improve the efficiency and performance of iterative upgrades of autonomous driving models.

[0006] In a first aspect, a data generation method is provided, the method comprising:

[0007] Acquire operation data of a first autonomous driving model on a smart device, where a sensor is provided on the smart device, and the sensor is used to collect environmental data of an environment where the smart device is located. The first autonomous driving model is used to determine a first autonomous driving decision according to the environmental data when the smart device enters an intelligent driving mode, and the operation data includes the environmental data.

[0008] Acquire first attribute information of a target in the environment where the smart device is located according to the environment data in the operation data, and annotate the target according to the first attribute information to obtain annotated information of the target, where the annotated information is used to indicate the first attribute information;

[0009] Generate environment annotation data according to the environment data and the annotation information of the target, wherein the environment annotation data is used to train the first autonomous driving model;

[0010] in,

[0011] A second autonomous driving model is provided on the smart device, and the second autonomous driving model is used to determine a second autonomous driving decision according to the environmental data and control the smart device to execute the second autonomous driving decision when the smart device enters the intelligent driving mode.

[0012] In a technical solution of the above data generation method, the target is an obstacle, and the first attribute information of the target includes a latest braking time based on a safety target;

[0013] The latest braking time is the latest time when the smart device starts braking, and the safety goal is to prevent the smart device from colliding with the target.

[0014] In a technical solution of the above data generation method, the operation data includes the first autonomous driving decision, the first autonomous driving decision includes a future trajectory of the smart device, and obtaining first attribute information of a target in the environment where the smart device is located according to the environmental data in the operation data includes:

[0015] predicting a target trajectory of a target in the environment based on the environmental data;

[0016] Based on the safety goal and according to the future trajectory of the smart device and the target trajectory, determining the latest braking time;

[0017] According to the latest braking time, first attribute information of the target is acquired.

[0018] In a technical solution of the above data generation method, the method further includes:

[0019] Filtering a specific target according to the target's annotation information, where the specific target is a target whose first attribute information meets a preset filtering condition;

[0020] In response to a user operation, modify the annotation information of the specific target, and generate environmental annotation data according to the modified annotation information and the environmental data;

[0021] The modified annotation information is used to indicate the second attribute information of the specific target.

[0022] In a technical solution of the above data generation method, the target is a target, the first attribute information and the second attribute information are both the latest braking time based on a safety target, the latest braking time is the latest time when the smart device starts braking, and the safety target is to prevent the smart device from colliding with the target;

[0023] The preset screening condition is that the latest braking time is less than a preset time threshold.

[0024] In a technical solution of the above data generation method, generating environmental annotation data according to the environmental data and the annotation information of the target includes:

[0025] Using a first perception model to perform target perception on the environmental data to obtain a detection frame of the target;

[0026] Generate environmental annotation data according to the environmental data, the detection frame and the annotation information;

[0027] in,

[0028] The first perception model is the same as an actual perception model set on the smart device, and the actual perception model is used to perform target perception on the environmental data collected by the sensor to obtain first perception data;

[0029] The first autonomous driving model and the second autonomous driving model are respectively used to determine the first autonomous driving decision and the second autonomous driving decision based on the first perception data.

[0030] In a technical solution of the above data generation method, obtaining first attribute information of a target in the environment where the smart device is located according to the environment data in the operation data includes:

[0031] Using a second perception model to perform target perception on the environmental data in the operation data to obtain second perception data;

[0032] Acquire first attribute information of a target in the environment according to the second perception data;

[0033] in,

[0034] The perceptual accuracy of the second perceptual model is higher than the perceptual accuracy of the first perceptual model.

[0035] In one technical solution of the above-mentioned data generation method, the first autonomous driving model and / or the second autonomous driving model is an end-to-end deep learning model.

[0036] In a second aspect, a data generation system is provided, the system comprising:

[0037] a data acquisition module, configured to acquire operation data of a first autonomous driving model on a smart device, wherein the smart device is provided with a sensor, and the sensor is used to collect environmental data of an environment where the smart device is located, and the first autonomous driving model is used to determine a first autonomous driving decision according to the environmental data when the smart device enters an intelligent driving mode, and the operation data includes the environmental data;

[0038] a first data annotation module, configured to obtain first attribute information of a target in the environment where the smart device is located according to the environment data in the operation data, and annotate the target according to the first attribute information to obtain annotation information of the target, wherein the annotation information is used to indicate the first attribute information;

[0039] a data generation module, configured to generate environment annotation data according to the environment data and the annotation information of the target, wherein the environment annotation data is used to train the first autonomous driving model;

[0040] in,

[0041] A second autonomous driving model is provided on the smart device, and the second autonomous driving model is used to determine a second autonomous driving decision according to the environmental data and control the smart device to execute the second autonomous driving decision when the smart device enters the intelligent driving mode.

[0042] In a technical solution of the above data generation system, the system includes a second data annotation module, and the second data annotation module is configured to:

[0043] Filtering a specific target according to the target's annotation information, where the specific target is a target whose first attribute information meets a preset filtering condition;

[0044] In response to a user operation, modify the annotation information of the specific target, and generate environmental annotation data according to the modified annotation information and the environmental data;

[0045] The modified annotation information is used to indicate the second attribute information of the specific target.

[0046] In a third aspect, a server is provided, which includes at least one processor; and a memory communicatively connected to the at least one processor; wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method described in any one of the technical solutions of the above-mentioned data generation method is implemented.

[0047] In a fourth aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored in the computer-readable storage medium, wherein the program codes are suitable for being loaded and run by a processor to execute the method described in any one of the technical solutions of the above-mentioned data generation method.

[0048] The above one or more technical solutions of the present application have at least one or more of the following beneficial effects:

[0049] In a technical solution for implementing the data generation method provided in the present application, the operation data of the first autonomous driving model on the smart device can be obtained, the smart device is provided with a sensor, the sensor is used to collect environmental data of the environment where the smart device is located, the first autonomous driving model is used to determine the first autonomous driving decision according to the environmental data when the smart device enters the intelligent driving mode, and the operation data includes environmental data; according to the environmental data in the operation data, the first attribute information of the target in the environment where the smart device is located is obtained, and the target is labeled according to the first attribute information to obtain the labeling information of the target, and the labeling information is used to indicate the first attribute information; according to the environmental data and the labeling information of the target, the environmental labeling data is generated, and the environmental labeling data is used to train the first autonomous driving model; wherein, the smart device is provided with a second autonomous driving model, and the second autonomous driving model is used to determine the second autonomous driving decision according to the environmental data when the smart device enters the intelligent driving mode and control the smart device to execute the second autonomous driving decision. That is to say, although the first and second autonomous driving models are set on the smart device at the same time, the smart device executes the second autonomous driving decision when entering the intelligent driving mode, and does not execute the first autonomous driving decision.

[0050] In the above method, the environmental annotation data is obtained using the operating data of the first autonomous driving model, and the environmental annotation data is used to train the first autonomous driving model, which is equivalent to realizing a closed-loop link of data acquisition, data generation, and data use. Based on this closed-loop link, as long as the smart device enters the intelligent driving mode, the environmental annotation data can be obtained quickly, accurately, and automatically, without the need for professionals to go to different driving scenes to obtain environmental annotation data. In addition, the above method has no restrictions on driving scenes. As long as the smart device can enter the scene of the intelligent driving mode, the environmental annotation data can be obtained. Therefore, the above method can significantly improve the efficiency and quality of obtaining environmental annotation data, which is conducive to improving the efficiency and performance of iterative upgrades of the first autonomous driving model.

[0051] In a technical solution for implementing the data generation system provided by the present application, the data generation system includes a data acquisition module, a first data annotation module, and a data generation module. The data acquisition module can be configured to acquire the operation data of the first autonomous driving model on the smart device, the smart device is provided with a sensor, the sensor is used to collect environmental data of the environment where the smart device is located, the first autonomous driving model is used to determine the first autonomous driving decision according to the environmental data when the smart device enters the intelligent driving mode, and the operation data includes environmental data; the first data annotation module can be configured to acquire the first attribute information of the target in the environment where the smart device is located according to the environmental data in the operation data, and annotate the target according to the first attribute information to obtain the annotation information of the target, and the annotation information is used to indicate the first attribute information; the data generation module can be configured to generate environmental annotation data according to the environmental data and the annotation information of the target, and the environmental annotation data is used to train the first autonomous driving model; wherein a second autonomous driving model is provided on the smart device, and the second autonomous driving model is used to determine the second autonomous driving decision according to the environmental data when the smart device enters the intelligent driving mode and control the smart device to execute the second autonomous driving decision.

[0052] Similar to the aforementioned data generation method, the above system implementation scheme can also significantly improve the efficiency and quality of obtaining environmental annotation data, which is conducive to improving the efficiency and performance of iterative upgrades of the first autonomous driving model. In addition, based on modular design, the fault tolerance of the above system can be improved. When any module fails, it can be replaced with a normal module in time. The flexibility of the above system is also improved based on modular design. Any module in the system can be split out separately and added to other systems for use. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The disclosure of the present application will become easier to understand with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present application. Among them:

[0054] Figure 1 is a flowchart of the main steps of a data generation method according to an embodiment of the present application;

[0055] Figure 2 This is a schematic diagram of the main steps of obtaining the first attribute information of the target according to an embodiment of the present application;

[0056] Figure 3 is a schematic flow chart of the main steps of generating environment annotation data according to an embodiment of the present application;

[0057] Figure 4 It is a schematic flow chart of the main steps of modifying the annotation information and generating the environment annotation data according to an embodiment of the present application;

[0058] Figure 5 is a flowchart of main steps of a data generation method according to another embodiment of the present application;

[0059] Figure 6 is a main structural block diagram of a data system method according to an embodiment of the present application;

[0060] Figure 7 is a main structural block diagram of a data system method according to another embodiment of the present application;

[0061] Figure 8 It is a schematic diagram of the main structure of a server according to an embodiment of the present application.

[0062] Reference numerals:

[0063] 11: data acquisition module; 12: first data labeling module; 13: data generation module; 14: second data labeling module; 21: memory; 22: processor. DETAILED DESCRIPTION

[0064] Some embodiments of the present application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and are not intended to limit the protection scope of the present application.

[0065] In the description of this application, "module" and "processor" may include hardware, software or a combination of the two. A module may include hardware circuits, various suitable sensors, communication ports, and memories, and may also include software parts, such as program codes, or a combination of software and hardware. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, hardware, or a combination of the two. Computer-readable storage media include any suitable media that can store program codes, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, etc.

[0066] The relevant user personal information that may be involved in the various embodiments of this application is strictly in accordance with the requirements of laws and regulations, following the principles of legality, legitimacy and necessity, based on the reasonable purposes of business scenarios, to process the personal information that users actively provide during the use of products / services or generated due to the use of products / services, as well as the personal information obtained with the user's authorization.

[0067] The user personal information processed by this application will vary depending on the specific product / service scenario, and shall be based on the specific scenario in which the user uses the product / service, and may involve the user's account information, device information, driving information, vehicle information or other related information. This application will treat the user's personal information and its processing with a high degree of diligence.

[0068] This application attaches great importance to the security of user personal information and has taken reasonable and feasible security protection measures that meet industry standards to protect user information and prevent personal information from being accessed, disclosed, used, modified, damaged or lost without authorization.

[0069] The following is a description of an embodiment of the data generation method provided in the present application.

[0070] See attached Figure 1 , Figure 1 FIG. 1 is a flow chart of the main steps of a data generation method according to an embodiment of the present application. Figure 1 As shown, the data generation method in the embodiment of the present application mainly includes the following steps S101 to S104.

[0071] Step S101: Obtain operating data of the first autonomous driving model on the smart device.

[0072] Smart devices may include driving devices, smart cars, robots and other devices.

[0073] Smart devices may be equipped with sensors such as cameras and radars (such as lidar), which can be used to collect environmental data of the environment in which the smart devices are located. For example, a camera can be used to collect images of the environment in which the smart devices are located, and a radar can be used to collect point clouds of the environment.

[0074] The first autonomous driving model can be used to determine a first autonomous driving decision based on environmental data collected by the sensor when the smart device enters the smart driving mode, and the smart device can be guided to drive automatically or assist in driving based on the first autonomous driving decision. The operation data of the first autonomous driving model can be data used and / or generated during its operation. In this embodiment, the operation data can include the environmental data collected by the above-mentioned sensor.

[0075] A second autonomous driving model may be provided on the smart device. The second autonomous driving model is used to determine a second autonomous driving decision based on environmental data when the smart device enters the intelligent driving mode. Based on the second autonomous driving decision, the smart device can also be guided to drive automatically or assist in driving. Unlike the first autonomous driving model, when the smart device enters the intelligent driving mode, the second autonomous driving model can control the smart device to execute the second autonomous driving decision, so that the smart device drives automatically or assists in driving according to the second autonomous driving decision. Although the first autonomous driving model also has the function of controlling the smart device to execute the first autonomous driving decision, in this embodiment, the first autonomous driving model does not start this function. The first autonomous driving model can be understood as a companion model of the second autonomous driving model.

[0076] In addition, a perception model may be provided on the smart device, and the perception model may be used to perform target perception on the environmental data collected by the sensor to obtain the perception data of the target, and the perception data of the target may include the detection frame of the target, etc. The target may include obstacles, lane lines, traffic signs, etc. For example, the detection frame of the obstacle in the image may be perceived by the perception model. The first and second autonomous driving models may use the perception data of the target obtained by the perception model to determine the first and second autonomous driving decisions. In some embodiments, the first autonomous driving model may be an end-to-end deep learning model, which may directly obtain the first autonomous driving decision based on the input perception data of the target (such as the detection frame of the obstacle), without the need to process the perception data of the target through other models or modules to obtain an intermediate result, and then use this intermediate result to obtain the first autonomous driving decision. In addition, in some embodiments, the second autonomous driving model may also be an end-to-end deep learning model, which may also directly obtain the second autonomous driving decision based on the input perception data of the target.

[0077] Step S102: Acquire first attribute information of a target in the environment where the smart device is located according to the environment data in the operation data.

[0078] The attribute information of a target may include the category, location, etc. of the target.

[0079] Specifically, the target perception data can be obtained by performing target perception on the environmental data, and the attribute information of the target can be obtained according to the target perception data. For example, the target perception data can include the detection frame of the target, and the attribute of the target can be obtained according to the detection frame; the target perception data can include the category of the target, and the category can be directly obtained according to the target perception data.

[0080] Step S103: labeling the target according to the first attribute information to obtain labeling information of the target, wherein the labeling information is used to indicate the first attribute information of the target.

[0081] Step S104: Generate environment annotation data according to the environment data and the annotation information of the target, wherein the environment annotation data is used to train the first automatic driving model.

[0082] Specifically, the environmental data and the labeling information of the target in the environmental data can be combined together to obtain the environmental labeling data. When using the environmental labeling data to train the first autonomous driving model, a part of the environmental labeling data can be used as training data, and another part of the environmental labeling data can be used as test data. The first autonomous driving model is trained with the training data, and the first autonomous driving model is tested with the test data after the training is completed. In some embodiments, before training the first autonomous driving model, the data format of the environmental labeling data can also be converted into a data format that is conducive to improving the efficiency of model training, and then the environmental labeling data with the converted data format can be used for training. In addition, the environmental labeling data can also be compressed to reduce the storage space or storage cost of the environmental labeling data.

[0083] The methods described in steps S101 to S104 above can be applied to the server. In some embodiments, the method of training the first autonomous driving data using environmental annotation data can also be applied to the server. Specifically, an autonomous driving model that is exactly the same as the first autonomous driving model can be set up on the server. Using environmental annotation data to train the autonomous driving model on the server is equivalent to upgrading the autonomous driving model, and then replacing the first autonomous driving model on the smart device with the autonomous driving model that has been trained or upgraded on the server is equivalent to achieving the training or upgrading of the first autonomous driving model on the smart device. After completing one training or upgrade, the environmental annotation data can continue to be generated by the methods described in steps S101 to S104 above, and the first autonomous driving model can be trained using the environmental annotation data, that is, the first autonomous driving model can be continuously iteratively trained or iteratively upgraded.

[0084] In the method described in steps S101 to S104 above, the environmental annotation data is obtained using the operating data of the first autonomous driving model, and the environmental annotation data is used to train the first autonomous driving model, which is equivalent to realizing a closed-loop link of data acquisition, data generation, and data use. Based on this closed-loop link, as long as the smart device enters the intelligent driving mode, the environmental annotation data can be obtained quickly, accurately, and automatically. In addition, the above method has no restrictions on driving scenarios. As long as the smart device can enter the intelligent driving mode, the environmental annotation data can be obtained. Therefore, the above method can significantly improve the efficiency and quality of obtaining environmental annotation data, which is conducive to improving the efficiency and performance of iterative upgrades of the first autonomous driving model.

[0085] The above-mentioned step S102 and step S104 are described below.

[0086] 1. Explain step S102.

[0087] In some implementations of the above step S102, the target is an obstacle, and the first attribute information of the target includes a latest braking time based on a safety target.

[0088] The latest braking time can be understood as the latest time when the smart device starts braking. Taking a vehicle as an example, this time is the latest time when the vehicle starts braking.

[0089] The safety goal is to prevent the smart device from colliding with obstacles. That is, if the smart device starts braking at or before the latest braking time, it will not collide with the obstacle; if the smart device starts braking after (not including) the latest braking time, it will collide with the obstacle.

[0090] In some implementations of step S102, the first autonomous driving decision may include the future trajectory of the smart device, which may be understood as the autonomous driving path planned by the first autonomous driving model based on the environmental data. The operation data of the first autonomous driving model may include the future trajectory of the smart device, and the future trajectory may be used to Figure 2 The following steps S1021 to S1023 shown in the figure obtain the first attribute information of the target.

[0091] Step S1021: predicting the target trajectory of the target in the environment according to the environmental data. Specifically, if the target is an obstacle, the environmental data can be subjected to obstacle perception to obtain the motion information of the obstacle (such as position, speed, etc.), and then predicting the obstacle trajectory according to the motion information.

[0092] Step S1022: Based on the safety goal and according to the future trajectory of the smart device and the target trajectory, determine the latest braking time.

[0093] Specifically, based on the future trajectory of the smart device and the target trajectory, it is possible to analyze whether the smart device will collide with the target after starting braking at different times, and then determine the latest braking time from these braking start times based on the analysis results.

[0094] Step S1023: Acquire the first attribute information of the target according to the latest braking time, that is, take the latest braking time as the first attribute information of the target.

[0095] Based on the method described in steps S1021 to S1023 above, the latest braking time of the smart device can be determined conveniently and accurately using the trajectories of the smart device and the obstacle.

[0096] 2. Explain step S104.

[0097] In some implementations of the above step S104, Figure 3 The following steps S1041 to S1042 shown generate environmental annotation data.

[0098] Step S1041: Use the first perception model to perform target perception on the environmental data to obtain a detection frame of the target.

[0099] The first perception model is the same as the actual perception model set on the smart device, and the actual perception model is used to perform target perception on the environmental data collected by the sensor to obtain the first perception data. In addition, the first and second autonomous driving models are used to determine the first and second autonomous driving decisions respectively based on the above-mentioned first perception data.

[0100] Step S1042: Generate environmental annotation data based on the environmental data, the detection frame and the annotation information. Specifically, the detection frame can be used in the environmental data to represent the target, and the annotation information can be associated with the detection frame, and the environmental data with the detection frame and the annotation information associated with the detection frame can be used as environmental annotation data. For example, if the environmental data is an image and the target is a vehicle, the detection frame of the vehicle in the image can be perceived by the first perception model, and then the detection frame can be associated with the annotation information of the vehicle.

[0101] In the method described in steps S1041 to S1042 above, the first perception model is the same as the actual perception model set on the smart device, which can ensure that after the first autonomous driving model is trained using the environmental annotation data and the trained first autonomous driving model is deployed on the smart device, the first autonomous driving model can operate normally, that is, the first autonomous driving decision can be determined normally based on the perception data of the target obtained according to the actual perception model set on the smart device.

[0102] Step S102 will be described again below.

[0103] In some implementations of the above step S102, when obtaining the first attribute information of the target, it can be obtained through the following steps S1024 to S1025.

[0104] Step S1024: Use the second perception model to perform target perception on the environmental data in the operating data to obtain second perception data.

[0105] The perceptual accuracy of the second perceptual model is higher than the perceptual accuracy of the first perceptual model, and the first perceptual model is the first perceptual model described in the aforementioned steps S1041 to S1042.

[0106] Step S1025: Obtain first attribute information of the target in the environment based on the second perception data.

[0107] Since the perception accuracy of the second perception model is higher than that of the first perception model, the accuracy of the second perception data will also be higher than that of the first perception data. The first attribute information obtained using the high-precision second perception data will also be more accurate, thereby effectively ensuring the accuracy of the target labeling information.

[0108] When using the environmental annotation data to train the first autonomous driving model, the target's annotation information will be used as the target's true value information, and the target attribute information obtained by the first autonomous driving model based on the environmental annotation data will be used as the prediction information. The purpose of model training is to reduce the deviation between the above-mentioned true value information and the prediction information. The more accurate the true value information is, the better the training effect of the model will be. Therefore, through the method described in steps S1024 to S1025 above, the accuracy of the target annotation information can be improved, which is conducive to improving the training effect of the first autonomous driving model.

[0109] The following is a description of an embodiment of the data generation method provided by the present application.

[0110] See attached Figure 4 , Figure 4 FIG. 1 is a flow chart of the main steps of a data generation method according to an embodiment of the present application. Figure 4 As shown, in this embodiment, after obtaining the target labeling information through step S103 in the above method embodiment, it is also possible to Figure 4 The following steps S201 to S202 are shown to modify the annotation information and generate environmental annotation data.

[0111] Step S201: Filter specific targets according to the target's annotation information.

[0112] A specific target is a target whose first attribute information satisfies a preset screening condition.

[0113] When setting preset filtering conditions, technical personnel in this field can obtain targets that may affect the safe driving of smart devices, analyze the attribute information of these targets, and determine the filtering conditions that can filter out these targets based on the results of the analysis.

[0114] In some embodiments, the target is an obstacle, and the first attribute information and the second attribute information are both the latest braking time based on the safety target. The preset screening condition can be that the latest braking time is less than a preset time threshold. The latest braking time is the time when the smart device starts braking. If this time is less than the preset time threshold, it means that the smart device is relatively close to the obstacle, and the time left for the smart device to avoid the obstacle and prevent collision with it is relatively short. This obstacle is a high-risk obstacle and needs to be screened out; if this time is greater than or equal to the preset time threshold, it means that the smart device is relatively far from the obstacle, and the time left for the smart device to avoid the obstacle and prevent collision with it is relatively long. This obstacle is a low-risk obstacle and does not need to be screened out.

[0115] When setting the preset time threshold, obstacles with a high risk of collision with the smart device can be obtained, and the maximum time can be obtained from the latest braking time of these obstacles, and the preset time threshold can be set according to the maximum time. For example, the maximum time can be directly used as the preset time threshold.

[0116] Step S202: in response to the user's operation, modify the annotation information of the specific target, and generate environmental annotation data according to the modified annotation information and the environmental data, wherein the modified annotation information is used to indicate the second attribute information of the specific target.

[0117] Specifically, the server can provide a user interaction interface, and the user can operate on this user interaction interface, and the operation can be to enter information in the control of the interface, click the control on the interface, drag the control on the interface, etc. Through the operation, the second attribute information can be provided to the server, that is, the second attribute information is actively provided by the user, and the first attribute information automatically obtained by step S103 in the aforementioned method embodiment is modified to the second attribute information provided by the user. In addition, through the above operation, the detection frame of the specific target can also be provided to the server, and the server can generate environmental annotation data based on the environmental data, the detection frame of the specific target, and the modified annotation information of the specific target.

[0118] Through the method described in steps S201 to S202 above, targets that may affect the safe driving of smart devices can be screened out, and the user can confirm and modify the attribute information of these targets to ensure that after the first autonomous driving model is trained using environmental annotation data containing these attribute information, the first autonomous driving model can obtain a first autonomous driving decision with higher safety.

[0119] The following is combined with Figure 5 , the data generation method provided in this application is explained.

[0120] Figure 5The main process of generating environment annotation data and using the environment annotation data to train the autonomous driving model is shown by example. Figure 5 The model in is the first automatic driving model set on the vehicle. The vehicle is provided with sensors and perception models. The sensors include cameras, radars, etc. The sensors can collect environmental data of the environment where the vehicle is located. The perception model can perform target perception on the environmental data to obtain target perception data (such as target detection box, etc.). In this embodiment, the target is an obstacle.

[0121] like Figure 5 As shown, when the vehicle enters the intelligent driving mode, the model can determine the first autonomous driving decision based on the perception data of the obstacle obtained by the perception model, and the first autonomous driving decision can include the future trajectory of the vehicle. The environmental data collected by the above sensors and the future trajectory of the vehicle can constitute the operation data of the model. The server can generate environmental annotation data through the following steps S301 to S304.

[0122] Step S301: Acquire operation data.

[0123] Step S302: Acquire environmental data in the operation data, pre-mark obstacles on the environmental data, and annotate attribute information based on the environmental data after the obstacle pre-marking.

[0124] When pre-marking obstacles in the environmental data, the second perception model can be used to perceive obstacles in the environmental data in the operating data to obtain second perception data. This process can be understood as pre-marking the second perception data in the environmental data. The second perception model is the same as the second perception model in the aforementioned steps S1024 to S1025.

[0125] When performing attribute information labeling, the method described in the above steps S102 to S103 may be used to obtain the labeling information of the obstacle.

[0126] Step S303: Screen the data and perform manual labeling.

[0127] Specifically, the method described in the aforementioned steps S201 to S202 may be used to screen the target obstacle, modify the labeling information of the target obstacle according to the second attribute information, and generate environmental labeling data.

[0128] Step S304: Obtain an obstacle detection frame and generate environment annotation data.

[0129] Specifically, the method described in the above steps S1041 to S1042 can be used to obtain the detection frame of the obstacle (the obstacle not screened out in the above step S303), and generate environmental annotation data based on the environmental data, the detection frame and the annotation information, and then store the environmental annotation data in the data set.

[0130] After the environment annotation data is generated through the above steps S301 to S304, the model is trained using the environment annotation data.

[0131] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effect of the present application, different steps do not have to be executed in such an order, and they can be executed simultaneously (in parallel) or in other orders. These adjusted schemes are equivalent to the technical schemes described in this application, and therefore will also fall within the scope of protection of this application.

[0132] It is understood by those skilled in the art that all or part of the processes in the method for implementing the above-mentioned embodiment of the present application can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code.

[0133] The following is a description of an embodiment of the data generation system provided by the present application.

[0134] See attached Figure 6 , Figure 6 The main structure of a data generation system according to an embodiment of the present application is exemplified. Figure 6 As shown, the data generation system of the embodiment of the present application includes a data acquisition module 11, a first data marking module 12, and a data generation module 13.

[0135] The data acquisition module 11 can be configured to acquire operating data of a first autonomous driving model on a smart device. A sensor is provided on the smart device, and the sensor is used to collect environmental data of the environment in which the smart device is located. The first autonomous driving model is used to determine a first autonomous driving decision based on the environmental data when the smart device enters an intelligent driving mode, and the operating data includes environmental data.

[0136] The first data labeling module 12 can be configured to obtain first attribute information of a target in the environment where the smart device is located according to the environmental data in the operating data, and label the target according to the first attribute information to obtain labeling information of the target, where the labeling information is used to indicate the first attribute information.

[0137] The data generation module 13 can be configured to generate environment annotation data based on the environment data and the target's annotation information, and the environment annotation data is used to train the first autonomous driving model.

[0138] In this embodiment, a second autonomous driving model is provided on the smart device, and the second autonomous driving model is used to determine a second autonomous driving decision according to environmental data and control the smart device to execute the second autonomous driving decision when the smart device enters the smart driving mode.

[0139] The data generation system is used to perform Figure 1-5 The data generation method embodiment shown in the figure has similar technical principles, technical problems solved and technical effects produced. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process and related instructions of the data generation system can refer to the contents described in the embodiment of the data generation method, which will not be repeated here. In addition, the above-mentioned data generation system is based on modular design, which can improve the fault tolerance of the above-mentioned system. When any module fails, it can be replaced with a normal module in time. The flexibility of the above-mentioned system is also improved based on modular design. Any module in the system can be separated out separately and added to other systems for use.

[0140] In some embodiments, Figure 7 As shown, the data generation system may further include a second data annotation module 14, which may be configured to: filter a specific target according to the target's annotation information, wherein the specific target is a target whose first attribute information satisfies a preset screening condition; modify the annotation information of the specific target in response to a user's operation, and generate environmental annotation data according to the modified annotation information and environmental data; wherein the modified annotation information is used to indicate the second attribute information of the specific target. Based on the second data annotation module 14, manual annotation of information can be realized, and environmental annotation data can be generated.

[0141] In this embodiment, the data generation system is still based on modular design, and the modules will not affect each other, which can ensure that the generation of environmental annotation data can be carried out at the fastest speed. In addition, the first data annotation module 12 and the second data annotation module 14 can be carried out in parallel. Even if the second data annotation module 14 is not enabled or it takes a long time to generate environmental annotation data, it will not affect the normal operation of the first data annotation module 12. A large amount of environmental annotation data obtained by the first data annotation module 12 can still be obtained, and these environmental annotation data can still be used to train the first autonomous driving model, which will not affect the training or upgrade efficiency of the first autonomous driving model.

[0142] Another aspect of the present application also provides a computer-readable storage medium.

[0143] In an embodiment of a computer-readable storage medium according to the present application, the computer-readable storage medium may be configured to store a program for executing the data generation method of the above method embodiment, and the program may be loaded and run by a processor to implement the above data generation method. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The computer-readable storage medium may be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present application is a non-temporary computer-readable storage medium.

[0144] Another aspect of the present application also provides a server.

[0145] In an embodiment of a server according to the present application, the smart device may include at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores a computer program, and when the computer program is executed by the at least one processor, the method described in any of the above embodiments is implemented. The smart device described in the present application may include a driving device, a smart car, a robot, and the like. Figure 8 , Figure 8 FIG. 2 exemplarily shows that the memory 21 and the processor 22 are communicatively connected via a bus.

[0146] So far, the technical solution of the present application has been described in conjunction with an embodiment shown in the accompanying drawings, but it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments. Without departing from the principles of the present application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present application.

Claims

1. A data generation method, characterized in that: The method comprises: Acquire operation data of a first autonomous driving model on a smart device, where a sensor is provided on the smart device, and the sensor is used to collect environmental data of an environment where the smart device is located. The first autonomous driving model is used to determine a first autonomous driving decision according to the environmental data when the smart device enters an intelligent driving mode, and the operation data includes the environmental data. Acquire first attribute information of a target in the environment where the smart device is located according to the environment data in the operation data, and annotate the target according to the first attribute information to obtain annotated information of the target, where the annotated information is used to indicate the first attribute information; Generate environment annotation data according to the environment data and the annotation information of the target, wherein the environment annotation data is used to train the first autonomous driving model; in, A second autonomous driving model is provided on the smart device, and the second autonomous driving model is used to determine a second autonomous driving decision according to the environmental data and control the smart device to execute the second autonomous driving decision when the smart device enters the intelligent driving mode; The target is an obstacle, and the first attribute information of the target includes a latest braking time based on a safety goal; the latest braking time is the latest time when the smart device starts braking, and the safety goal is to prevent the smart device from colliding with the target.

2. The method according to claim 1, characterized in that The operation data includes the first autonomous driving decision, the first autonomous driving decision includes a future trajectory of the smart device, and acquiring first attribute information of a target in an environment where the smart device is located according to the environmental data in the operation data includes: predicting a target trajectory of a target in the environment based on the environmental data; Based on the safety goal and according to the future trajectory of the smart device and the target trajectory, determining the latest braking time; According to the latest braking time, first attribute information of the target is acquired.

3. The method according to claim 1, characterized in that: The method further comprises: Filtering a specific target according to the target's annotation information, where the specific target is a target whose first attribute information meets a preset filtering condition; In response to a user operation, modify the annotation information of the specific target, and generate environmental annotation data according to the modified annotation information and the environmental data; The modified annotation information is used to indicate the second attribute information of the specific target.

4. The method according to claim 3, characterized in that The target is an obstacle, the first attribute information and the second attribute information are both the latest braking time based on a safety goal, the latest braking time is the latest time when the smart device starts braking, and the safety goal is to prevent the smart device from colliding with the target; The preset screening condition is that the latest braking time is less than a preset time threshold.

5. The method according to claim 1, characterized in that The step of generating environment annotation data according to the environment data and the annotation information of the target includes: Using a first perception model to perform target perception on the environmental data to obtain a detection frame of the target; Generate environmental annotation data according to the environmental data, the detection frame and the annotation information; in, The first perception model is the same as an actual perception model set on the smart device, and the actual perception model is used to perform target perception on the environmental data collected by the sensor to obtain first perception data; The first autonomous driving model and the second autonomous driving model are respectively used to determine the first autonomous driving decision and the second autonomous driving decision based on the first perception data.

6. The method according to claim 5, characterized in that The acquiring, according to the environment data in the operation data, first attribute information of a target in the environment where the smart device is located comprises: Using a second perception model to perform target perception on the environmental data in the operation data to obtain second perception data; Acquire first attribute information of a target in the environment according to the second perception data; in, The perceptual accuracy of the second perceptual model is higher than the perceptual accuracy of the first perceptual model.

7. The method according to any one of claims 1 to 6, characterized in that The first autonomous driving model and / or the second autonomous driving model is an end-to-end deep learning model.

8. A data generation system, characterized in that: The system comprises: a data acquisition module, configured to acquire operation data of a first autonomous driving model on a smart device, wherein the smart device is provided with a sensor, and the sensor is used to collect environmental data of an environment where the smart device is located, and the first autonomous driving model is used to determine a first autonomous driving decision according to the environmental data when the smart device enters an intelligent driving mode, and the operation data includes the environmental data; a first data annotation module, configured to obtain first attribute information of a target in the environment where the smart device is located according to the environment data in the operation data, and annotate the target according to the first attribute information to obtain annotation information of the target, wherein the annotation information is used to indicate the first attribute information; a data generation module, configured to generate environment annotation data according to the environment data and the annotation information of the target, wherein the environment annotation data is used to train the first autonomous driving model; in, A second autonomous driving model is provided on the smart device, and the second autonomous driving model is used to determine a second autonomous driving decision according to the environmental data and control the smart device to execute the second autonomous driving decision when the smart device enters the intelligent driving mode; The target is an obstacle, and the first attribute information of the target includes a latest braking time based on a safety goal; the latest braking time is the latest time when the smart device starts braking, and the safety goal is to prevent the smart device from colliding with the target.

9. The system according to claim 8, characterized in that The system includes a second data annotation module, wherein the second data annotation module is configured to: Filtering a specific target according to the target's annotation information, where the specific target is a target whose first attribute information meets a preset filtering condition; In response to a user operation, modify the annotation information of the specific target, and generate environmental annotation data according to the modified annotation information and the environmental data; The modified annotation information is used to indicate the second attribute information of the specific target.

10. A server, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores a computer program, and when the computer program is executed by the at least one processor, the data generation method according to any one of claims 1 to 7 is implemented.

11. A computer-readable storage medium storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by a processor to execute the data generating method according to any one of claims 1 to 7.

Images