Intelligent data acquisition method, device and equipment and readable storage medium

Abstract

An intelligent data acquisition method, apparatus and device, and a readable storage medium, the method comprising: monitoring a reasoning result of an online model, and performing confidence assessment and error pattern analysis based on the reasoning result to obtain a model performance blind area; generating a data demand instruction according to the model performance blind area, and generating an acquisition strategy according to the data demand instruction and the resource constraint condition; executing the collection strategy to collect sample data, and performing quality evaluation screening and sample labeling on the sample data to obtain high-quality sample data; after high-quality sample data is supplemented to a training set of an online model, incremental learning and model updating are performed on the online model. According to the method, the model performance blind area is determined through confidence evaluation and error pattern analysis, and the data demand is dynamically analyzed and the acquisition strategy is generated according to the model performance blind area, so that the acquisition strategy can respond to the dynamic demand of model training in real time, and the pertinence and value density of data acquisition are greatly improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence sample acquisition technology, specifically to an intelligent data acquisition method, device, equipment, and readable storage medium. Background Technology

[0002] With the rapid development of intelligent driving technology, it is gradually being tested and implemented in various vehicle models. However, intelligent driving still lags significantly behind human driving, and its operational stability in complex and ever-changing scenarios urgently needs improvement. In the training and testing of intelligent driving models, scene data refers to the real-time environmental perception data collected by various onboard sensors, such as visual sensor data, ultrasonic radar data, and real-time positioning data. This data is a crucial resource for intelligent driving development, training, and testing. High-value scene data that is comprehensive, has numerous corner cases, high acquisition accuracy, and is compatible with various vehicle models can effectively improve the performance ceiling of intelligent driving models, enabling intelligent driving systems to adapt to more complex road conditions.

[0003] In related technologies, during the training of intelligent driving models, scene data is typically collected through test vehicles equipped with various sensors. This data is collected in real-time during test or real-world driving scenarios and then processed and stored. Alternatively, trigger rules are set so that when the test vehicle meets the predefined data collection criteria, data collection begins through the various sensors.

[0004] However, existing scene data acquisition methods generally employ a "broad-based" approach, collecting a large amount of general data, but the proportion of high-value data that is truly effective in improving the performance of current models is very low. Furthermore, existing data acquisition methods typically use static rules; once the rules and objectives are set, they do not change within a given period, failing to respond promptly to real-time dynamic issues exposed by the model during online operation. For example, when the model's recognition rate drops for a newly emerging scene, static rules cannot immediately target that scene with specific data collection. Therefore, existing technologies suffer from a low proportion of high-value data and an inability to adapt to the dynamic training needs of models, requiring improvement. Summary of the Invention

[0005] This application provides an intelligent data acquisition method, apparatus, device, and readable storage medium, which can solve the technical problems of low proportion of high-value data and inability to adapt to the dynamic training needs of models in the prior art.

[0006] In a first aspect, embodiments of this application provide an intelligent data acquisition method, including: Monitor the inference results of the online model, and conduct confidence assessment and error pattern analysis based on the inference results to obtain the blind spots of model performance; Data requirement instructions are generated based on the model's performance blind spots, and data acquisition strategies are generated based on the data requirement instructions and resource constraints. The data collection strategy is implemented to collect sample data, and the sample data is quality-assessed, screened, and labeled to obtain high-quality sample data. After supplementing the training set of the online model with high-quality sample data, the online model is incrementally learned and updated.

[0007] In conjunction with the first aspect, in one implementation, the model performance blind spot is obtained by performing confidence assessment and error pattern analysis based on the inference results, including: The confidence level of the inference results is evaluated and a preset threshold is used to filter out low-confidence inference results. The decision boundary of the model is determined based on the low-confidence inference results. Cluster analysis of misidentified cases in the reasoning results yields high-frequency error patterns; The model performance blind spot is determined based on the model decision boundary and high-frequency error patterns.

[0008] In conjunction with the first aspect, in one implementation, a data acquisition strategy is generated based on data demand instructions and resource constraints, including: The data acquisition strategy is obtained through multi-objective optimization based on data demand instructions and resource constraints. Among them, the resource constraints are determined based on the device status of the intelligent acquisition terminal, network bandwidth limitations, and storage space limitations; The optimization objectives of multi-objective optimization include data acquisition cost, data acquisition time, and data diversity indicators; The data acquisition strategy includes the data acquisition location, data acquisition time, data acquisition sensor type, data acquisition sensor device parameters, and / or data acquisition trigger conditions.

[0009] In conjunction with the first aspect, in one implementation, high-quality sample data is obtained by performing quality assessment screening and sample labeling on the sample data, including: The relevance score between the sample data and the data requirement instructions is used to obtain the requirement relevance. The sample quality score is determined based on demand relevance, sample data clarity, and sample data information content. After screening samples based on sample quality scores, samples are labeled using a pre-defined labeling strategy to obtain high-quality sample data.

[0010] In conjunction with the first aspect, in one implementation, the preset annotation strategy includes: The sample data to be labeled are sorted from high to low according to the sample quality score; The top-ranked sample data to be labeled are sent to the manual labeling queue, while the other sample data to be labeled are sent to the automated tool labeling queue.

[0011] In conjunction with the first aspect, in one implementation method, the intelligent data acquisition method further includes: Extract data features from sample data; Based on the data characteristics of each sample, duplicate sample data are identified and removed.

[0012] In conjunction with the first aspect, in one implementation method, the intelligent data acquisition method further includes: Identify and classify the data collection scenarios in the data requirement instructions; If the data collection scenario is a sparse collection scenario, the synthetic data is generated by generating large model samples according to the data demand instructions, and the synthetic data is added to the training set of the online model.

[0013] Secondly, embodiments of this application provide an intelligent data acquisition device, comprising: The performance blind spot identification module is used to monitor the inference results of the online model in real time, and to obtain the model performance blind spot by performing confidence evaluation and error pattern analysis based on the inference results. The data acquisition strategy generation module is used to generate data requirement instructions based on the model performance blind spot, and to generate data acquisition strategies based on the data requirement instructions and resource constraints. The data acquisition and filtering module is used to execute acquisition strategies to collect sample data, and to perform quality assessment, filtering, and sample labeling on the sample data to obtain high-quality sample data. The model update module is used to supplement the online model's training set with high-quality sample data, and then perform incremental learning and model updates on the online model.

[0014] Thirdly, embodiments of this application provide an intelligent data acquisition device, which includes a processor, a memory, and an intelligent data acquisition program stored in the memory and executable by the processor, wherein when the intelligent data acquisition program is executed by the processor, it implements the steps of the intelligent data acquisition method as described above.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium, characterized in that the computer-readable storage medium stores an intelligent data acquisition program, wherein when the intelligent data acquisition program is executed by a processor, it implements the steps of the intelligent data acquisition method as described above.

[0016] The beneficial effects of the technical solutions provided in this application include: This application identifies model performance blind spots through confidence assessment and error mode analysis, and dynamically analyzes data requirements and generates acquisition strategies based on these blind spots. This enables the acquisition strategies to respond in real time to the dynamic needs of model training, greatly improving the relevance and value density of data acquisition. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating an embodiment of the intelligent data acquisition method of this application; Figure 2 This is a flowchart illustrating the process of determining the performance blind zone of a model according to an embodiment of this application; Figure 3 This is a schematic diagram of the preliminary sample screening and filtering process in an embodiment of this application; Figure 4 This is a schematic diagram of the sample quality assessment and screening process in an embodiment of this application; Figure 5 This is a flowchart illustrating the annotation strategy of an embodiment of this application; Figure 6 This is a schematic diagram of the functional modules of an embodiment of the intelligent data acquisition device of this application; Figure 7 This is a schematic diagram of the hardware structure of the intelligent data acquisition device involved in the embodiments of this application. Detailed Implementation

[0018] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0020] In a first aspect, embodiments of this application provide an intelligent data acquisition method.

[0021] In one embodiment, reference is made to Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the intelligent data acquisition method of this application. Figure 1 As shown, intelligent data acquisition methods include: S101. Monitor the inference results of the online model, and conduct confidence assessment and error pattern analysis based on the inference results to obtain the blind spots of model performance; S102. Generate data requirement instructions based on the model performance blind spot, and generate a data acquisition strategy based on the data requirement instructions and resource constraints. S103. Execute the data collection strategy to collect sample data, and perform quality assessment, screening, and sample labeling on the sample data to obtain high-quality sample data. S104. After supplementing the training set of the online model with high-quality sample data, perform incremental learning and model updates on the online model.

[0022] Specifically, firstly, the implementation example analyzes the blind spots in model performance by monitoring the inference results of the online model and performing confidence assessment and error pattern analysis, thereby determining the data required for model training.

[0023] Confidence assessment calculates the confidence level of the inference results output by the model. When the confidence level is lower than a set threshold, the data point is considered to be on the model's decision boundary, and the model has difficulty accurately judging that data point. Error pattern analysis involves clustering cases where the model makes incorrect judgments to identify high-frequency or serious error patterns that the model struggles to correctly identify. Combining the model's decision boundary and high-frequency error patterns can determine the model's performance blind spots, thereby determining the data needed for incremental training or parameter fine-tuning of the online model.

[0024] Then, based on the model's performance blind spots, corresponding data requirement instructions can be generated, and combined with resource constraints, such as the device status of the intelligent acquisition terminal, network bandwidth limitations, and storage space limitations, the acquisition strategy can be determined.

[0025] In generating the data acquisition strategy, multi-objective optimization can be performed using optimization algorithms. Optimization objectives include acquisition cost, acquisition time, and data diversity metrics. Iterative optimization using these algorithms generates an executable, optimal data acquisition strategy.

[0026] Then, the samples collected after implementing the collection strategy are screened through quality assessment and labeled to obtain high-quality sample data for incremental training.

[0027] The quality assessment and screening process involves a comprehensive evaluation based on multiple dimensions, including the relevance of the sample to the requirement, sample clarity, and information content. Samples meeting the quality score requirements are retained. Sample annotation utilizes both automated tools and a manual annotation queue. The highest-scoring samples are sent to the manual annotation queue to maximize the value of manual annotation, while other qualified samples are annotated using automated tools.

[0028] Finally, the high-quality and labeled new data is added to the model's training set, incremental training or fine-tuning of the model is started, the updated model is redeployed, and the process returns to the model inference result monitoring step, forming a closed loop.

[0029] In this embodiment, confidence assessment and error pattern analysis are used to determine the model performance blind spots. Based on the model performance blind spots, data requirements are dynamically analyzed and acquisition strategies are generated. This allows the acquisition strategy to respond to the dynamic needs of model training in real time, greatly improving the targeting and value density of data acquisition.

[0030] Furthermore, in one embodiment, Figure 2 This is a flowchart illustrating the process of determining the performance blind zone of a model according to an embodiment of this application, as shown below. Figure 2 As shown, confidence assessment and error pattern analysis based on the inference results reveal the model performance blind spots, including: S201. Calculate the confidence level of the inference results and filter them using a preset threshold to obtain low-confidence inference results. Determine the model decision boundary based on the low-confidence inference results. S202. Cluster analysis is performed on the incorrectly identified cases in the reasoning results to obtain high-frequency error patterns; S203. Determine the blind zone of model performance based on the model decision boundary and high-frequency error patterns.

[0031] Specifically, for the inference results of the model, the implementation example uses two methods—confidence assessment and error mode analysis—to determine the performance blind spots of the model.

[0032] During the confidence assessment process, it is considered that in intelligent driving of vehicles, the relevant models may have various prediction tasks, such as image recognition, object detection, and trajectory prediction, and the output results of these prediction tasks also have different forms. Therefore, for different types of prediction tasks, the implementation can adaptively sample different methods to calculate the confidence level.

[0033] For example, for classification tasks, for models that support probability outputs, such as logistic regression, the probability value of the predicted class can be directly used as the confidence level. For regression tasks, methods such as quantile regression or bootstrap can be used to estimate the possible range of values.

[0034] Then, based on the threshold set for the model, the inference results with low confidence are filtered out. Based on these low-confidence inference results, the model decision boundary is determined, that is, the model's accuracy is insufficient when facing which prediction tasks.

[0035] It should be noted that the confidence threshold in this application is not fixed and can be dynamically adjusted according to the model. For example, models performing different prediction tasks are usually set with different confidence thresholds. Moreover, the confidence threshold is dynamically adjusted according to the model's performance. That is, when the model's prediction performance is average, the confidence threshold is appropriately lowered to prioritize training and strengthening scenarios with particularly poor prediction results; conversely, when the prediction performance is already very good, the confidence threshold is appropriately raised.

[0036] During error pattern analysis, the implementation example clusters the model's incorrect recognition cases to identify high-frequency error patterns (such as low vehicle recognition rate under specific lighting conditions). Additionally, some low-frequency but serious error patterns can also be considered as error patterns requiring further processing.

[0037] In this embodiment, by determining the blind spots in model performance through confidence assessment and error pattern analysis, the data needed for incremental training or parameter fine-tuning of the model can be identified, providing guidance for the direction of subsequent data collection and improving the relevance and value density of data collection.

[0038] Furthermore, in one embodiment, generating a data acquisition strategy based on data requirement instructions and resource constraints includes: The data acquisition strategy is obtained through multi-objective optimization based on data demand instructions and resource constraints. Among them, the resource constraints are determined based on the device status of the intelligent acquisition terminal, network bandwidth limitations, and storage space limitations; The optimization objectives of multi-objective optimization include data acquisition cost, data acquisition time, and data diversity indicators; The data acquisition strategy includes the data acquisition location, data acquisition time, data acquisition sensor type, data acquisition sensor device parameters, and / or data acquisition trigger conditions.

[0039] Specifically, in this embodiment, the data request instruction is a formatted text type, such as: "Urgently need to collect rear 45-degree angle images of 'nighttime, rainy day, bus', with target bounding box confidence between 0.3 and 0.7". Based on the data request instruction and the resource constraints of the acquisition terminal, an optimal acquisition strategy can be generated through a multi-objective optimization algorithm.

[0040] In multi-objective optimization, the optimization objectives include data collection cost, data collection time, and data diversity metrics. The data diversity metric can be calculated by comprehensively considering the number of scene types and the evenness of distribution of the collected samples.

[0041] Resource constraints can be determined based on the device status of the intelligent data acquisition terminal, network bandwidth limitations, and storage space limitations. Constraints can also be added by incorporating factors such as weather and time. For example, if a weather forecast indicates rain at a certain time, samples from rainy weather scenarios can be prioritized for collection during that period.

[0042] Multi-objective optimization algorithms can be implemented using genetic algorithms, such as ant colony optimization. The final data acquisition strategy needs to include the acquisition location, acquisition time, sensor type, sensor device parameters, and / or data acquisition trigger conditions.

[0043] In this embodiment, by generating a data collection strategy based on the model's requirements, the goal of "collecting only the data the model needs" is achieved, which greatly improves the targeting and value density of data collection and reduces the storage and processing costs of invalid data.

[0044] Furthermore, in one embodiment, Figure 3 This is a schematic diagram of the preliminary sample screening and filtering process in an embodiment of this application, as shown below. Figure 3 As shown, the initial sample screening and filtering process includes: S301. Extract data features from sample data; S302. Based on the data characteristics of each sample data, identify and remove duplicate sample data.

[0045] Specifically, during the data acquisition phase, the embodiment performs preliminary data filtering at the edge. The edge calculates the features of the newly acquired sample data and compares them with the features of the already uploaded data to avoid duplicate data uploads.

[0046] It is important to note that, in order to reduce the amount of computation in the initial screening process, for each sample data, it is not necessary to compare its features with all the data that have already been uploaded. It is only necessary to compare it with a few recently uploaded data (generally speaking, duplicate data are more likely to appear in data collected at similar times).

[0047] In addition, preliminary quality checks (such as blurriness and brightness) need to be performed on the data at the edge to filter out clear but invalid data.

[0048] In this embodiment, preliminary screening and filtering of samples can avoid uploading duplicate and invalid data, thus saving bandwidth resources.

[0049] Furthermore, in one embodiment, Figure 4 This is a schematic diagram of the sample quality assessment and screening process in an embodiment of this application, as shown below. Figure 4 As shown, high-quality sample data is obtained through quality assessment, screening, and labeling, including: S401. The relevance score between the sample data and the data requirement instructions is used to obtain the requirement relevance. S402. Determine the sample quality score based on demand relevance, sample data clarity, and sample data information content. S403. After screening samples based on sample quality scores, label the samples based on a preset labeling strategy to obtain high-quality sample data.

[0050] Furthermore, in one embodiment, Figure 5 This is a flowchart illustrating the annotation strategy of an embodiment of this application, as shown below. Figure 5 As shown, the preset annotation strategies include: S501. Sort the sample data to be labeled from high to low according to the sample quality score; S502. Send the top-ranked sample data to the manual annotation queue, and send the other sample data to the automated tool annotation queue.

[0051] Specifically, the returned data needs to be quality assessed and labeled before it can be added to the training set as training data.

[0052] In quality assessment, it is necessary to calculate the relevance of the sample data to the requirement instructions, the clarity and information content of the sample data, and then combine the requirement relevance, clarity and information content to determine the quality score of the sample.

[0053] Demand relevance can be evaluated using a lightweight neural network model, which determines demand relevance by identifying possible label types of samples and comparing them with structured demand instructions.

[0054] The sharpness of a sample is adaptively calculated based on the sample type. For example, for image samples, sharpness can be calculated by measuring the average gradient magnitude; for radar sensor data samples, sharpness can be calculated using the signal-to-noise ratio and point cloud density.

[0055] The information content of a sample is also adaptively calculated based on the sample type. For example, for image samples, it can be calculated using color entropy and edge density. For radar sensing data samples, the information content can be calculated using the standard deviation or entropy of the effective point cloud distribution in the range, azimuth, and velocity dimensions.

[0056] After obtaining a sample quality score by comprehensively considering the relevance of demand, the clarity of sample data, and the amount of information in the sample data, the implementation example retains only the samples that meet the quality requirements and filters out the low-quality samples.

[0057] It should be noted that when calculating and determining the sample quality score, demand relevance should have a higher weight in the calculation of the sample quality score, because samples with high demand relevance are usually the samples that the model needs more at this time.

[0058] Then, during the annotation process, samples are sorted according to their quality scores. The highest-quality samples, which also have the least model uncertainty, are prioritized for manual annotation, maximizing the value of manual annotation. Other samples that meet the quality requirements are annotated using automated tools.

[0059] In this embodiment, by screening samples based on their quality, the relevance and value density of the sample data can be further improved, thereby enhancing the data quality.

[0060] Furthermore, in one embodiment, the intelligent data acquisition method further includes: Identify and classify the data collection scenarios in the data requirement instructions; If the data collection scenario is a sparse collection scenario, the synthetic data is generated by generating large model samples according to the data demand instructions, and the synthetic data is added to the training set of the online model.

[0061] Specifically, considering that there may be some rare or difficult-to-collect scenarios (such as extreme accidents), in addition to the data collection steps above, the embodiments also provide data generation methods for rare or difficult-to-collect scenarios.

[0062] The implementation example identifies and classifies the data collection scenarios in the data request instructions. If the classified scenario is a rare collection scenario, the implementation example uses a generative large model (such as the Diffusion model) to transform the data request instructions into prompt words for input into the model, thereby generating synthetic data. At the same time, the quality of the synthetic data is also evaluated, and the high-quality data is added to the training set.

[0063] In this embodiment, the generative large model generates sample data from sparsely collected scenarios, which can supplement some samples that are needed for model training but are difficult to collect under normal circumstances, thereby further improving the relevance of the samples.

[0064] Secondly, embodiments of this application also provide an intelligent data acquisition device.

[0065] In one embodiment, reference is made to Figure 6 , Figure 6 This is a functional module diagram of an embodiment of the intelligent data acquisition device of this application. Figure 6 As shown, the intelligent data acquisition device includes: The performance blind spot identification module 601 is used to monitor the inference results of the online model in real time, and to obtain the model performance blind spot by performing confidence evaluation and error pattern analysis based on the inference results. The data acquisition strategy generation module 602 is used to generate data requirement instructions based on the model performance blind spot, and to generate a data acquisition strategy based on the data requirement instructions and resource constraints. The data acquisition and filtering module 603 is used to execute the acquisition strategy to collect sample data, and to perform quality assessment, filtering, and sample labeling on the sample data to obtain high-quality sample data. The model update module 604 is used to supplement the training set of the online model with high-quality sample data, and then perform incremental learning and model update on the online model.

[0066] The functions of each module in the above-mentioned intelligent data acquisition device correspond to the steps in the above-mentioned intelligent data acquisition method embodiment, and their functions and implementation processes will not be described in detail here.

[0067] Thirdly, embodiments of this application provide an intelligent data acquisition device, which can be a personal computer, vehicle terminal, or other device with data processing capabilities.

[0068] Reference Figure 7 , Figure 7 This is a schematic diagram of the hardware structure of the intelligent data acquisition device involved in the embodiments of this application. In the embodiments of this application, the intelligent data acquisition device may include a processor, a memory, a communication interface, and a communication bus.

[0069] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.

[0070] Communication interfaces include input / output (I / O) interfaces, physical interfaces, and logical interfaces used for interconnecting internal components of the intelligent data acquisition device, as well as interfaces used for interconnecting the intelligent data acquisition device with other devices (such as other computing devices or user equipment). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user equipment can be displays, keyboards, etc.

[0071] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0072] The processor can be a general-purpose processor, which can call the intelligent data acquisition program stored in the memory and execute the intelligent data acquisition method provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the intelligent data acquisition program is called can be referred to in the various embodiments of the intelligent data acquisition method of this application, and will not be repeated here.

[0073] Those skilled in the art will understand that Figure 7 The hardware structure shown does not constitute a limitation of this application and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0074] Fourthly, embodiments of this application also provide a computer-readable storage medium.

[0075] The present application has a computer-readable storage medium storing an intelligent data acquisition program, wherein when the intelligent data acquisition program is executed by a processor, it implements the steps of the intelligent data acquisition method described above.

[0076] The method implemented when the intelligent data acquisition program is executed can be referred to in various embodiments of the intelligent data acquisition method of this application, and will not be repeated here.

[0077] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0078] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.

[0079] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.

[0080] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0081] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.

[0082] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods of the various embodiments of this application.

[0083] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method of intelligent data acquisition, characterized by, The method comprises the following steps: monitoring inference results of an online model, performing confidence evaluation and error mode analysis based on the inference results to obtain a model performance blind area; generating a data requirement instruction according to the model performance blind area, and generating a collection strategy according to the data requirement instruction and a resource constraint condition; executing the collection strategy to collect sample data, and performing quality evaluation screening and sample labeling on the sample data to obtain high-quality sample data; supplementing the high-quality sample data to a training set of the online model, and performing incremental learning and model updating on the online model.

2. The intelligent data collection method of claim 1, wherein, The confidence evaluation and error mode analysis based on the inference results to obtain a model performance blind area comprises the following steps: performing confidence evaluation and preset threshold screening on the inference results to obtain low-confidence inference results, and determining a model decision boundary according to the low-confidence inference results; performing clustering analysis on error identification cases in the inference results to obtain high-frequency error modes; determining a model performance blind area according to the model decision boundary and the high-frequency error modes.

3. The intelligent data collection method of claim 1, wherein, The generation of the collection strategy according to the data requirement instruction and the resource constraint condition comprises the following steps: performing multi-objective optimization according to the data requirement instruction and the resource constraint condition to obtain a collection strategy; wherein the resource constraint condition is determined according to a device state, a network bandwidth limitation and a storage space limitation of an intelligent collection terminal; optimization objectives of the multi-objective optimization include a collection cost, a collection time and a data diversity index; the collection strategy includes a collection location, a collection time, a collection sensor type, a collection sensor device parameter and / or a data collection trigger condition.

4. The intelligent data collection method of claim 1, wherein, The quality evaluation screening and sample labeling on the sample data to obtain high-quality sample data comprises the following steps: performing correlation scoring on the sample data and the data requirement instruction to obtain a requirement relevance; determining a sample quality score according to the requirement relevance, a sample data clarity and a sample data information amount; after sample screening according to the sample quality score, performing sample labeling based on a preset labeling strategy to obtain high-quality sample data.

5. The intelligent data collection method of claim 4, wherein, The preset labeling strategy comprises the following steps: sorting sample data to be labeled from high to low according to the sample quality score; sending a plurality of sample data to be labeled at the top of the sorting to a manual labeling queue, and sending other sample data to be labeled to an automated tool labeling queue.

6. The intelligent data collection method of claim 1, wherein, The method further comprises the following steps: extracting data features of the sample data; identifying and removing duplicate sample data from the sample data according to the data features of the sample data.

7. The intelligent data collection method of claim 1, wherein, The method further comprises the following steps: identifying and classifying data collection scenarios in the data requirement instruction; if the data collection scenario is a rare collection scenario, generating synthetic data by performing generative large model sample generation according to the data requirement instruction, and supplementing the synthetic data to a training set of the online model.

8. An intelligent data acquisition device, characterized by The method comprises the following steps: a performance blind area identification module for monitoring inference results of an online model in real time, performing confidence evaluation and error mode analysis based on the inference results to obtain a model performance blind area; The collection strategy generation module is configured to generate a data requirement instruction according to the model performance blind area, and generate a collection strategy according to the data requirement instruction and a resource constraint condition; The data collection and screening module is configured to execute the collection strategy to collect sample data, and perform quality evaluation screening and sample labeling on the sample data to obtain high-quality sample data; The model updating module is configured to supplement the high-quality sample data to a training set of the online model, and perform incremental learning and model updating on the online model.

9. An intelligent data acquisition device, characterized by The intelligent data collection device comprises a processor, a memory, and an intelligent data collection program stored in the memory and executable by the processor, wherein the intelligent data collection program, when executed by the processor, implements the steps of the intelligent data collection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores an intelligent data collection program, wherein the intelligent data collection program, when executed by the processor, implements the steps of the intelligent data collection method according to any one of claims 1 to 7.

Images