Data processing method and device, equipment and computer medium
By adjusting and training the parameters of the initial prediction model, and optimizing the model output based on the prediction results analysis, the problem of low efficiency of annotation personnel is solved, and efficient training and accuracy improvement of the model under specific tasks are achieved.
Patent Information
- Application Number
- CN202411466333.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-18
- Publication Date
- 2026-04-21
AI Technical Summary
In existing technologies, the method of having annotators label the segmentation results results in low model training efficiency, especially when there are large areas of errors. In fact, model-assisted annotation reduces the efficiency of annotators.
By inputting sample data into the initial prediction model, the first prediction result is obtained, and the prediction result analysis information is determined based on the target reference result. If the preset conditions are not met, the parameters of the initial prediction model are adjusted and training continues until the conditions are met, and the target prediction model is obtained.
It improves the efficiency of model training, saves annotation time, and enhances the accuracy and performance of the model under specific tasks.
Smart Images
Figure CN121904334A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of image recognition technology, and in particular relates to a data processing method, apparatus, device and computer medium. Background Technology
[0002] In AI model training, manual annotation is a crucial step in building high-quality training sets. With increased computing power and performance optimization of large models, annotators can improve annotation efficiency by using high-precision pre-trained models to assist in annotation. However, in specific tasks, pre-trained models may not provide optimal prediction results on the first attempt. In such cases, interactive annotation methods are often used to further improve efficiency. This involves annotators manually reducing errors in the pre-trained model's output and improving the accuracy of its predictions.
[0003] In instance segmentation tasks, annotators need to provide accurate mask annotations for each target to be labeled. To obtain the most accurate mask results possible, annotators use trained segmentation models. After configuring the categories to be segmented, they provide bounding boxes to instruct the segmentation model to segment objects only in specific regions, improving annotation efficiency. However, this method has a significant drawback: the model cannot correct errors in the segmentation results. For large-area errors, annotators can only delete the model's predictions and manually annotate. In this case, model-assisted annotation actually reduces the annotation efficiency of the annotators. In related technologies, the method of having annotators annotate the segmentation results results in low model training efficiency. Summary of the Invention
[0004] This application provides an implementation scheme that differs from related technologies, in order to solve the technical problem that the model training efficiency is low in related technologies where segmentation results are labeled by annotators.
[0005] Firstly, this application provides a data processing method, including:
[0006] The sample data is input into the initial prediction model to obtain the first prediction result corresponding to the sample data;
[0007] Based on the target reference result corresponding to the sample data and the first prediction result, the prediction result analysis information of the first prediction result is determined;
[0008] Determine whether the prediction result analysis information indicates that the first prediction result meets the first preset condition;
[0009] If the first prediction result does not meet the first preset condition, the parameters of the initial prediction model are adjusted, and the initial prediction model is trained again based on the prediction result analysis information, the sample data, and the first prediction result to obtain the target prediction model corresponding to the initial prediction model. The target prediction model is used to output the corresponding target prediction result based on the input data to be processed.
[0010] Secondly, this application provides a data processing apparatus, comprising:
[0011] An input unit is used to input sample data into an initial prediction model to obtain a first prediction result corresponding to the sample data;
[0012] The determining unit is used to determine the prediction result analysis information of the first prediction result based on the target reference result corresponding to the sample data and the first prediction result;
[0013] The determining unit is further configured to determine whether the prediction result analysis information indicates that the first prediction result meets the first preset condition;
[0014] The training unit is configured to adjust the parameters of the initial prediction model if the first prediction result does not meet the first preset condition, and continue to train the initial prediction model based on the prediction result analysis information, the sample data, and the first prediction result to obtain the target prediction model corresponding to the initial prediction model. The target prediction model is configured to output the corresponding target prediction result based on the input data to be processed.
[0015] Thirdly, this application provides an electronic device, comprising:
[0016] Processor; and
[0017] Memory for storing the executable instructions of the processor;
[0018] The processor is configured to execute the first aspect, or any of the possible implementations of the first aspect, by executing the executable instructions.
[0019] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the first aspect or any of the methods in various possible implementations of the first aspect.
[0020] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect or any of the methods in various possible implementations of the first aspect.
[0021] This application provides a scheme for inputting sample data into an initial prediction model to obtain a first prediction result corresponding to the sample data; determining prediction result analysis information of the first prediction result based on the target reference result corresponding to the sample data and the first prediction result; determining whether the prediction result analysis information indicates that the first prediction result meets a first preset condition; if the first prediction result does not meet the first preset condition, adjusting the parameters of the initial prediction model, and continuing to train the initial prediction model based on the prediction result analysis information, the sample data, and the first prediction result to obtain a target prediction model corresponding to the initial prediction model. The target prediction model is used to output a corresponding target prediction result based on the input data to be processed. This scheme introduces analysis information on the first prediction result, i.e., prediction result analysis information, and trains the initial prediction model based on the prediction result analysis information. The prediction result analysis information can be regarded as automatically labeled information for the first prediction result, saving labeling time and improving the efficiency of model training. Attached Figure Description
[0022] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:
[0023] Figure 1 This is a schematic diagram of the structure of an initial prediction model provided in an embodiment of this application;
[0024] Figure 2a A schematic flowchart illustrating a data processing method provided in an embodiment of this application;
[0025] Figure 2b A schematic flowchart illustrating a data processing method provided in an embodiment of this application;
[0026] Figure 2c A schematic flowchart illustrating a data processing method provided in an embodiment of this application;
[0027] Figure 2d A schematic flowchart illustrating a data processing method provided in an embodiment of this application;
[0028] Figure 2e This is a schematic diagram illustrating the relationship between a first prediction result and a target reference result provided in an embodiment of this application.
[0029] Figure 3 This is a schematic diagram of the structure of a data processing apparatus provided in an embodiment of this application;
[0030] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0031] The embodiments of this application are described in detail below, with examples of these embodiments illustrated in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.
[0032] The terms "first" and "second," etc., used in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the present application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0033] The inventors discovered through research that in natural language processing tasks, due to the complexity and ambiguity of language, pre-trained models can produce ambiguities when performing word segmentation, part-of-speech classification, or translation tasks. To obtain more accurate annotation results, annotators will correct some of the predictions made by the pre-trained models, and the pre-trained models will then re-produce predictions based on the corrected content. Through this interactive process, annotators can efficiently obtain accurate and high-quality annotation results using the pre-trained models.
[0034] Segment Anything Model (SAM) is a highly accurate and robust interactive segmentation model. Trained on massive amounts of data from the internet, it accurately segments various common objects in everyday life scenarios. Compared to traditional segmentation methods, SAM has an irreplaceable advantage: it can accept various prior information from users (text, bounding boxes, key points, etc.) to segment specified objects or object categories, greatly improving the flexibility of image segmentation tasks.
[0035] In instance segmentation and annotation tasks, annotators use trained segmentation models for auxiliary annotation. However, this method has a significant drawback: the model cannot correct the segmentation results. For large-scale errors, annotators can only delete the model's predictions and switch to manual annotation. In this case, model-assisted annotation actually reduces the annotation efficiency of the annotators.
[0036] Compared to instance segmentation models that require "configuring categories before drawing bounding boxes," the SAM series models eliminate the need for pre-configuration of categories for the target object. They also offer a more direct and convenient annotation method: annotators can directly left-click on the target object (specifying a positive sample point), and the SAM series model will output the mask prediction result for that target. Furthermore, the SAM series models introduce the concept of negative samples. When the model's prediction is incorrect, annotators can right-click on the incorrect region (specifying a negative sample point), and the SAM series model will combine the previous prediction result with the newly input negative sample region to provide an optimized annotation result. These two new features significantly improve data annotation efficiency.
[0037] Taking arm segmentation and annotation as an example: When annotating data, annotators need to label nine categories, including the user's left hand, right hand, their own body, held objects, and other people in the image, with different labels. The SAM series models are well-suited for annotation tasks involving a wide range of categories in everyday life scenarios. However, as mentioned earlier, the SAM series models are trained using massive amounts of internet images, and the human body data used during training is primarily from a third-person perspective. Therefore, the SAM series models often perform poorly on first-person images captured with distorted fisheye lenses. Annotators have found that labeling an image using the original SAM model can be slower than labeling from scratch. Therefore, this solution uses arm segmentation data to fine-tune and train the SAM series models to improve their accuracy on specific tasks.
[0038] Fine-tuning, a crucial technique in deep learning, is widely applied across various fields. It involves fine-tuning a pre-trained model for a specific task or dataset to improve its performance in particular tasks and scenarios. During fine-tuning, the model typically updates only a small portion of its parameters. This approach fully utilizes the general knowledge learned by the pre-trained model while ensuring that it outperforms the original model in new scenarios.
[0039] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.
[0040] Current mainstream model evaluation schemes do not set up separate evaluation metrics for interactive scenarios. Instead, they focus on evaluating the model's first output result under a specific input. Therefore, existing training strategies are designed around this goal, which to some extent causes the model to overemphasize the performance of the first output and ignore the human-computer interaction scenario.
[0041] Fine-tuning schemes are used to optimize and improve pre-trained models with large parameters and difficult training scenarios for specific applications. Much research focuses on reducing computational costs during fine-tuning while maintaining good model performance. Therefore, mainstream fine-tuning techniques use different training data (i.e., application-specific data) and the same training strategies as the model's initial training. However, mainstream model training and evaluation schemes are not designed for interactive scenarios, making it impossible to reasonably and objectively evaluate model performance in interactive environments using existing model evaluation methods.
[0042] The inventors discovered through research that when a positive sample is specified in the missed detection area, the model also predicts a false detection area. Then, when a negative sample is specified in the false detection area, the model's prediction reverts to the previous result that included the missed detection area. This cycle repeats itself, and the model's prediction results are never optimized, resulting in a waste of time. Furthermore, when the target edges are complex (such as between fingers), the model's annotation results do not fit the edges well, requiring additional time for manual adjustments.
[0043] In addition, in related technologies, after fine-tuning training, there is a phenomenon that the model cannot remove the wrong prediction regions even when given negative sample points. This indicates that the existing fine-tuning training strategy cannot perfectly adapt to real human-computer interaction scenarios.
[0044] In this application, the structure of the initial prediction model can be as follows: Figure 1 As shown, Figure 1 A schematic diagram of the MobileSAM structure can be provided. The initial image encoder is TinyViT, distilled from ViT-H. MobileSAM is based on the distillation of the SAM, which has a large number of parameters. Since the SAM series models already use a lightweight mask decoder, MobileSAM does not need to adjust this part of the structure. Regarding data, MobileSAM uses 1% of the original SAM training data as training data. Optionally, this application uses a simple fine-tuning strategy to fine-tune MobileSAM: during training, a low learning rate is used to update only the parameters of the mask decoder part, and the data uses first-view images captured by a binocular fisheye RGB camera, strictly aligned with the actual use scenario of the model.
[0045] In this scheme, adjusting the model parameters mainly refers to adjusting the parameters of the initial image encoder. This scheme proposes an iterative training method that focuses on simulating the real interaction process between the user and the model, and iteratively optimizes the model's output during this process. Experiments demonstrate that this method can significantly improve the model's performance during user interaction. By manually formulating a series of interaction strategies (such as deleting incorrect predictions from the model), interactive information that can be used as model input is generated in each iteration. Simultaneously, a dynamic judgment module is added after the model outputs the prediction result. When the model's prediction result fails to meet the target and the cumulative number of interactions is insufficient, new input information is generated based on the current output result, and the model re-infers and generates a new prediction result. When the model's prediction result meets the target or the cumulative number of interactions reaches the requirement, the training process for the current sample ends.
[0046] This application provides a data processing method applicable to any electronic device, comprising the following steps S201-S204:
[0047] S201. Input the sample data into the initial prediction model to obtain the first prediction result corresponding to the sample data;
[0048] Specifically, there are multiple sample data used to train the initial prediction model, and the sample data in S201 is one of them. In actual execution, a batch of sample data (including at least one sample data) can be input into the initial prediction model each time.
[0049] S202. Based on the target reference result corresponding to the sample data and the first prediction result, determine the prediction result analysis information of the first prediction result;
[0050] In some optional embodiments of this application, the prediction result analysis information includes first similarity information for indicating the similarity between the first prediction result and the target reference result.
[0051] Optionally, the function of the aforementioned initial prediction model is not limited in this application. For example, the initial prediction model can be a model that recognizes the language of text information in sample data and outputs the language recognition result, or it can be a model that recognizes the region where an object is located in an image in sample data and outputs the region occupied by the object in the image, such as a mask model.
[0052] S203. Determine whether the prediction result analysis information indicates that the first prediction result meets the first preset condition;
[0053] S204. If the first prediction result does not meet the first preset condition, the parameters of the initial prediction model are adjusted, and the initial prediction model is further trained based on the prediction result analysis information, the sample data, and the first prediction result to obtain the target prediction model corresponding to the initial prediction model. The target prediction model is used to output the corresponding target prediction result based on the input data to be processed.
[0054] Specifically, see Figure 2b The diagram shows a flowchart of the data processing method.
[0055] Optionally, when the value of the first similarity information is greater than the preset similarity value, or when the number of training iterations of the initial prediction model reaches the preset number, the first prediction result is considered to meet the first preset condition; otherwise, the first prediction result is considered not to meet the first preset condition.
[0056] The preset similarity value can be set by relevant personnel.
[0057] Optionally, when adjusting the parameters of the initial prediction model, the corresponding loss information can be determined based on the first prediction result and the target reference result according to a preset loss function, so as to adjust the parameters of the initial prediction model. For the principles of model training, please refer to the description of relevant technologies, which will not be repeated here.
[0058] In some optional embodiments of this application, in the aforementioned S204, the step of further training the initial prediction model based on the prediction result analysis information, the sample data, and the first prediction result to obtain the target prediction model corresponding to the initial prediction model includes S2041-S2044:
[0059] S2041. Determine the information to be input based on the prediction results analysis;
[0060] In some optional embodiments of this application, when the prediction result analysis information includes first similarity information indicating the similarity between the first prediction result and the target reference result, determining the input information based on the prediction result analysis information includes:
[0061] When the first similarity information is greater than the preset similarity value, the information to be input is determined to be a first prompt indicating that the first prediction result is close to the correct answer. When the first similarity information is not greater than the preset similarity value, the information to be input is determined to be a second prompt indicating that the first prediction result is not close to the correct answer.
[0062] Optionally, the first and second prompts can be text or symbols.
[0063] In some optional embodiments of this application, the sample data includes a sample image, the first region center coordinates of the target region occupied by the object to be labeled in the sample image, and the type of the target region.
[0064] The object to be labeled is the object in the sample image, and the target area occupied by the object to be labeled in the sample image is the image area occupied by the object to be labeled in the sample image.
[0065] The first region center coordinates refer to the two-dimensional coordinates of the center of the target region in the coordinate system of the sample image. The origin of the coordinate system of the sample image can be the pixel at the top left corner of the sample image.
[0066] Optionally, the region can be of either positive or negative sample type, and the target region can be of positive sample type.
[0067] Among them, when the region type is positive sample type, it means that the region belongs to the target region, and when the region type is negative sample type, it means that the region does not belong to the target region.
[0068] Optionally, when the sample data includes sample images, and the sample images include objects to be labeled, the initial prediction model is a model that labels the region where the objects to be labeled are located in the sample images and outputs the predicted region occupied by the objects to be labeled in the sample images.
[0069] In some optional embodiments of this application, in S2041, the step of determining the input information based on the prediction result analysis information includes the following S311-S312:
[0070] S311. Based on the prediction result analysis information, determine the first effective error region with the largest area in the first prediction result;
[0071] In this application, the effective area refers to an area whose area is greater than the preset area.
[0072] In some optional embodiments of this application, in S202, determining prediction result analysis information of the first prediction result based on the target reference result corresponding to the sample data and the first prediction result includes: matching the target reference result corresponding to the sample data with the first prediction result to obtain at least one valid error region in the first prediction result, wherein the prediction result analysis information includes at least one valid error region.
[0073] Each valid error region in at least one valid error region is of type positive sample or negative sample. A valid error region of type positive sample means that the valid error region should belong to the target region, but was not identified. A valid error region of type negative sample means that the valid error region does not belong to the target region, but was identified as belonging to the target region.
[0074] In the aforementioned S311, determining the first effective error region with the largest area in the first prediction result based on the prediction result analysis information includes: determining the first effective error region with the largest area among at least one effective error region included in the first prediction result based on the prediction result analysis information.
[0075] The first prediction result also includes the correct regions that were predicted correctly. The correct regions refer to the regions that belong to the target region and are identified as belonging to the target region.
[0076] S312. Use the center coordinates of the second region of the first valid error region and the type of the first valid error region as the input information.
[0077] The second region center coordinates refer to the region center coordinates of the area occupied by the first valid error region in the first prediction result. The type of the first valid error region is either positive sample type or negative sample type.
[0078] S2042. Input the information to be input, the sample data, and the first prediction result into the initial prediction model to obtain the second prediction result corresponding to the sample data;
[0079] S2043. Take the second prediction result as the new first prediction result;
[0080] S2044. Return to the step of determining the prediction result analysis information of the first prediction result based on the target reference result corresponding to the sample data and the first prediction result, until the prediction result analysis information indicates that the first prediction result meets the first preset condition, and take the initial prediction model obtained in the most recent training as the target prediction model corresponding to the initial prediction model.
[0081] Specifically, see Figure 2c The diagram shows a flowchart of the data processing method.
[0082] In some optional embodiments of this application, the method further includes: determining whether first similarity information used to indicate the similarity between the first prediction result and the target reference result is greater than second similarity information used to indicate the similarity between the third prediction result previously output by the initial prediction model and the target reference result; if so, triggering the step of determining whether the prediction result analysis information indicates that the first prediction result meets the first preset condition.
[0083] Specifically, see Figure 2d The diagram shows a flowchart of the data processing method.
[0084] In some optional embodiments of this application, the method further includes: if the first similarity information used to indicate the similarity between the first prediction result and the target reference result is not greater than the second similarity information used to indicate the similarity between the third prediction result previously output by the initial prediction model and the target reference result, then the error region with the largest area in the first prediction result is removed, and the step of determining the input information based on the prediction result analysis information is triggered.
[0085] The step of determining the first effective error region with the largest area in the first prediction result based on the prediction result analysis information includes: determining the first effective error region with the largest area in the first prediction result other than the first effective error region with the largest area based on the prediction result analysis information.
[0086] In S311, determining the first effective error region with the largest area in the first prediction result based on the prediction result analysis information includes: determining the first effective error region with the largest area in the first prediction result other than the largest error region based on the prediction result analysis information.
[0087] Optionally, when the prediction result analysis information indicates that the Mean IoU evaluation index of the first prediction result is greater than a preset threshold, or when the number of training iterations of the initial prediction model reaches a preset number, it is considered that the prediction result analysis information indicates that the first prediction result meets the first preset condition; otherwise, it is considered that the prediction result analysis information indicates that the first prediction result does not meet the first preset condition.
[0088] Optionally, the preset number of times can be 10, and the preset threshold can be 98%.
[0089] Furthermore, in this scheme, for the same sample data, N first prediction results can be output based on the same initial prediction model, and the best of the N results is taken as the final first prediction result for the sample data. Optionally, the number of sample data is not limited in this application; for example, the number of sample data can be 1000. N is any positive integer greater than or equal to 2.
[0090] In this application, the sample image can be a first-person arm image. The solution of this application can be applied to VR scenes, in which users need to interact with the scene screen through arm or other hand movements.
[0091] Furthermore, in this approach, observation and analysis of error regions revealed that the largest error regions often possess more significant characteristics and impact. Therefore, selecting the largest error region as the primary focus can more effectively improve the representativeness and accuracy of the samples. The center point typically better reflects the overall characteristics of the error region and aligns more closely with the operational habits of annotation personnel. This selection method reduces the randomness and uncertainty of the samples, improving the efficiency and accuracy of the algorithm.
[0092] Furthermore, the relationship between the first prediction result and the target reference result can be found in [reference needed]. Figure 2e As shown, the target reference result is the true value of the mask, and the first prediction result is the predicted value of the mask.
[0093] The training strategy proposed in this solution has no architectural conflicts with mainstream fine-tuning and data augmentation solutions in the industry. Both the training strategy and evaluation scheme proposed in this solution can be used simultaneously. By fine-tuning the MobileSAM model on the arm segmentation data annotation task, the performance of the MobileSAM model in real human-computer interaction scenarios has been significantly improved. After fine-tuning the MobileSAM model on the arm segmentation task using this solution, the accuracy metric improved by approximately 1.5 times: mIoU increased from 59.92% to 91.79%. The visualization comparison is as follows: the fine-tuned model has a larger range for the first point, and the final segmentation edges are also more closely aligned.
[0094] The proposed solution offers several significant advantages. First, it greatly reduces training time because the model has already been pre-trained on large-scale data, acquiring certain general features and knowledge. Second, fine-tuning effectively avoids overfitting because the model adaptively adjusts to new data during fine-tuning, rather than retraining. Furthermore, it enhances the model's generalization ability, enabling it to perform well not only on the original training set but also on new data scenarios through learning from the fine-tuned training set.
[0095] In practical applications, fine-tuning technology is widely used in image recognition, natural language processing, and speech recognition. For example, in image recognition tasks, a model pre-trained on the ImageNet dataset can be used and then fine-tuned on a specific image dataset to improve the accuracy of recognizing specific objects or scenes. In natural language processing, a pre-trained language model, such as GPT-3, can be used and then fine-tuned on a specific text dataset to perform tasks such as text classification and sentiment analysis.
[0096] The mainstream fine-tuning method uses data specific to a particular scenario, modifying only the parameters of the output layer (head part) while keeping the main weights of the large model (backbone part) unchanged. Another fine-tuning method adds an adapter module while keeping the original weights of the large model unchanged. During the training phase, only the newly added parameters are updated. Usually, the number of newly added parameters is less than the number of parameters in the head part, thus enabling low-cost fine-tuning of large models.
[0097] This application provides a scheme for inputting sample data into an initial prediction model to obtain a first prediction result corresponding to the sample data; determining prediction result analysis information of the first prediction result based on the target reference result corresponding to the sample data and the first prediction result; determining whether the prediction result analysis information indicates that the first prediction result meets a first preset condition; if the first prediction result does not meet the first preset condition, adjusting the parameters of the initial prediction model, and continuing to train the initial prediction model based on the prediction result analysis information, the sample data, and the first prediction result to obtain a target prediction model corresponding to the initial prediction model. The target prediction model is used to output a corresponding target prediction result based on the input data to be processed. This scheme introduces analysis information on the first prediction result, i.e., prediction result analysis information, and trains the initial prediction model based on the prediction result analysis information. The prediction result analysis information can be regarded as automatically labeled information for the first prediction result, saving labeling time and improving the efficiency of model training.
[0098] Figure 3 A schematic diagram of the structure of a data processing apparatus provided for an exemplary embodiment of this application;
[0099] The device includes:
[0100] Input unit 31 is used to input sample data into the initial prediction model to obtain the first prediction result corresponding to the sample data;
[0101] The determining unit 32 is used to determine the prediction result analysis information of the first prediction result based on the target reference result corresponding to the sample data and the first prediction result;
[0102] The determining unit 32 is further configured to determine whether the prediction result analysis information indicates that the first prediction result meets the first preset condition;
[0103] Training unit 33 is used to adjust the parameters of the initial prediction model if the first prediction result does not meet the first preset condition, and to continue training the initial prediction model based on the prediction result analysis information, the sample data and the first prediction result to obtain the target prediction model corresponding to the initial prediction model, wherein the target prediction model is used to output the corresponding target prediction result according to the input data to be processed.
[0104] In some optional embodiments of this application, the prediction result analysis information includes first similarity information for indicating the similarity between the first prediction result and the target reference result.
[0105] In some optional embodiments of this application, when the aforementioned apparatus is used to further train the initial prediction model based on the prediction result analysis information, the sample data, and the first prediction result to obtain the target prediction model corresponding to the initial prediction model, it is specifically used for:
[0106] Based on the analysis of the prediction results, the information to be input is determined;
[0107] The input information, the sample data, and the first prediction result are input into the initial prediction model to obtain the second prediction result corresponding to the sample data;
[0108] The second prediction result is used as the new first prediction result;
[0109] Return to the step of performing the prediction result analysis information based on the target reference result corresponding to the sample data and the first prediction result to determine the first prediction result, until the prediction result analysis information indicates that the first prediction result meets the first preset condition, and use the most recently trained initial prediction model as the target prediction model corresponding to the initial prediction model.
[0110] In some optional embodiments of this application, the sample data includes a sample image, the first region center coordinates of the target region occupied by the object to be labeled in the sample image, and the type of the target region.
[0111] In some optional embodiments of this application, when the aforementioned device is used to determine the input information based on the prediction result analysis information, it is specifically used for:
[0112] Based on the analysis of the prediction results, the first effective error region with the largest area in the first prediction results is determined;
[0113] The center coordinates of the second region of the first valid error region and the type of the first valid error region are used as the input information.
[0114] In some optional embodiments of this application, the aforementioned apparatus is further configured to: determine whether a first similarity information used to indicate the similarity between the first prediction result and the target reference result is greater than a second similarity information used to indicate the similarity between the third prediction result previously output by the initial prediction model and the target reference result; if so, trigger the execution of the step of determining whether the prediction result analysis information indicates that the first prediction result satisfies a first preset condition.
[0115] In some optional embodiments of this application, the aforementioned device is further configured to: if the first similarity information used to indicate the similarity between the first prediction result and the target reference result is not greater than the second similarity information used to indicate the similarity between the third prediction result previously output by the initial prediction model and the target reference result, then remove the error region with the largest area in the first prediction result and trigger the execution of the step of determining the input information based on the prediction result analysis information;
[0116] In some optional embodiments of this application, when the aforementioned device is used to determine the first effective error region with the largest area in the first prediction result based on the prediction result analysis information, it is specifically used to: determine the first effective error region with the largest area in the first prediction result other than the error region with the largest area based on the prediction result analysis information.
[0117] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, they will not be repeated here. Specifically, the device can execute the above method embodiments, and the foregoing and other operations and / or functions of each module in the device correspond to the corresponding processes in the various methods in the above method embodiments, which will not be repeated here for the sake of brevity.
[0118] The apparatus of this application embodiment has been described above from the perspective of functional modules in conjunction with the accompanying drawings. It should be understood that this functional module can be implemented in hardware, in software instructions, or in a combination of hardware and software modules. Specifically, the steps of the method embodiments in this application can be completed by integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in this application embodiment can be directly embodied as being executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. Optionally, the software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps in the above method embodiments.
[0119] Figure 4 This is a schematic block diagram of an electronic device provided in an embodiment of this application. The electronic device may include:
[0120] The system includes a memory 401 and a processor 402. The memory 401 stores computer programs and transfers the program code to the processor 402. In other words, the processor 402 can retrieve and run the computer programs from the memory 401 to implement the methods described in the embodiments of this application.
[0121] For example, the processor 402 can be used to execute the above-described method embodiments according to instructions in the computer program.
[0122] In some embodiments of this application, the processor 402 may include, but is not limited to:
[0123] General-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
[0124] In some embodiments of this application, the memory 401 includes, but is not limited to:
[0125] Volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).
[0126] In some embodiments of this application, the computer program may be divided into one or more modules, which are stored in the memory 401 and executed by the processor 402 to perform the method provided in this application. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device.
[0127] like Figure 4 As shown, the electronic device may also include:
[0128] Transceiver 403, which can be connected to processor 402 or memory 401.
[0129] The processor 402 can control the transceiver 403 to communicate with other devices; specifically, it can send information or data to other devices or receive information or data sent by other devices. The transceiver 403 may include a transmitter and a receiver. The transceiver 403 may further include antennas, and the number of antennas may be one or more.
[0130] It should be understood that the various components in the electronic device are connected through a bus system, which includes a data bus, a power bus, a control bus, and a status signal bus.
[0131] This application also provides a computer storage medium storing a computer program thereon, which, when executed by a computer, enables the computer to perform the methods of the above-described method embodiments. Alternatively, embodiments of this application also provide a computer program product containing instructions that, when executed by a computer, cause the computer to perform the methods of the above-described method embodiments.
[0132] When implemented using software, it can be implemented entirely or partially as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)).
[0133] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0134] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.
[0135] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. For example, the functional modules in the various embodiments of this application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.
[0136] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A data processing method, characterized in that, include: The sample data is input into the initial prediction model to obtain the first prediction result corresponding to the sample data; Based on the target reference result corresponding to the sample data and the first prediction result, the prediction result analysis information of the first prediction result is determined; Determine whether the prediction result analysis information indicates that the first prediction result meets the first preset condition; If the first prediction result does not meet the first preset condition, the parameters of the initial prediction model are adjusted, and the initial prediction model is trained again based on the prediction result analysis information, the sample data, and the first prediction result to obtain the target prediction model corresponding to the initial prediction model. The target prediction model is used to output the corresponding target prediction result based on the input data to be processed.
2. The method according to claim 1, characterized in that, The prediction result analysis information includes first similarity information used to indicate the similarity between the first prediction result and the target reference result.
3. The method according to claim 1, characterized in that, The step of further training the initial prediction model based on the prediction result analysis information, the sample data, and the first prediction result to obtain the target prediction model corresponding to the initial prediction model includes: Based on the analysis of the prediction results, the information to be input is determined; The input information, the sample data, and the first prediction result are input into the initial prediction model to obtain the second prediction result corresponding to the sample data; The second prediction result is used as the new first prediction result; Return to the step of performing the prediction result analysis information based on the target reference result corresponding to the sample data and the first prediction result to determine the first prediction result, until the prediction result analysis information indicates that the first prediction result meets the first preset condition, and use the most recently trained initial prediction model as the target prediction model corresponding to the initial prediction model.
4. The method according to claim 3, characterized in that, The sample data includes a sample image, the center coordinates of the first region of the target area occupied by the object to be labeled in the sample image, and the type of the target area.
5. The method according to claim 4, characterized in that, The step of determining the input information based on the prediction results analysis includes: Based on the analysis of the prediction results, the first effective error region with the largest area in the first prediction results is determined; The center coordinates of the second region of the first valid error region and the type of the first valid error region are used as the input information.
6. The method according to claim 5, characterized in that, The method further includes: If the first similarity information used to indicate the similarity between the first prediction result and the target reference result is greater than the second similarity information used to indicate the similarity between the third prediction result previously output by the initial prediction model and the target reference result, then the step of determining whether the prediction result analysis information indicates that the first prediction result meets the first preset condition is triggered.
7. The method according to claim 5, characterized in that, The method further includes: If the first similarity information used to indicate the similarity between the first prediction result and the target reference result is not greater than the second similarity information used to indicate the similarity between the third prediction result output by the initial prediction model and the target reference result, then the error region with the largest area in the first prediction result is removed, and the step of determining the input information based on the prediction result analysis information is triggered. The step of determining the first effective error region with the largest area in the first prediction result based on the prediction result analysis information includes: determining the first effective error region with the largest area in the first prediction result other than the first effective error region with the largest area based on the prediction result analysis information.
8. A data processing apparatus, characterized in that, include: An input unit is used to input sample data into an initial prediction model to obtain a first prediction result corresponding to the sample data; The determining unit is used to determine the prediction result analysis information of the first prediction result based on the target reference result corresponding to the sample data and the first prediction result; The determining unit is further configured to determine whether the prediction result analysis information indicates that the first prediction result meets the first preset condition; The training unit is configured to adjust the parameters of the initial prediction model if the first prediction result does not meet the first preset condition, and continue to train the initial prediction model based on the prediction result analysis information, the sample data, and the first prediction result to obtain the target prediction model corresponding to the initial prediction model. The target prediction model is configured to output the corresponding target prediction result based on the input data to be processed.
9. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the method of any one of claims 1-7 by executing the executable instructions.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-7.