Model determination method and apparatus

Abstract

The method comprises the following steps: determining loss values corresponding to a plurality of training samples respectively according to a computer vision model and a loss function, the training samples being image samples or video samples; performing screening processing on the loss values corresponding to the plurality of training samples respectively to obtain target loss values, the target loss values being loss values located in a middle part among the loss values corresponding to the plurality of training samples respectively; and adjusting parameters of the computer vision model based on the target loss values. Thus, by means of loss value screening, the influence of dirty data on model training is reduced, and the convergence speed of the computer vision model is prevented from being reduced due to too small loss values, thereby improving the model quality and convergence speed of the computer vision model.

Description

Technical Field

[0001] This disclosure relates to the field of computer vision, and more particularly to a model determination method and apparatus. Background Technology

[0002] The training process of a computer vision model is essentially the process of learning the model parameters.

[0003] Typically, during training, the model output and loss function obtained after inputting multiple training samples into the computer vision model are combined to determine the loss value corresponding to the multiple training samples. The model parameters are then updated based on the average of the loss values ​​corresponding to the multiple training samples.

[0004] However, in the above training methods, how to balance the convergence speed and model quality of computer vision models is one of the key issues that urgently needs to be addressed. Summary of the Invention

[0005] This disclosure provides a model determination method and apparatus to balance the convergence speed and model quality of computer vision models.

[0006] In a first aspect, embodiments of this disclosure provide a model determination method, including:

[0007] Based on the computer vision model and loss function, the loss values ​​corresponding to multiple training samples are determined respectively, wherein the training samples are image samples or video samples;

[0008] The loss values ​​corresponding to the multiple training samples are filtered to obtain the target loss value, which is the loss value in the middle part among the loss values ​​corresponding to the multiple training samples.

[0009] Based on the target loss value, the parameters of the computer vision model are adjusted.

[0010] Secondly, embodiments of this disclosure provide a data processing method, including:

[0011] Acquire data to be processed, which may be image data or video data;

[0012] The data to be processed is processed by a computer vision model, which is trained using the model determination method described in the first aspect or various possible designs of the first aspect.

[0013] Thirdly, embodiments of this disclosure provide a model determination device, comprising:

[0014] The determining unit is used to determine the loss value corresponding to multiple training samples based on the computer vision model and the loss function, wherein the training samples are image samples or video samples;

[0015] A filtering unit is used to filter the loss values ​​corresponding to the plurality of training samples respectively to obtain a target loss value, wherein the target loss value is the loss value located in the middle part among the loss values ​​corresponding to the plurality of training samples respectively;

[0016] The adjustment unit is used to adjust the parameters of the computer vision model based on the target loss value.

[0017] Fourthly, embodiments of this disclosure provide a data processing apparatus, including:

[0018] An acquisition unit is used to acquire data to be processed, wherein the data to be processed is image data or video data;

[0019] The processing unit is configured to process the data to be processed using a computer vision model, wherein the computer vision model is trained using the model determination method described in the first aspect or various possible designs of the first aspect.

[0020] Fifthly, embodiments of this disclosure provide an electronic device, including: at least one processor and a memory. The memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, causing the at least one processor to perform a model determination method as described in the first aspect or various possible designs of the first aspect, or to perform a data processing method as described in the second aspect or various possible designs of the second aspect.

[0021] In a sixth aspect, embodiments of this disclosure provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the model determination method as described in the first aspect or various possible designs of the first aspect, or implement the data processing method as described in the second aspect or various possible designs of the second aspect.

[0022] In a seventh aspect, embodiments of this disclosure provide a computer program product comprising computer execution instructions, which, when executed by a processor, implement the model determination method as described in the first aspect or various possible designs of the first aspect, or implement the data processing method as described in the second aspect or various possible designs of the second aspect.

[0023] In the model determination method and device provided in this embodiment, the loss values ​​corresponding to multiple training samples are determined according to the computer vision model and the loss function. The loss values ​​corresponding to the multiple training samples are then filtered to obtain the target loss value. The target loss value is the loss value located in the middle part among the loss values ​​corresponding to the multiple training samples. Based on the target loss value, the parameters of the computer vision model are adjusted.

[0024] Therefore, in the training process of a computer vision model, firstly, some large loss values ​​are filtered out. These loss values ​​are likely to be from dirty training samples. Filtering out these loss values ​​helps reduce the impact of dirty data loss values ​​on the computer vision model's errors, thus improving the model quality. Secondly, some small loss values ​​are filtered out. These loss values ​​have little impact on the computer vision model. Filtering out these loss values ​​helps improve the convergence speed of the computer vision model, thus improving the training efficiency. In this way, both model quality and convergence speed are balanced. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 A schematic diagram illustrating an application scenario provided by an embodiment of this disclosure;

[0027] Figure 2 Flowchart of the model determination method provided in the embodiments of this disclosure Figure 1 ;

[0028] Figure 3 Flowchart of the model determination method provided in the embodiments of this disclosure Figure 2 ;

[0029] Figure 4 Flowchart of the model determination method provided in the embodiments of this disclosure Figure 3 ;

[0030] Figure 5 A schematic flowchart illustrating the data processing method provided in this embodiment of the disclosure;

[0031] Figure 6 The structural block diagram of the model determination device provided in the embodiments of this disclosure;

[0032] Figure 7This is a structural block diagram of a data processing device provided in an embodiment of the present disclosure;

[0033] Figure 8 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0035] Computer vision models typically employ neural network models to process image or video data. During the training process of a neural network model, a loss function is used to determine the loss values ​​of multiple training samples. Based on the loss values ​​of these training samples, the gradients backpropagated from each network layer are determined (hereafter referred to as backpropagation gradients for ease of indirect description). The parameters of each network layer are then updated based on these backpropagation gradients.

[0036] Typically, to accelerate the convergence of computer vision models, the larger loss values ​​from multiple training samples are selected for parameter tuning of the computer vision model.

[0037] If the training samples contain a large amount of dirty data (also known as bad data, which refers to incorrectly labeled training samples, such as image samples of vehicles in a face recognition task), then a significant portion of the larger loss values ​​will correspond to the losses from the dirty data. In this case, the above method has the following problems:

[0038] 1) If the number of loss values ​​selected in the above method is small, such as selecting the first 10% of loss values, then dirty data plays a decisive role in the direction of backpropagation gradient, the loss function converges in the wrong direction, the model cannot fit well, and the quality of the finally trained model is poor.

[0039] 2) If a large number of loss values ​​are selected using the above method (e.g., selecting the top 50% of loss values), the model's convergence speed will slow down. Especially in the later stages of training when the computer vision model is close to convergence, the gradients of 90% of the selected loss values ​​may be close to 0, and only the gradients of 10% of the loss values ​​will have an effect. This causes the overall backpropagation gradient to approach 0, making it difficult for the computer vision model to find a suitable gradient descent direction with a large step size, thus reducing the convergence speed.

[0040] To address the aforementioned problems, this disclosure provides a model determination method and apparatus. In this method, from the loss values ​​corresponding to multiple training samples, the loss value located in the middle range is selected as the target loss value. Based on the target loss value, the parameters of the computer vision model are adjusted. This process filters out both some large and small loss values, eliminating interference from some dirty data in the training of the computer vision model and preventing the parameter adjustment step size from becoming too small when there are many small loss values, thereby improving the model quality and convergence speed of the computer vision model.

[0041] The computer vision model can be an image processing model (e.g., an image recognition model, an image segmentation model, an image stitching model, an image classification model) or a video processing model (e.g., a moving target detection model in a video, a moving target tracking model in a video, a video classification model).

[0042] The method and apparatus are based on the same concept of the application. Since the methods and apparatus solve problems in similar ways, their implementations can be referred to each other, and repeated parts will not be described again.

[0043] refer to Figure 1 , Figure 1 This is a schematic diagram illustrating an application scenario provided by an embodiment of this disclosure.

[0044] exist Figure 1 In the application scenario shown, the devices involved include a model training device 101 and a model application device 102. The model training device 101 and the model application device 102 can be the same device or different devices. When the model training device 101 and the model application device 102 are different devices, they can communicate, for example, via a network. The model training device 101 is used to train a computer vision model, and the model application device 102 is used to process data based on the trained computer vision model.

[0045] On the model training device 101, the computer vision model is trained multiple times using image data or video data as training data to obtain the trained computer vision model. On the model application device 102, for example, image data or video data input by a user is received, and the image data or video data is processed by the computer vision model. For example, in an image target recognition scenario, the model application device 102 performs target recognition on image data from a user or camera device using the computer vision model; as another example, in a moving target tracking scenario in a video, the model application device 102 performs target tracking on video data from a user or camera device using the computer vision model.

[0046] The model training device 101 and the model application device 102 can be electronic devices with data processing capabilities, such as terminals and servers. Figure 1 Taking the model training device 101 as the server and the model application device 102 as the terminal as an example.

[0047] For example, the execution subject of the model determination method provided in this embodiment is an electronic device, which is a terminal or a server.

[0048] The terminal can be a personal digital assistant (PDA) device, a handheld device with wireless communication capabilities (such as a smartphone or tablet), a computing device (such as a personal computer (PC)), an in-vehicle device, a wearable device (such as a smartwatch or smart bracelet), or a smart home device (such as a smart display device). The server can be a distributed server, a centralized server, or a cloud server.

[0049] refer to Figure 2 , Figure 2 Flowchart of the model determination method provided in the embodiments of this disclosure Figure 1 .

[0050] like Figure 2 As shown, the model determination method includes:

[0051] S201. Based on the computer vision model and loss function, determine the loss values ​​corresponding to multiple training samples, where the training samples are image samples or video samples.

[0052] The training of the computer vision model involves multiple iterations. In each training iteration, the model parameters are adjusted based on the training samples, and the training process is repeated using the adjusted model. This embodiment represents one training iteration of the computer vision model.

[0053] The loss function is used to calculate the loss value; for example, the cross-entropy loss function or the squared error loss function. No specific limitation is made to the loss function here. The loss function serves as the objective function, and the computer vision model is trained in the direction of continuously optimizing the loss function (i.e., continuously minimizing the loss value).

[0054] The training samples can be either image samples or video samples. When the training samples are image samples, for example, multiple images and their corresponding labeled data are pre-collected, and these images and their labeled data are used as image samples. In face recognition tasks, the labeled data can be images with bounded boxes around the faces; in super-resolution image reconstruction tasks, the labeled data can be high-resolution images. When the training samples are video samples, for example, multiple videos and their corresponding labeled data are pre-collected, and these videos and their labeled data are used as video samples. In moving target tracking tasks, the labeled data can be videos with bounded boxes around the tracked targets; in video classification tasks, the labeled data can be the category corresponding to the video.

[0055] In this step, multiple training samples are acquired during a single training iteration of the computer vision model. For each training sample, the computer vision model processes it to obtain its output data. Based on the output data, the label data from the training samples, and the loss function, the loss value for each training sample is calculated. For example, the loss value for a given training sample is determined by the difference between the output data of the computer vision model and the label data in the training samples, calculated using the loss function. This process yields the loss values ​​for each of the multiple training samples.

[0056] S202. Filter the loss values ​​corresponding to multiple training samples to obtain the target loss value, which is the loss value in the middle part among the loss values ​​corresponding to multiple training samples.

[0057] The target loss value is the middle value among the loss values ​​corresponding to multiple training samples. Specifically, it refers to the loss value that is in the middle range based on its numerical value among the loss values ​​corresponding to multiple training samples. For example, if multiple training samples include a1 to a10, and their corresponding loss values ​​are 10, 6, 8, 5, 3, 2, 2, 1, 8, and 7 respectively, then the target loss value could be (3, 5, 6, 7) or (3, 5), which are in the middle range. This allows us to eliminate some excessively large loss values ​​that might be caused by dirty data, and also eliminate some excessively small loss values ​​that would slow down the convergence speed of the computer vision model.

[0058] In this step, after obtaining the loss values ​​corresponding to multiple training samples, the target loss value located in the middle part of these loss values ​​is selected from the loss values ​​corresponding to multiple training samples.

[0059] In one example, the median or mean of the loss values ​​corresponding to multiple training samples is determined. Based on the median or mean, the range of values ​​for the target loss value is determined. For example, the absolute value of the difference between the maximum value and the median or mean in the range is less than a preset threshold, and the absolute value of the difference between the minimum value and the median or mean in the range is less than a preset threshold. Based on the range of values ​​for the target loss value, the target loss value is selected from the loss values ​​corresponding to the multiple training samples.

[0060] In another example, the loss values ​​corresponding to multiple training samples are sorted. From these sorted loss values, loss values ​​are selected to the left and right of the middle loss value, until the number of selected loss values ​​reaches a threshold. The target loss value is then determined to be one of these selected loss values.

[0061] In another example, the maximum and minimum values ​​of the target loss can be preset. After obtaining the loss values ​​corresponding to multiple training samples, the loss values ​​corresponding to the multiple training samples can be filtered according to the maximum and minimum values. Loss values ​​greater than the maximum value are filtered out, and loss values ​​less than the minimum value are filtered out, thus determining the target loss value as the remaining loss value.

[0062] S203. Adjust the parameters of the computer vision model based on the target loss value.

[0063] The number of target loss values ​​can be one or more.

[0064] In this step, after obtaining the target loss value, the backpropagation gradients of multiple model parameters in the computer vision model can be determined based on all target loss values. Based on these backpropagation gradients, the model parameters of the network layers are adjusted. This completes one training iteration of the computer vision model. Then, based on the parameter-adjusted computer vision model, the next training iteration can begin. In this next training iteration, the same loss value selection method can be used to select the loss value located in the middle of the loss values ​​corresponding to multiple training samples for parameter adjustment. The condition for whether the computer vision model continues training is, for example, whether the target loss value is less than a preset threshold. If it is, training does not continue; otherwise, training continues. There are no restrictions on the iteration conditions of the computer vision model.

[0065] In the model determination method provided in this embodiment, loss values ​​corresponding to multiple training samples are determined based on the computer vision model and the loss function. Among the loss values ​​corresponding to the multiple training samples, loss values ​​with a value in the middle range are selected for parameter adjustment of the computer vision model. This removes some loss values ​​that are too large due to dirty data and some loss values ​​that are too small, thus avoiding dirty data from misleading the model training direction and loss values ​​that are too small from reducing the model convergence speed, thereby improving the model quality and training efficiency of the computer vision model.

[0066] refer to Figure 3 , Figure 3 Flowchart of the model determination method provided in the embodiments of this disclosure Figure 2 .

[0067] like Figure 3 As shown, the model determination method includes:

[0068] S301. Based on the computer vision model and loss function, determine the loss values ​​corresponding to multiple training samples, where the training samples are image samples or video samples.

[0069] The implementation principle and technical effect of S301 can be referred to the aforementioned embodiments, and will not be repeated here.

[0070] S302. Sort the loss values ​​corresponding to multiple training samples according to their numerical values.

[0071] In this step, the loss values ​​corresponding to multiple training samples can be sorted in descending order of numerical value to obtain multiple loss values ​​sorted from largest to smallest. Alternatively, the loss values ​​corresponding to multiple training samples can be sorted in ascending order of numerical value to obtain multiple loss values ​​sorted from smallest to largest.

[0072] S303. The target loss value is determined as the loss value that is located after the N% loss value and before the M% loss value in the sorted multiple loss values, where N is less than M.

[0073] Here, N and M are two different thresholds.

[0074] When multiple loss values ​​are sorted in descending order, the first N% of loss values ​​are likely to include excessively large loss values ​​caused by dirty data, while the loss values ​​after the first M% are too small, which will slow down the convergence speed of the computer vision model. Therefore, the sorted loss values ​​can be filtered based on N% and M% to obtain the target loss values ​​in the middle range.

[0075] In this step, among the sorted loss values, based on the total number of these loss values, the loss values ​​that fall after N% of the loss values ​​and before M% of the loss values ​​are selected as target loss values. For example, if the total number of sorted loss values ​​is 100, N is 10, and M is 30, then from the beginning, the loss values ​​that fall after 10% of the loss values ​​and before 30% of the loss values ​​are selected as target loss values. That is, the target loss values ​​are the loss values ​​between the 10th and 30th loss values ​​(inclusive).

[0076] To put it another way, from the sorted loss values, we filter out the first (N% * total number of loss values ​​- 1) loss values ​​and the last (total number of loss values ​​- M% * total number of loss values) loss values. For example, if the total number of sorted loss values ​​is 100, N is 10, and M is 30, we can filter out the first 9 loss values ​​and the last 70 loss values. The target loss value is then determined as the loss value remaining after filtering from the sorted loss values.

[0077] S304. Adjust the parameters of the computer vision model based on the target loss value.

[0078] The implementation principle and technical effect of S304 can be referred to the aforementioned embodiments, and will not be repeated here.

[0079] In the model determination method provided in this embodiment, the loss values ​​corresponding to multiple training samples are sorted. Among the sorted loss values, the target loss value is determined to be the loss value located after the N% and in the M% from the beginning. Thus, by sorting and using percentage thresholds, loss values ​​that may be caused by dirty data and excessively large loss values ​​are removed, as well as some loss values ​​that are too small. This avoids dirty data misleading the model training direction and excessively small loss values ​​reducing the model convergence speed, thereby improving the model quality and training efficiency of the computer vision model.

[0080] In some embodiments, N and / or M can be flexibly adjusted so that different values ​​of N and / or M can be set to accommodate different training samples used by different computer vision models, or to flexibly adjust the values ​​of N and / or M to accommodate different training samples used by the same computer vision model in different training iterations, thereby improving the training effect of the computer vision model.

[0081] Based on N and / or M, which can be flexibly adjusted, refer to... Figure 4 , Figure 4 Flowchart of the model determination method provided in the embodiments of this disclosure Figure 3 .like Figure 4 As shown, the model determination method includes:

[0082] S401. Determine the training set in the current training iteration. The training set includes multiple training samples.

[0083] The training set used in different training iterations can be different. When training for the current iteration, the training set used for that iteration is obtained.

[0084] For example, the training set for the current training iteration can be obtained from a database or collected from the internet. Different training sets contain different training samples. For instance, when the training samples are image samples, the training set includes multiple image samples, and the image samples in different training sets may be partially or completely different; when the training samples are video samples, the training set includes multiple video samples, and the video samples in different training sets may be partially or completely different.

[0085] S402. Adjust N and / or M according to the training set.

[0086] In this scenario, assuming the subsequent sorting involves ranking the multiple loss values ​​from largest to smallest, N is used to filter out excessively large loss values ​​caused by dirty data, while M is used to filter out some excessively small loss values. Therefore, N can be adjusted based on the amount of dirty data in the training set, and / or M can be adjusted based on the number of training samples in the training set.

[0087] In one example, N is adjusted according to the proportion of dirty data in the training set to adapt to the proportion of dirty data. Adjusting N adjusts the degree of filtering of loss values ​​caused by dirty data.

[0088] Specifically, the proportion of dirty data in the training set is known. Once the proportion of dirty data in the training set is known, N can be adjusted based on this proportion.

[0089] In cases where the proportion of dirty data is larger, to avoid excessive dirty data misleading the training of computer vision models, N can be increased to filter out more loss values ​​that are too large, i.e., to filter out more loss values ​​that may be caused by dirty data. In cases where the proportion of dirty data is smaller, considering that large loss values ​​play a greater role in the training process of computer vision models, N can be decreased to filter out only a small number of loss values ​​that are too large.

[0090] In one approach, a mapping relationship between the proportion of dirty data and N is pre-defined. In each training iteration, N corresponding to the proportion of dirty data in the training set during the current training iteration is determined within this mapping relationship.

[0091] In another approach, the percentage of dirty data in the training set during previous training iterations (e.g., the last training iteration) can be recorded. If the percentage of dirty data in the training set increases compared to previous iterations, then N is increased; otherwise, N is decreased. Thus, by leveraging the property that a larger N filters out more loss values ​​caused by dirty data, when the percentage of dirty data in the training set increases, increasing N helps to filter out as many loss values ​​as possible caused by dirty data, minimizing the misleading influence of dirty data on the training of the computer vision model; conversely, when the percentage of dirty data in the training set decreases, decreasing N avoids missing more high-value loss values.

[0092] In one example, M is adjusted based on the total number of training samples in the training set. Thus, by adjusting M, the degree of filtering for loss values ​​that are too small is adjusted to suit the total number of training samples in the training set. Specifically, when the total number of training samples is larger, to accelerate the convergence speed of the computer vision model, M can be decreased to filter out more loss values ​​that are too small; conversely, when the total number of training samples is smaller, to ensure the quality of the computer vision model, M can be increased to ensure that there are enough loss values ​​available for parameter tuning of the computer vision model.

[0093] In one approach, a mapping relationship between the total number of training samples and N is pre-set. In each training iteration, N corresponding to the total number of training samples in the training set in the current training iteration is determined within this mapping relationship.

[0094] In another approach, the total number of training samples in the training set during previous training iterations (e.g., the last training iteration) can be recorded. If the total number of training samples in the current training iteration increases compared to the total number of training samples in the previous iterations, then M is decreased; otherwise, M is increased. Thus, by utilizing the property that a smaller M filters out more excessively small loss values, when the total number of training samples in the training set increases, a smaller M filters out as many excessively small loss values ​​as possible, reducing the impact of small loss values ​​on the convergence speed of the computer vision model. When the total number of training samples in the training set decreases, N is increased to ensure sufficient loss values ​​for parameter tuning of the computer vision model, thus ensuring the model quality.

[0095] S403. Based on the computer vision model and loss function, determine the loss values ​​corresponding to multiple training samples, where the training samples are image samples or video samples.

[0096] S404. Sort the loss values ​​corresponding to multiple training samples according to their numerical values.

[0097] S405. The target loss value is determined as the loss value that is located after the N% loss value and before the M% loss value in the sorted multiple loss values, where N is less than M.

[0098] S406. Adjust the parameters of the computer vision model based on the target loss value.

[0099] The implementation principles and technical effects of S403 to S406 can be referred to in the aforementioned embodiments, and will not be repeated here.

[0100] In the model determination method provided in this disclosure, N and / or M are adjusted for the training set in the current training iteration. During the training process of the current iteration, loss values ​​corresponding to multiple samples are filtered based on N and M, and then the parameters of the computer vision model are adjusted based on the filtered loss values. Thus, by flexibly adjusting N and / or M, the accuracy of loss value filtering in each training iteration is improved, better balancing the model quality and training efficiency of the computer vision model.

[0101] Based on any of the foregoing embodiments, in one possible implementation, parameter adjustment of the computer vision model based on the target loss value includes: determining the backpropagation gradient of the computer vision model based on the arithmetic mean or weighted average of the target loss values; and adjusting the parameters of the computer vision model based on the backpropagation gradient to improve the parameter adjustment effect of the computer vision model.

[0102] In the process of determining the backpropagation gradient based on the arithmetic mean or weighted average of the target loss values, it is calculated based on the backpropagation gradient calculation formula of each parameter in the computer vision model. Here, there are no restrictions on the backpropagation gradient calculation formula.

[0103] After training, a computer vision model can be used for image processing or video processing. (Reference) Figure 5 , Figure 5 This is a schematic flowchart illustrating the data processing method provided in an embodiment of this disclosure. Figure 5 As shown, the data processing method includes:

[0104] S501. Obtain the data to be processed, which may be image data or video data.

[0105] For example, the execution entity of the data processing method can be the same device as the execution entity of the model determination method, or they can be different devices. A computer vision model can be trained first using the model determination method, and then the computer vision model can be used for data processing in the data processing method. The training process of the computer vision model and the data processing process can also be carried out concurrently.

[0106] In this step, the device can receive user-inputted data, retrieve data from a database, or receive data sent from third-party devices. For example, the device may receive image or video data captured by a camera.

[0107] S502. The data to be processed is processed using a computer vision model, wherein the computer vision model is trained using the model determination method provided in any of the foregoing embodiments.

[0108] In this step, a computer vision model is used to process the image or video data to be processed. For example, feature extraction and feature matching are performed on image data to identify facial targets. Similarly, feature extraction and feature matching are performed on image frames in video data to identify and track human targets. Because the training process of the computer vision model improves its quality by selecting target loss values ​​that fall in the middle of the loss values ​​from multiple training samples, the accuracy of data processing is naturally improved when using the computer vision model for data processing.

[0109] In the model determination method provided in this disclosure, the computer vision model trained by the model determination method provided in any of the foregoing embodiments is used to process image data or video data, which effectively improves the accuracy of data processing.

[0110] Corresponding to the model determination method in the above embodiments, Figure 6 A structural block diagram of the model determining device provided for embodiments of this disclosure is shown. For ease of explanation, only the parts relevant to embodiments of this disclosure are shown. (Refer to...) Figure 6 The model determination device includes: a determination unit 601, a screening unit 602, and an adjustment unit 603.

[0111] The determining unit 601 is used to determine the loss values ​​corresponding to multiple training samples based on the computer vision model and the loss function. The training samples are image samples or video samples.

[0112] The filtering unit 602 is used to filter the loss values ​​corresponding to multiple training samples to obtain the target loss value, which is the loss value located in the middle part among the loss values ​​corresponding to multiple training samples.

[0113] The adjustment unit 603 is used to adjust the parameters of the computer vision model based on the target loss value.

[0114] In one embodiment of this disclosure, the filtering unit 602 is specifically used to: sort the loss values ​​corresponding to multiple training samples according to their numerical order; and determine the target loss value as the loss value that is located after the N% loss value and before the M% loss value in the sorted multiple loss values, where N is less than M.

[0115] In one embodiment of this disclosure, the model determination device further includes: an initialization unit 604, configured to determine the training set in the current training iteration, the training set including multiple training samples; and to adjust N and / or M according to the training set.

[0116] In one embodiment of this disclosure, the initialization unit 604 is specifically used to: adjust N according to the proportion of dirty data in the training set.

[0117] In one embodiment of this disclosure, the initialization unit 604 is specifically used to: increase N if the proportion of dirty data in the training set increases, otherwise decrease N.

[0118] In one embodiment of this disclosure, the initialization unit 604 is specifically used to: adjust M according to the total number of training samples in the training set.

[0119] In one embodiment of this disclosure, the initialization unit 604 is specifically used to: decrease M if the total number of training samples in the training samples increases, otherwise increase M.

[0120] In one embodiment of this disclosure, the adjustment unit 603 is specifically used to: determine the backpropagation gradient of the computer vision model based on the arithmetic mean or weighted average of the target loss values; and adjust the parameters of the computer vision model based on the backpropagation gradient.

[0121] The model determination device provided in this embodiment can be used to execute the technical solution of the above-described model determination method embodiment. Its implementation principle and technical effect are similar, and will not be described again in this embodiment.

[0122] Corresponding to the data processing method in the above embodiments, Figure 7 This is a structural block diagram of a data processing apparatus provided in an embodiment of the present disclosure. For ease of explanation, only the parts relevant to the embodiments of the present disclosure are shown. (Refer to...) Figure 7 The data processing device includes: an acquisition unit 701 and a processing unit 702.

[0123] The acquisition unit 701 is used to acquire data to be processed, which is image data or video data.

[0124] The processing unit 702 is used to process the data to be processed through a computer vision model, which is trained using the model determination method provided in any of the foregoing embodiments.

[0125] The data processing device provided in this embodiment can be used to execute the technical solutions of the above data processing method embodiments. Its implementation principle and technical effects are similar, and will not be described again here.

[0126] refer to Figure 8 The diagram illustrates a structural schematic of an electronic device 800 suitable for implementing embodiments of the present disclosure. The electronic device 800 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, personal digital assistants (PDAs), portable Android devices (PADs), portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 8 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0127] like Figure 8As shown, the electronic device 800 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 into a random access memory (RAM) 803. The RAM 803 also stores various programs and data required for the operation of the electronic device 800. The processing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0128] Typically, the following devices can be connected to I / O interface 805: input devices 806 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 807 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 808 including, for example, magnetic tapes, hard disks, etc.; and communication devices 809. Communication device 809 allows electronic device 800 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 8 An electronic device 800 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0129] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 809, or installed from a storage device 808, or installed from a ROM 802. When the computer program is executed by a processing device 801, it performs the functions defined in the methods of embodiments of this disclosure.

[0130] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0131] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0132] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.

[0133] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0134] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0135] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0136] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0137] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0138] In a first aspect, according to one or more embodiments of this disclosure, a model determination method is provided, comprising: determining loss values ​​corresponding to multiple training samples respectively based on a computer vision model and a loss function, wherein the training samples are image samples or video samples; filtering the loss values ​​corresponding to the multiple training samples respectively to obtain a target loss value, wherein the target loss value is the loss value located in the middle part among the loss values ​​corresponding to the multiple training samples respectively; and adjusting the parameters of the computer vision model based on the target loss value.

[0139] According to one or more embodiments of this disclosure, the step of filtering the loss values ​​corresponding to the plurality of training samples to obtain a target loss value includes: sorting the loss values ​​corresponding to the plurality of training samples according to their numerical order; determining the target loss value as the loss value that is located after the N% loss value and before the M% loss value in the sorted plurality of loss values, where N is less than M.

[0140] According to one or more embodiments of this disclosure, before determining the loss values ​​corresponding to the plurality of training samples based on the computer vision model and the loss function, the method further includes: determining the training set in the current training iteration, the training set including the plurality of training samples; and adjusting the N and / or the M based on the training set.

[0141] According to one or more embodiments of this disclosure, adjusting N and / or M based on the training set includes: adjusting N based on the proportion of dirty data in the training set.

[0142] According to one or more embodiments of this disclosure, adjusting N based on the proportion of dirty data in the training set includes: increasing N if the proportion of dirty data in the training set increases, and decreasing N otherwise.

[0143] According to one or more embodiments of this disclosure, adjusting N and / or M based on the training set includes: adjusting M based on the total number of training samples in the training set.

[0144] According to one or more embodiments of this disclosure, adjusting M based on the total number of training samples in the training set includes: decreasing M if the total number of training samples in the training set increases, and increasing M otherwise.

[0145] According to one or more embodiments of this disclosure, adjusting the parameters of the computer vision model based on the target loss value includes: determining the backpropagation gradient of the computer vision model based on the arithmetic mean or weighted average of the target loss values; and adjusting the parameters of the computer vision model based on the backpropagation gradient.

[0146] Secondly, according to one or more embodiments of this disclosure, a data processing method is provided, comprising: acquiring data to be processed, the data to be processed being image data or video data; and processing the data to be processed using a computer vision model, wherein the computer vision model is trained using the model determination method described in the first aspect or various possible designs of the first aspect.

[0147] Thirdly, according to one or more embodiments of this disclosure, a model determination device is provided, comprising: a determination unit, configured to determine loss values ​​corresponding to multiple training samples respectively based on a computer vision model and a loss function, wherein the training samples are image samples or video samples; a filtering unit, configured to filter the loss values ​​corresponding to the multiple training samples respectively to obtain a target loss value, wherein the target loss value is the loss value located in the middle part among the loss values ​​corresponding to the multiple training samples respectively; and an adjustment unit, configured to adjust the parameters of the computer vision model based on the target loss value.

[0148] According to one or more embodiments of this disclosure, the filtering unit is specifically used to: sort the loss values ​​corresponding to the plurality of training samples in numerical order; and determine the target loss value as the loss value that is located after the N% loss value and before the M% loss value in the sorted plurality of loss values, wherein N is less than M.

[0149] According to one or more embodiments of this disclosure, the model determination apparatus further includes: an initialization unit, configured to determine a training set in the current training iteration, the training set including the plurality of training samples; and to adjust N and / or M according to the training set.

[0150] According to one or more embodiments of this disclosure, the initialization unit is specifically used to: adjust N according to the proportion of dirty data in the training set.

[0151] According to one or more embodiments of this disclosure, the initialization unit is specifically used to: increase N if the proportion of dirty data in the training set increases, otherwise decrease N.

[0152] According to one or more embodiments of this disclosure, the initialization unit is specifically used to: adjust M according to the total number of training samples in the training set.

[0153] According to one or more embodiments of this disclosure, the initialization unit is specifically configured to: decrease M if the total number of training samples in the training samples increases, otherwise increase M.

[0154] According to one or more embodiments of this disclosure, the adjustment unit is specifically configured to: determine the backpropagation gradient of the computer vision model based on the arithmetic mean or weighted average of the target loss values; and adjust the parameters of the computer vision model based on the backpropagation gradient.

[0155] Fourthly, according to one or more embodiments of this disclosure, a data processing apparatus is provided, comprising: an acquisition unit for acquiring data to be processed, the data to be processed being image data or video data; and a processing unit for processing the data to be processed using a computer vision model, the computer vision model being trained using the model determination method as described in the first aspect or various possible designs of the first aspect.

[0156] Fifthly, according to one or more embodiments of the present disclosure, an electronic device is provided, comprising: at least one processor and a memory; the memory storing computer-executable instructions; the at least one processor executing the computer-executable instructions stored in the memory, causing the at least one processor to perform a model determination method as described in the first aspect or various possible designs of the first aspect, or to perform a data processing method as described in the second aspect or various possible designs of the second aspect.

[0157] In a sixth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein computer-executable instructions are stored therein, which, when executed by a processor, implement the model determination method as described in the first aspect or various possible designs of the first aspect, or perform the data processing method as described in the second aspect or various possible designs of the second aspect.

[0158] In a seventh aspect, according to one or more embodiments of the present disclosure, an electronic device is provided, comprising: at least one processor and a memory; the memory storing computer-executable instructions; the at least one processor executing the computer-executable instructions stored in the memory, such that the at least one processor performs a model determination method as described in the first aspect or various possible designs of the first aspect, or performs a data processing method as described in the second aspect or various possible designs of the second aspect.

[0159] Eighthly, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, the computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the model determination method as described in the first aspect or various possible designs of the first aspect, or perform the data processing method as described in the second aspect or various possible designs of the second aspect.

[0160] Ninthly, according to one or more embodiments of the present disclosure, a computer program product is provided, the computer program product comprising computer execution instructions that, when executed by a processor, implement the model determination method as described in the first aspect or various possible designs of the first aspect, or implement the data processing method as described in the second aspect or various possible designs of the second aspect.

[0161] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0162] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0163] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A model determination method, comprising: Based on the computer vision model and loss function, the loss values ​​corresponding to multiple training samples are determined respectively, wherein the training samples are image samples or video samples; The loss values ​​corresponding to the multiple training samples are sorted in order of numerical value. The target loss value is determined as the loss value that, from the beginning to the end, is located after the N% of loss values ​​and before the M% of loss values, where N is less than M; wherein, N is adjusted according to the proportion of dirty data in the training set in the current training iteration, and M is adjusted according to the total number of training samples in the training set; Based on the target loss value, the parameters of the computer vision model are adjusted.

2. The model determination method according to claim 1, before determining the loss values ​​corresponding to multiple training samples based on the computer vision model and the loss function, further includes: Determine the training set in the current training iteration, wherein the training set includes the plurality of training samples; The N and / or M are adjusted based on the training set.

3. The model determination method according to claim 2 further includes: If the proportion of dirty data in the training set increases, then N is increased; otherwise, N is decreased.

4. The model determination method according to claim 2 further includes: If the total number of training samples in the training samples increases, then M is decreased; otherwise, M is increased.

5. The model determination method according to any one of claims 1-4, wherein adjusting the parameters of the computer vision model based on the target loss value includes: The backpropagation gradient of the computer vision model is determined based on the arithmetic mean or weighted average of the target loss values. The parameters of the computer vision model are adjusted based on the returned gradient.

6. A data processing method, comprising: Acquire data to be processed, which may be image data or video data; The data to be processed is processed by a computer vision model, which is trained using the model determination method described in any one of claims 1-5.

7. A model determining device, comprising: The determining unit is used to determine the loss value corresponding to multiple training samples based on the computer vision model and the loss function, wherein the training samples are image samples or video samples; A filtering unit is used to sort the loss values ​​corresponding to the plurality of training samples according to their numerical order; and to determine the target loss value as the loss value that is located after the N% loss value and before the M% loss value in the sorted loss value, wherein N is less than M, wherein N is adjusted according to the proportion of dirty data in the training set in the current training iteration, and M is adjusted according to the total number of training samples in the training set. An adjustment unit is used to adjust the parameters of the computer vision model based on the target loss value.

8. A data processing device, comprising: An acquisition unit is used to acquire data to be processed, wherein the data to be processed is image data or video data; The processing unit is used to process the data to be processed using a computer vision model, wherein the computer vision model is trained using the model determination method described in any one of claims 1-5.

9. An electronic device, comprising: At least one processor and memory; The memory stores computer-executed instructions; The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the model determination method as described in any one of claims 1-5 or the data processing method as described in claim 6.

10. A computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the model determination method as described in any one of claims 1-5 or the data processing method as described in claim 6.

11. A computer program product comprising computer-executable instructions, which, when executed by a processor, implement the model determination method as described in any one of claims 1-5 or the data processing method as described in claim 6.

Images