Method, device, equipment and medium for generating negative sample pairs of contrast learning model

Abstract

Methods, apparatuses, devices and media for generating negative sample pairs for a contrastive learning model are provided. In one method, a first data segment is obtained from a first data sequence of a plurality of data sequences used to train the contrastive learning model, and a second data segment is obtained from a second data sequence of the plurality of data sequences. A data frame is selected from other data sequences than the second data sequence of the plurality of data sequences. A third data segment is generated based on the second data segment and the data frame. A negative sample pair used to train the contrastive learning model is determined based on the first data segment and the third data segment. With the exemplary implementation of the present disclosure, richer semantic information can be introduced into the negative sample pair in terms of appearance, thereby improving the accuracy of the contrastive learning model.

Description

Technical Field

[0001] The exemplary embodiments of this disclosure generally relate to machine learning, and more particularly to methods, apparatus, devices, and computer-readable storage media for generating negative sample pairs for contrastive learning models. Background Technology

[0002] In self-supervised learning, manual labeling of samples is unnecessary. Positive and negative samples can be constructed by modifying existing samples. Furthermore, these positive and negative samples can be used to train a contrastive learning model. The quality and quantity of negative samples significantly impact the performance of the contrastive learning model, and consequently, the performance of downstream models that call upon it. Therefore, how to generate suitable negative samples to improve the accuracy of the contrastive learning model becomes a pressing issue. Summary of the Invention

[0003] In a first aspect of this disclosure, a method for generating negative sample pairs for a contrastive learning model is provided. In this method, a first data segment is obtained from a first data sequence among a plurality of data sequences used to train the contrastive learning model, and a second data segment is obtained from a second data sequence among the plurality of data sequences. A data frame is selected from other data sequences besides the second data sequence among the plurality of data sequences. A third data segment is generated based on the second data segment and the data frame. Negative sample pairs for training the contrastive learning model are determined based on the first data segment and the third data segment.

[0004] In a second aspect of this disclosure, an apparatus for generating negative sample pairs for a contrastive learning model is provided. The apparatus includes: an acquisition module configured to acquire a first data segment from a first data sequence among a plurality of data sequences used for training the contrastive learning model, and to acquire a second data segment from a second data sequence among the plurality of data sequences; a selection module configured to select a data frame from other data sequences besides the second data sequence among the plurality of data sequences; a generation module configured to generate a third data segment based on the second data segment and the data frame; and a determination module configured to determine negative sample pairs for training the contrastive learning model based on the first data segment and the third data segment.

[0005] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform the method of the first aspect.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. A computer program is stored on the medium, which, when executed by a processor, implements the method of the first aspect.

[0007] It should be understood that the content described in this content section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0008] The above and other features, advantages, and aspects of the various embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein: Figure 1 A schematic diagram of an example environment in which embodiments of the present disclosure can be implemented is shown; Figure 2 A block diagram illustrating the process of training a contrastive learning model according to a technical solution is shown. Figure 3 A block diagram is shown illustrating a process for generating negative sample pairs for a contrastive learning model according to some embodiments of the present disclosure; Figure 4 A block diagram is shown illustrating a process for generating noisy data frames according to some embodiments of the present disclosure; Figure 5 A block diagram is shown illustrating a process for generating noisy data frames according to some embodiments of the present disclosure; Figure 6 A block diagram is shown illustrating a process for generating a data frame in a third data segment according to some embodiments of the present disclosure; Figure 7 A block diagram is shown illustrating a process for generating a data frame in a third data segment according to some embodiments of the present disclosure; Figure 8 A block diagram is shown illustrating a process for generating negative sample pairs according to some embodiments of the present disclosure; Figure 9 A block diagram illustrating a process for adjusting the sampling frequency of a data segment according to some embodiments of the present disclosure is shown; Figure 10 A flowchart is shown of a method for generating negative sample pairs for a contrastive learning model according to some embodiments of the present disclosure; Figure 11 A block diagram of an apparatus for generating negative sample pairs for a contrastive learning model according to some implementations of this disclosure is shown; and Figure 12 An electronic device in which one or more embodiments of the present disclosure may be implemented is shown. Detailed Implementation

[0009] Implementations of this disclosure will now be described in more detail with reference to the accompanying drawings. While some implementations of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the implementations set forth herein. Rather, these implementations are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and implementations of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0010] In the description of the implementation methods disclosed herein, the term "comprising" and similar terms should be understood as open inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one implementation" or "the implementation" should be understood as "at least one implementation". The term "some implementations" should be understood as "at least some implementations". Other explicit and implicit definitions may also be included below. As used herein, the term "model" can represent the relationships between various data. For example, the aforementioned relationships can be obtained based on various currently known and / or future-developed technical solutions.

[0011] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0012] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure through appropriate means in accordance with relevant laws and regulations, and user authorization should be obtained.

[0013] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0014] As an optional but non-restrictive implementation, in response to a user's active request, a prompt message can be sent to the user, for example, via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose whether to "agree" or "disagree" to provide personal information to the electronic device.

[0015] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0016] The term "in response to" as used herein refers to a state in which a corresponding event occurs or a condition is satisfied. It will be understood that the timing of subsequent actions performed in response to such event or condition is not necessarily strongly correlated with the time when the event occurs or the condition is met. For example, in some cases, subsequent actions may be performed immediately upon the occurrence of the event or the fulfillment of the condition; while in others, they may be performed some time after the occurrence of the event or the fulfillment of the condition.

[0017] Example Environment Figure 1 A block diagram of an example environment 100 in which the implementation of this disclosure can be implemented is shown. Figure 1 In environment 100, it is desired to train and use a machine learning model (i.e., model 130) configured for various application environments, such as identifying video similarity, etc. Figure 1 As shown, environment 100 includes model training system 150 and model application system 152. Figure 1 The upper part illustrates the model training phase, and the lower part illustrates the model application phase. Before training, the parameter values ​​of model 130 can have initial values ​​or pre-trained parameter values ​​obtained through a pre-training process. Model 130 can be trained via forward and backward propagation, during which the parameter values ​​of model 130 can be updated and adjusted. After training is complete, model 130' is obtained. At this point, the parameter values ​​of model 130' have been updated, and based on the updated parameter values, model 130 can be used to achieve the prediction task in the model application phase.

[0018] During the model training phase, model 130 can be trained using a training sample set 110 comprising multiple samples 112 and a model training system 150. In contrastive learning, positive and negative samples can be constructed using the multiple samples 112, and the training process can be iteratively executed using a large number of samples. After training is complete, model 130 can include knowledge about the task to be processed. During the model application phase, model 130' (which now has the trained parameter values) can be invoked using a model application system 152. Furthermore, a downstream model 140 can be invoked after model 130' to perform a specific task.

[0019] exist Figure 1In this context, the model training system 150 and the model application system 152 can include any computing system with computing capabilities, such as various computing devices / systems, terminal devices, servers, etc. Terminal devices can involve any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. Servers include, but are not limited to, mainframes, edge computing nodes, computing devices in cloud environments, etc.

[0020] It should be understood that Figure 1 The components and arrangements shown in environment 100 are merely examples, and a computing system suitable for implementing the exemplary implementations described in this disclosure may include one or more different components, other components, and / or different arrangements. For example, although shown as separate, model training system 150 and model application system 152 may be integrated in the same system or device. Implementations of this disclosure are not limited in this respect. Exemplary implementations of model training and model application will continue to be described below with reference to the accompanying drawings.

[0021] For ease of description, this disclosure uses video processing as the application environment for the contrastive learning model, which can be used to perform video self-supervised tasks. For example, a set of video sequences can be obtained as samples, and samples can be constructed using the individual video sequences in the set.

[0022] Figure 2 A block diagram 200 illustrates the process of training a contrastive learning model according to a given technical solution. For example... Figure 2 As shown, an original sample 210 can be determined from a video, and sample 220 (i.e., a positive sample) can be generated by modifying this sample. Further, sample 230 (e.g., a sample from another video) can be used as a negative sample. Model 130 can be trained using samples 210 and 220 as positive sample pairs and samples 210 and 230 as negative sample pairs. During training, the encoder 240 in model 130 can output features of samples 210, 220, and 230, i.e., features 212, 222, and 232, respectively. As indicated by arrow 241, model 130 can be trained in a direction that brings features 212 and 222 closer together; as indicated by arrow 242, model 130 can be trained in a direction that pushes features 212 and 232 further apart. The training process can be performed iteratively using a large number of positive and negative positive sample pairs.

[0023] Several techniques have been proposed for constructing negative samples. In the BE (Beta-Obstruction) technique, videos other than positive samples are directly used as negative samples. This causes the contrastive learning model to fail to perceive dynamic information of the video content (e.g., playback speed), and requires a large number of training videos to obtain sufficient negative samples. In the Pace (Plan-Ahead) technique, the sampling frequency of videos other than positive samples can be adjusted (i.e., speed can be varied, for example, from 0.25 seconds per frame to 0.5 seconds per frame, or other values), and the adjusted videos are used as negative samples. In this case, the difference between positive and negative samples is too obvious, leading to a decrease in learning difficulty and thus unsatisfactory training results.

[0024] In the RSPNet approach, the sampling frequency of positive video samples can be adjusted, and the adjusted video is used as a negative sample. The adjusted video has a different playback speed than the original video, allowing the contrastive learning model to distinguish between positive and negative samples in terms of dynamic information. However, since the positive and negative samples have the same video content (i.e., appearance), the contrastive learning model cannot acquire semantic knowledge about appearance differences.

[0025] It will become clear that the construction of negative samples significantly impacts the semantic knowledge acquired during training, thereby affecting the accuracy of the contrastive learning model. Therefore, it is desirable to generate high-quality negative samples in a simpler and more effective way.

[0026] A summary of the process of generating negative samples To at least partially address the shortcomings described above, a method for generating negative sample pairs for contrastive learning models is proposed. See also Figure 3 This describes a summary of an exemplary implementation according to the present disclosure. Figure 3 A block diagram 300 illustrates a process for generating negative sample pairs for a contrastive learning model according to some embodiments of the present disclosure. Figure 3 As shown, multiple data sequences 350 can be acquired for training the contrastive learning model. It will be understood that although video sequences are used as an example to describe details of an exemplary implementation according to this disclosure, alternatively and / or additionally, data sequences may include sequences of other formats. For example, in an audio processing application, the data sequence may include an audio sequence. In other application scenarios, the data sequence may also include, for example, thermal image sequences, temperature sequences, humidity sequences, or sequences of other monitored parameters.

[0027] A first data sequence 310 and a second data sequence 320 can be determined from multiple data sequences 350, respectively. Here, each of the two data sequences may include multiple data frames, and their resolution and length may be the same or different. Further, a first data segment 330 can be obtained from the first data sequence 310. In this case, the first data segment 330 may include a portion of the data frames from the multiple data frames in the first data sequence 310. The first data segment 330 may have a predetermined length, assuming the first data sequence includes... (positive integer) data frames, the first data segment 330 may include (positive integer) data frames ).

[0028] Furthermore, a second data segment 332 can be determined from a second data sequence 320 among multiple data sequences 350 in a similar manner. The first data segment 330 and the second data segment 332 may have the same length (e.g., including...). (a data frame). Further, a data frame 334 can be selected from other data sequences besides the second data sequence 320 among the multiple data sequences 350. Here, data frame 334 will be used as a noise factor, and noise can be added to the generated negative samples, thereby increasing the training difficulty and improving the accuracy of the contrastive learning model.

[0029] Data frame 334 can be selected in various ways. For example, it can be selected randomly, from a specific location, or based on the content of the data sequence. Furthermore, a third data segment 360 (i.e., negative samples) can be generated based on the second data segment 332 and data frame 334. In other words, data frame 334 can be used to modify individual data frames in the second data segment 332 to generate the third data segment 360. Further, the first data segment 330 and the third data segment 360 can be used to determine negative sample pairs 340 for training the contrastive learning model.

[0030] In video processing applications, two distinct video segments can be identified from different video sequences. These two segments can be used as the basis for constructing negative sample pairs. One segment can be used as the original video segment, and the other as the negative sample. Furthermore, video frames from different video sequences can be used to perturb the appearance of the negative sample (e.g., fusing the content of video frames into the various video frames of the negative sample, i.e., overlapping two video frames). Then, the original video segment and the perturbated negative sample can be used as a negative sample pair.

[0031] It will be understood that the negative samples here are modified from video frames from other video sequences, thus introducing more aggressive appearance perturbations into the negative samples, thereby destroying the original information in the negative samples. Furthermore, video frames from other video sequences typically have different color information, which, compared to techniques based solely on adjusting the video's sampling frequency, can introduce stronger perturbations in appearance into the negative samples, thus providing negative samples with richer semantic information. Furthermore, different video frames can be used as perturbations to generate a large number of negative samples. In this way, more negative samples with richer appearance information can be generated without additionally expanding the training sample set, thereby improving the accuracy of the contrastive learning model. Furthermore, this can improve the training efficiency and accuracy of downstream models.

[0032] Detailed process of generating negative samples The outline of generating negative sample pairs has been described, and further details of generating negative sample pairs will be described below with reference to the accompanying drawings. According to an exemplary implementation of this disclosure, a plurality of data sequences 350 can be selected from a disclosed set of video sequences. A first data sequence 310 and a second data sequence 320 can be selected from the plurality of data sequences 350. The two data sequences can be selected randomly, alternatively and / or additionally, based on the resolution, length, speed, content, etc., of the video sequences. Further, a first data segment 330 can be obtained from the first data sequence 310, and a second data segment 332 can be obtained from the second data sequence 320. The first data segment 330 and the second data segment 332 can have the same length.

[0033] Typically, video sequences may have different lengths and may include multiple shots. Suppose the first data sequence 310 involves automotive content and includes two shots: the car driving on the road (shot 1) and the car's interior (shot 2). When obtaining the first data segment 330 from the first data sequence 310, single-shot data needs to be selected to avoid the first data segment 330 including content from two weakly related shots due to shot switching, thus affecting the sample quality. For example, the first data segment 330 can be selected from shot 1 (or shot 2). The first data segment 330 can be selected from the first data sequence 310 according to a predetermined length. The second data segment 332 can be determined from the second data sequence 320 in a similar manner. Specifically, the second data segment 332 is selected from the portion of the second data sequence 320 that includes a single shot according to a predetermined length.

[0034] Furthermore, a data frame 334 can be selected from other data sequences besides the second data sequence 320 among the multiple data sequences 350. It will be understood that the data frame 334 here is used to interfere with the appearance information of the second data segment 332, thus requiring the selection of a data frame 334 from other data sequences besides the second data sequence 320. To further introduce more interference information, a data frame 334 can be selected from a data sequence that has a significant content difference from the second data sequence 320. At this time, a second data range for each data frame in the second data sequence 320 can be determined. Further, the data range of each data frame in the other data sequences among the multiple data sequences 350 can be determined, and by comparison, a data sequence with a significant data difference from the second data range can be determined; subsequently, a data frame 334 can be selected from that data sequence.

[0035] Specifically, in a video sequence scenario, a second color range can be determined for multiple data frames in the second data sequence 320, and a data sequence with a large color contrast can be selected from the multiple data sequences 350. The color range of each data sequence in the multiple data sequences 350 can be determined. If the difference between the color range of a certain data sequence and the second color range meets a predetermined condition, that data sequence can be selected, and data frame 334 can be chosen from it.

[0036] Here, the pixel values ​​of each video frame in the video sequence can be used to determine predetermined conditions. For example, a predetermined condition could state that the numerical difference between corresponding pixels in two videos should exceed a predetermined threshold. For instance, when pixel colors in three channels are represented by RGB triples, the predetermined threshold could represent the spatial distance between pixels in the three channels. Assuming the second data sequence 320 includes train video and its overall color range is blue-gray, for example, a video sequence including a reddish-brown color range can be selected from multiple video sequences, and video frames can be selected from there.

[0037] According to an exemplary implementation of this disclosure, data frame 334 can be selected in various ways. For example, any data frame can be randomly selected from a given data sequence, or a data frame at a specified position (e.g., start position, center position, or end position) can be selected. Since the colors in the two data sequences are different, using data frame 334 to modify the second data segment 332 can interfere with the original color distribution of the second data segment 332, thereby increasing the learning difficulty. This will help the contrastive learning model acquire more semantic knowledge.

[0038] Here, the first data segment 330 can be used as the original data segment and the second data segment 332 as the negative sample. Further, the second data segment 332 can be modified using data frame 334 to introduce more interference information. According to an exemplary implementation of this disclosure, data frame 334 can be directly fused into each data frame in the second data segment 332 (i.e., data frame 334 is superimposed onto each data frame) to generate a third data segment 360, which is then used as the negative sample. In this way, data frame 334 is directly fused into each video frame of the second data segment 332, and the negative sample can include richer semantic information, thereby improving the generalization ability of the contrastive learning model. Furthermore, the fusion process involves only simple processing and does not significantly increase the workload of generating negative samples. In this way, negative samples can be generated in a simple and efficient manner.

[0039] According to an exemplary implementation of this disclosure, to further enrich the appearance information of negative samples, high-frequency information can be added to the third data segment 360. Specifically, high-frequency noise data frames can be generated based on data frame 334, and the noise data frames can be used to update the various data frames in the second data segment 332. See also Figure 4 Describe the process of generating noisy data frames. Figure 4 A block diagram 400 illustrates a process for generating noisy data frames according to some embodiments of the present disclosure.

[0040] like Figure 4 As shown, data frame 334 represents a data frame used as interference data. The dimensions of data frame 334 can be adjusted according to a predetermined ratio (e.g., reducing the height and width of the video frame to 1 / K of its original value, such as K=3 or other values) to generate intermediate data frame 420. Intermediate data frame 420 can be copied to generate multiple copies, and these copies can then be concatenated to generate noise data 410. At this time, noise data frame 410 will include... A reduced data frame.

[0041] It will be understood that, although the above shows the same proportional adjustment in both height and width dimensions, alternatively and / or additionally, different adjustment methods can be performed. Figure 5 A block diagram 500 illustrates a process for generating noisy data frames according to some embodiments of the present disclosure. For example... Figure 5 As shown, the proportions of the height dimension alone can be adjusted to generate noise data 510. Similarly, the proportions of the width dimension alone can be adjusted to generate noise data 520. Furthermore, the height and width dimensions can be adjusted at different proportions to generate noise data frames 530.

[0042] It will be understood that, although the above description shows the case where the noisy data frame includes exactly an integer multiple of the intermediate image, alternatively and / or additionally, the noisy data may include intermediate images that are not integer multiples. For example, noisy data frame 540 may include 6 complete intermediate data frames and 3 incomplete intermediate data frames.

[0043] It will be understood that, although the above shows the case where the noisy data frame has the same resolution as the data frame in the second data segment 332, the noisy data frame and the data frame may have different resolutions. In this case, the noisy data frame can be cropped to the resolution of the data frame in the second data segment 332. Alternatively and / or additionally, the noisy data frame can be scaled to the resolution of the data frame in the second data segment 332. In this way, it can be ensured that the noisy data and the various data frames in the second data segment 332 have the same resolution, thereby facilitating the generation of the various data frames in the third data segment 360 in a simpler and faster manner based on a unified processing time.

[0044] The following section describes how to generate the noisy data frame 410 using specific formulas. (Assume symbols...) This refers to data frame 334. The height and width dimensions of data frame 334 can be adjusted to adjust the height of data frame 334. and width Adjusted to the original state respectively. and At this point, the dimension of the intermediate data frame 420 is... It can be spliced ​​together. 420 intermediate data frames were copied to generate a dimension of Noisy data frame 410 (in symbol) express).

[0045] Furthermore, the noisy data frame 410 can be fused into each data frame in the second data segment 332 to generate the third data segment 360. According to an exemplary implementation of this disclosure, the noisy data frame 410 can be used to update each data frame in the second data segment 332. Figure 6 A block diagram 600 illustrates a process for generating a data frame in a third data segment according to some embodiments of the present disclosure. For example... Figure 6 As shown, only data frame 610 is described as an example of a data frame in the second data segment 332. Data frame 610 can be merged with noisy data frame 410 to generate data frame 620.

[0046] According to an exemplary implementation of this disclosure, each pixel in the data frame 620 can be determined one by one. For example, assuming that the symbol is... Indicates the second data segment 332, using the symbol This indicates the first data segment in the second data segment 332. The frame can be determined based on the following formula 1, which is 620 data frames.

[0047] Formula 1 In Formula 1, This indicates the third data segment, 360. This indicates the third data segment 360. Pixel data of a frame This represents the predetermined weight of the noisy data frame 410, and This represents the pixel data of the noisy data frame 510. Each data frame in the second data segment 332 can be processed based on Formula 1 to generate the data frames in the third data segment 360.

[0048] Specifically, for a given data point in data frame 610 of the second data segment 332 (e.g., position is...) The data value (e.g., pixel value) of a given data point can be obtained. Then, the corresponding data value of the data point in the noisy data frame 410 corresponding to the given data point can be obtained. Further, the data value of the data point in the data frame 620 corresponding to the given data point can be determined based on the data value, the corresponding data value, and the weight of the noisy data frame. In other words, the data value of the data point in the data frame 420 corresponding to the given data point can be determined based on the data value, the corresponding data value, and the weight of the noisy data frame. The pixel value of the pixel, the position in the noise data frame 410 is The pixel value and weight of the pixel are used to determine the position in the synthesized data frame 620. The pixel value of the pixel.

[0049] It will be understood that pixel values ​​can include three channels: RGB (red, green, and blue). Therefore, the pixel values ​​in each channel can be processed separately to determine the pixel values ​​in each channel of the data frame 620. Assume the... The RGB channels of a frame are represented as follows: , , The third data segment 360 can be determined based on the following formula 2. Pixel values ​​for each channel of the frame:

[0050]

[0051] Formula 2 In Formula 2, , and These represent the first data segment in the third data segment 360. Pixel data from the three channels of a frame. This represents the predetermined weight of the noisy data frame 510, and , and These represent the pixel data of the three channels of the noisy data frame 510. Each data frame in the second data segment 332 can be processed based on Formula 2 to generate the data frames in the third data segment 360.

[0052] For example, using weights To generate data frame 620. At this point, data frame 610 and noisy data frame 410 each contribute 50% to the synthesized data frame 620. According to an exemplary implementation of this disclosure, the weights can be modified. This is to adjust the interference level of noisy data frame 410 on each data frame in the second data segment 332. Specifically, the weights are reduced. The value can add less noise and increase the weight. The value can add more noise.

[0053] Figure 7 A block diagram 700 illustrates a process for generating a data frame in a third data segment according to some embodiments of the present disclosure. For example... Figure 7 As shown, using weights To generate data frame 710, and using weights To generate data frame 720. From Figure 7 As can be seen, different weights can adjust the degree of interference of noisy data frame 410 on data frame 610. Too small a weight will only introduce a small amount of noise, resulting in insufficient generalization ability of the contrastive learning model. Too large a weight will introduce a large amount of noise, which may cause the contrastive learning model to ignore the dynamic information carried by the original data segment (i.e., the relationship between actions and changes between data frames), thus introducing uncertainty into the contrastive learning model. A balance can be struck between these two aspects, and an appropriate weight can be selected.

[0054] Figure 8 A block diagram 800 illustrating a process for generating negative sample pairs according to some embodiments of the present disclosure is shown. With the third data segment 360 already generated, the first data segment 330 and the third data segment 360 can be combined to generate a negative sample pair 340. At this time, the negative sample pair 340 can be used to train a contrastive learning model to widen the distance between the features of the first data segment 330 and the third data segment 360.

[0055] It will be understood that although only the process of generating a single negative sample pair was described above, multiple negative sample pairs can be generated in a similar manner. Assume the data sequence set includes... A data sequence, can Any one of the data sequences can be used as the first data sequence 310 described above, and the remaining data sequences can be used as the first data sequence 310 described above. Any one of the data sequences is used as the second data sequence 320 described above. A first data segment 330 can be determined from the first data sequence 310, and a second data segment 332 can be determined from the second data sequence 320. A data frame 334 can be determined from other data sequences. Furthermore, the second data segment 332 can be updated using the data frame 334 to generate a third data segment 360. Subsequently, the first data segment 330 and the third data segment 360 can be combined to generate negative sample pairs 340. In this way, a large number of negative sample pairs 340 can be generated, thereby iteratively training the contrastive learning model.

[0056] The process of adding perturbation information to negative samples has been described above. Alternatively and / or additionally, the sampling frequency of the data segments used as negative samples can be further adjusted so that the contrastive learning model can more strongly perceive the dynamic information in the samples. Figure 9 A block diagram 900 illustrates a process for adjusting the sampling frequency of a data segment according to some embodiments of the present disclosure. For example... Figure 9 As shown, assume that data segment 910 is a negative sample and includes multiple data frames 912, 914, 916, 918, 920, ..., 922. In this case, a portion of the data frames can be selected from the multiple data frames at predetermined intervals, i.e., data frames 912, 916, 920, ..., 922, using, for example, a "downsampling" method, and data segment 930 after adjusting the sampling frequency can be generated.

[0057] Compared to data segment 910, the dynamic changes of each data frame in data segment 930 will be more significant. This allows the contrastive learning model to perceive dynamic information in addition to the appearance information of the data frames, thereby enhancing the performance of the contrastive learning model. Specifically, in video processing scenarios, data segment 930 can be a video with a faster playback speed, which allows the contrastive learning model to better grasp the dynamic changes between each video frame.

[0058] The process of constructing negative samples using data segments from different data sequences has been described above. Alternatively and / or additionally, negative samples can be constructed using data segments from the same data sequence. Specifically, the sampling frequency of multiple data frames in the first data sequence 310 can be adjusted to generate a fourth data segment. For example, it can be done according to... Figure 9As shown, a fourth data segment is generated from the first data sequence 310 using a "frame extraction" method. Since the fourth data segment has a different appearance and speed than the first data segment 330, the first data segment 330 and the fourth data segment can be combined to determine negative sample pairs for training the contrastive learning model. In this way, dynamic changes can be introduced into the negative samples, and the number of negative samples can be further increased.

[0059] Alternatively and / or additionally, appearance-based perturbations can be further introduced into the negative samples from the same data sequence. For example, a fifth data segment can be generated based on the fourth data segment and data frame 334, following the fusion method described above. Furthermore, negative sample pairs for training the contrastive learning model can be determined based on the first and fifth data segments. In this case, the negative samples from the same data sequence include both dynamic information and appearance-based perturbation information. In this way, the quantity and quality of samples can be further improved, thereby enhancing the performance of the contrastive learning model.

[0060] The process of generating negative samples has been described above. According to an exemplary implementation of this disclosure, positive samples can be further constructed based on data segments from the same data sequence. For example, a sixth data segment can be determined from the first data sequence 310, and interference can be added to the sixth data segment using data frame 334 to generate a seventh data segment.

[0061] At this point, both the first data segment 330 and the seventh data segment originate from the first data sequence 310. However, the seventh data segment includes interfering factors (i.e., it incorporates information from data frame 334). The first and seventh data segments can then be combined to determine positive sample pairs for training the contrastive learning model. The inclusion of interfering factors from other data sequences in these positive samples increases the learning difficulty of the positive samples, thereby improving the accuracy of the contrastive learning model in distinguishing positive samples.

[0062] Using the exemplary implementations of this disclosure, negative and positive sample pairs can be generated separately in various ways. In general, positive sample pairs can be generated using data segments from the same data sequence. Table 1 below illustrates, with symbols, multiple data segments from the same data sequence.

[0063] Table 1 From the data sequence data segment

[0064] Furthermore, negative sample pairs can be generated using data segments from different data sequences. For example, for any one of two data segments from different data sequences, interference information can be added to that data segment based on the formula described above to adjust the appearance information of the sample. Alternatively and / or additionally, the sampling frequency of that data segment can be modified to adjust the dynamic information of the sample. Alternatively and / or additionally, both of the above operations can be performed to adjust both the appearance and dynamic information of the sample. Table 2 below uses symbols to illustrate multiple data segments from different data sequences.

[0065] Table 2 From the data sequence data segment

[0066] According to one exemplary implementation of this disclosure, positive samples may include Negative samples can include data segments from the same data sequence. and In other words, data segments from the same data sequence, after undergoing a speed-changing operation, can be used as negative samples. Alternatively and / or additionally, negative samples may also include data segments from different data sequences. , , , In other words, data segments from different data sequences can be used as negative samples, and in order to improve the information richness of the negative samples, image fusion, speed adjustment, or both can be performed on data segments from different data sequences.

[0067] According to an exemplary implementation of this disclosure, positive sample pairs and negative sample pairs can be constructed respectively using the various data segments shown above. For example, a positive sample pair may include: ( Negative sample pairs can include ( 、( 、( 、( 、( 、( Using the exemplary implementation of this disclosure, positive and negative sample pairs that include rich semantic information can be constructed.

[0068] Furthermore, the positive and negative sample pairs described above can be used to train a contrastive learning model. In this way, positive sample pairs can bring the features between positive samples closer together, while negative sample pairs can push the features between negative samples further apart. This improves the accuracy of the contrastive learning model.

[0069] According to an exemplary implementation of this disclosure, the effectiveness of training a contrastive learning model using the proposed technique can be verified on multiple public datasets (e.g., the video datasets UCF101 and Kinetics-400). Table 3 below shows the evaluation results in downstream scenarios using full fine-tuning and linear evaluation. Here, full fine-tuning and linear evaluation are two forms used for training on downstream tasks, where full fine-tuning means that the trained model parameters directly participate in full parameter fine-tuning training in the downstream task, and linear evaluation means that only the last fully connected layer is adjusted while other parameters are fixed.

[0070] Table 3. Accuracy of the contrastive learning model trained with different samples

[0071] In Table 3, UCF and HMDB represent different datasets, "AD-Pos" represents the technique for generating positive samples by using image fusion based on data segments in the same video as described above, Intra-Neg represents the technique for generating negative samples by varying the speed of data segments in the same video, and AD-Intra-Neg represents the technique for generating negative samples by combining image fusion and speed variation of data segments in the same video.

[0072] For example, in a scenario where pre-training is performed on the UCF101 dataset, and full fine-tuning is performed on the UCF dataset, the model accuracy is 76.0% without using the method disclosed herein to generate negative samples. When the AD-Pos technique is used, the accuracy improves to 78.2%; when AD-Pos and Intra-Neg techniques are used, the accuracy improves to 79.6%; and when AD-Pos, Intra-Neg, and AD-Intra-Neg techniques are used, the accuracy improves to 80.9%. As shown in Table 3, the above three methods of generating negative samples can improve the accuracy of the contrastive learning model.

[0073] It will be understood that although the above description uses only video sequences as an example of data sequences to illustrate the process of generating sample pairs, alternatively and / or additionally, data sequences may also include, for example, audio sequences, thermal image sequences, temperature sequences, humidity sequences, or sequences of other monitored parameters. Specifically, in the case of processing audio data, audio data can be sampled at predetermined frequencies, and an audio data sequence at predetermined sampling points can be generated. The engine data sequence can be processed using the methods described above to generate corresponding positive sample pairs.

[0074] Using the exemplary implementation of this disclosure, more noise information can be added to positive samples, achieving a more aggressive debuff. For example, high-frequency information in the concatenated noisy data frame can increase the difficulty for the contrastive learning model to identify positive samples. In this way, the contrastive learning model can become more concerned with the semantic information in each pair of positive samples, thereby learning richer semantic knowledge.

[0075] Example process The specific process of generating samples has been described above. See below for further details. Figure 10 Describe the corresponding methods. Figure 10 A flowchart of a method 1000 for generating negative sample pairs for a contrastive learning model according to some embodiments of the present disclosure is shown. At block 1010, a first data segment is obtained from a first data sequence among a plurality of data sequences used to train the contrastive learning model, and a second data segment is obtained from a second data sequence among the plurality of data sequences; at block 1020, a data frame is selected from other data sequences among the plurality of data sequences besides the second data sequence; at block 1030, a third data segment is generated based on the second data segment and the data frame; and at block 1040, negative sample pairs for training the contrastive learning model are determined based on the first data segment and the third data segment.

[0076] According to an exemplary implementation of this disclosure, generating a third data segment includes: generating a noisy data frame based on a data frame; and using the noisy data frame to update the data frame in the second data segment.

[0077] According to an exemplary implementation of this disclosure, generating a noisy data frame includes: adjusting the dimension of the data frame according to a predetermined ratio to generate an intermediate data frame; copying the intermediate data frame to generate multiple copied intermediate data frames; and splicing the multiple copied intermediate data frames to generate noisy data.

[0078] According to one exemplary implementation of this disclosure, the dimensions of a data frame include at least one of width and height.

[0079] According to an exemplary implementation of this disclosure, the method 1000 further includes: in response to determining that the resolution of the noise data frame is different from the resolution of the data frame in the second data segment, performing at least one of the following: cropping the noise data frame to the resolution of the data frame in the second data segment; scaling the noise data frame to the resolution of the data frame in the second data segment.

[0080] According to an exemplary implementation of this disclosure, updating a data frame in a second data segment using a noisy data frame includes: for a given data point in a data frame in the second data segment, obtaining the data value of the given data point; obtaining the corresponding data value of the data point in the noisy data frame corresponding to the given data point; and determining the data value of the data point in the data frame corresponding to the given data point based on the data value, the corresponding data value, and the weight of the noisy data frame.

[0081] According to an exemplary implementation of this disclosure, obtaining the first data segment includes: selecting a first data segment that meets a predetermined length from a first data sequence, wherein the first data segment includes only a single shot.

[0082] According to an exemplary implementation of this disclosure, obtaining the second data segment includes: selecting a second data segment from the second data sequence that satisfies a predetermined length, wherein the second data segment comprises only a single shot.

[0083] According to an exemplary implementation of this disclosure, the method 1000 further includes: determining a data range of a plurality of data frames in a second data sequence; determining a given data range of a plurality of data frames in a given data sequence in another data sequence; and selecting a data frame from the given data sequence in response to the determination that the difference between the determined data range and the given data range satisfies a predetermined condition.

[0084] According to an exemplary implementation of this disclosure, generating the third data segment further includes: adjusting the sampling frequency of multiple data frames in the third data segment.

[0085] According to an exemplary implementation of this disclosure, the method 1000 further includes: adjusting the sampling frequency of multiple data frames in a first data sequence to generate a fourth data segment; and determining negative sample pairs for training a contrastive learning model based on the first data segment and the fourth data segment.

[0086] According to an exemplary implementation of this disclosure, the method 1000 further includes: generating a fifth data segment based on a fourth data segment and a data frame; and determining negative sample pairs for training a contrastive learning model based on a first data segment and a fifth data segment.

[0087] According to an exemplary implementation of this disclosure, the method 1000 further includes: obtaining a sixth data segment from a first data sequence; generating a seventh data segment based on the sixth data segment and a data frame; and determining positive sample pairs for training a contrastive learning model based on the first data segment and the seventh data segment.

[0088] According to an exemplary implementation of this disclosure, the method 1000 further includes: training a contrastive learning model using positive sample pairs and negative sample pairs.

[0089] Example devices and equipment Figure 11 A block diagram of an apparatus 1100 for generating negative sample pairs for a contrastive learning model according to some implementations of the present disclosure is shown. The apparatus 1100 includes: an acquisition module 1110 configured to acquire a first data segment from a first data sequence among a plurality of data sequences used for training the contrastive learning model, and to acquire a second data segment from a second data sequence among the plurality of data sequences; a selection module 1120 configured to select data frames from other data sequences besides the second data sequence among the plurality of data sequences; a generation module 1130 configured to generate a third data segment based on the second data segment and the data frame; and a determination module 1140 configured to determine negative sample pairs for training the contrastive learning model based on the first data segment and the third data segment.

[0090] According to an exemplary implementation of this disclosure, the generation module 1130 includes: a noise generation module configured to generate a noise data frame based on a data frame; and an update module configured to update a data frame in a second data segment using the noise data frame.

[0091] According to an exemplary implementation of this disclosure, the noise generation module includes: an adjustment module configured to adjust the dimension of a data frame according to a predetermined ratio to generate an intermediate data frame; a copying module configured to copy the intermediate data frame to generate multiple copied intermediate data frames; and a splicing module configured to splice the multiple copied intermediate data frames to generate noise data.

[0092] According to one exemplary implementation of this disclosure, the dimensions of a data frame include at least one of width and height.

[0093] According to an exemplary implementation of this disclosure, the apparatus 1100 further includes: a trimming module configured to trim the noise data frame to the resolution of the data frame in the second data segment in response to determining that the resolution of the noise data frame is different from the resolution of the data frame in the second data segment.

[0094] According to an exemplary implementation of this disclosure, the apparatus 1100 further includes: a scaling module configured to scale the noisy data frame to the resolution of the data frame in the second data segment in response to determining that the resolution of the noisy data frame is different from the resolution of the data frame in the second data segment.

[0095] According to an exemplary implementation of this disclosure, the update module includes: a data value acquisition module configured to acquire a data value of a given data point in a data frame in a second data segment; a corresponding value acquisition module configured to acquire a corresponding data value of a data point in a noisy data frame that corresponds to the given data point; and a data value determination module configured to determine the data value of a data point in a data frame that corresponds to the given data point based on the data value, the corresponding data value, and the weight of the noisy data frame.

[0096] According to an exemplary implementation of this disclosure, the acquisition module 1110 includes: a first data segment selection module, configured to select a first data segment of a predetermined length from a first data sequence, wherein the first data segment includes only a single shot.

[0097] According to an exemplary implementation of this disclosure, the acquisition module 1110 includes: a second data segment selection module, configured to select a second data segment from a second data sequence that satisfies a predetermined length, wherein the second data segment includes only a single shot.

[0098] According to an exemplary implementation of this disclosure, the apparatus 1100 further includes: a first data range determination module configured to determine a data range of a plurality of data frames in a second data sequence; a second data range determination module configured to determine a given data range of a plurality of data frames in a given data sequence in another data sequence; and a data frame selection module configured to select a data frame from the given data sequence in response to a predetermined condition being met by the difference between the determined data range and the given data range.

[0099] According to an exemplary implementation of this disclosure, the generation module 1130 further includes: an adjustment module configured to adjust the sampling frequency of multiple data frames in the third data segment.

[0100] According to an exemplary implementation of this disclosure, the apparatus 1100 further includes: a sampling frequency adjustment module configured to adjust the sampling frequency of a plurality of data frames in a first data sequence to generate a fourth data segment; and a determination module 1140 further configured to determine negative sample pairs for training a contrastive learning model based on the first data segment and the fourth data segment.

[0101] According to an exemplary implementation of this disclosure, the generation module 1130 is further configured to generate a fifth data segment based on the fourth data segment and the data frame; and the determination module 1140 is further configured to determine negative sample pairs for training the contrastive learning model based on the first data segment and the fifth data segment.

[0102] According to an exemplary implementation of this disclosure, the acquisition module 1110 is further configured to acquire a sixth data segment from the first data sequence; the generation module 1130 is further configured to generate a seventh data segment based on the sixth data segment and the data frame; and the determination module 1140 is further configured to determine positive sample pairs for training the contrastive learning model based on the first data segment and the seventh data segment.

[0103] According to an exemplary implementation of this disclosure, the apparatus 1100 further includes: a training module configured to train a contrastive learning model using positive sample pairs and negative sample pairs.

[0104] Figure 12 A block diagram is shown of an electronic device 1200 in which one or more embodiments of the present disclosure may be implemented. It should be understood that... Figure 12 The electronic device 1200 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein.

[0105] like Figure 12 As shown, electronic device 1200 is in the form of a general-purpose computing device. Components of electronic device 1200 may include, but are not limited to, one or more processors or processing units 1210, memory 1220, storage device 1230, one or more communication units 1240, one or more input devices 1250, and one or more output devices 1260. Processing unit 1210 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 1220. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 1200.

[0106] Electronic device 1200 typically includes multiple computer storage media. Such media can be any available media accessible to electronic device 1200, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 1220 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 1230 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data (e.g., samples for training) and can be accessed within electronic device 1200.

[0107] Electronic device 1200 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 12As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 1220 may include computer program product 1225 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0108] The communication unit 1240 enables communication with other electronic devices via a communication medium. Additionally, the functionality of the components of the electronic device 1200 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the electronic device 1200 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0109] Input device 1250 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 1260 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 1200 can also communicate with one or more external devices (not shown) via communication unit 1240 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 1200, or with any device that enables electronic device 1200 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0110] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0111] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0112] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0113] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0114] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0115] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for generating negative sample pairs for a contrastive learning model, comprising: A first data segment is obtained from a first data sequence among a plurality of data sequences used to train the contrastive learning model, and a second data segment is obtained from a second data sequence among the plurality of data sequences, wherein the contrastive learning model is a machine learning model for processing video data, and the plurality of data sequences are multiple videos; A data frame is selected from other data sequences besides the second data sequence among the plurality of data sequences, wherein the data frame is a video frame; Based on the second data segment and the data frame, a third data segment is generated, wherein the first data segment, the second data segment, and the third data segment are video segments; as well as Based on the first data segment and the third data segment, negative sample pairs are determined for training the contrastive learning model; The generation of the third data segment includes: Generate a noisy data frame based on the data frame; and The noisy data frames are used to update the data frames in the second data segment; The generation of the noise data frame includes: The dimensions of the data frame are adjusted according to a predetermined ratio to generate an intermediate data frame; The intermediate data frame is copied to generate multiple copied intermediate data frames; and The multiple copied intermediate data frames are spliced ​​together to generate the noise data.

2. The method of claim 1, wherein the dimension of the data frame includes at least one of width and height.

3. The method according to claim 1, further comprising: In response to determining that the resolution of the noisy data frame is different from the resolution of the data frame in the second data segment, perform at least one of the following: The noisy data frame is cropped to the resolution of the data frame in the second data segment; The noisy data frame is scaled to the resolution of the data frame in the second data segment.

4. The method of claim 1, wherein updating the data frame in the second data segment using the noisy data frame comprises: For a given data point in the data frame of the second data segment, Obtain the data value of the given data point; Obtain the corresponding data value of the data point in the noise data frame that corresponds to the given data point; as well as Based on the data value, the corresponding data value, and the weight of the noisy data frame, the data value of the data point in the data frame corresponding to the given data point is determined.

5. The method of claim 1, wherein obtaining the first data segment comprises: Select a first data segment of a predetermined length from the first data sequence, wherein the first data segment comprises only a single shot.

6. The method of claim 5, wherein acquiring the second data segment comprises: Select a second data segment that meets the predetermined length from the second data sequence, wherein the second data segment comprises only a single shot.

7. The method according to claim 1, further comprising: Determine the data range of multiple data frames in the second data sequence; Determine a given data range for multiple data frames in a given data sequence within the other data sequences; as well as In response to determining that the difference between the data range and the given data range satisfies a predetermined condition, the data frame is selected from the given data sequence.

8. The method of claim 1, wherein generating the third data segment further comprises: Adjust the sampling frequency of multiple data frames in the third data segment.

9. The method according to claim 1, further comprising: Adjust the sampling frequency of multiple data frames in the first data sequence to generate a fourth data segment; Based on the first data segment and the fourth data segment, negative sample pairs are determined for training the contrastive learning model.

10. The method of claim 9, further comprising: Based on the fourth data segment and the data frame, a fifth data segment is generated; as well as Based on the first data segment and the fifth data segment, negative sample pairs are determined for training the contrastive learning model.

11. The method of claim 1, further comprising: Obtain the sixth data segment from the first data sequence; Based on the sixth data segment and the data frame, a seventh data segment is generated; as well as Based on the first data segment and the seventh data segment, positive sample pairs are determined for training the contrastive learning model.

12. The method of claim 11, further comprising: The contrastive learning model is trained using the positive sample pairs and the negative sample pairs.

13. An apparatus for generating negative sample pairs for a contrastive learning model, comprising: An acquisition module is configured to acquire a first data segment from a first data sequence among a plurality of data sequences used to train the contrastive learning model, and to acquire a second data segment from a second data sequence among the plurality of data sequences, wherein the contrastive learning model is a machine learning model for processing video data, and the plurality of data sequences are multiple videos; The selection module is configured to select a data frame from other data sequences besides the second data sequence among the plurality of data sequences, the data frame being a video frame; A generation module is configured to generate a third data segment based on the second data segment and the data frame, wherein the first data segment, the second data segment, and the third data segment are video segments; as well as The determination module is configured to determine negative sample pairs for training the contrastive learning model based on the first data segment and the third data segment; The generation module is further configured as follows: Generate a noisy data frame based on the data frame; and The noisy data frames are used to update the data frames in the second data segment; The generation module is further configured as follows: The dimensions of the data frame are adjusted according to a predetermined ratio to generate an intermediate data frame; The intermediate data frame is copied to generate multiple copied intermediate data frames; as well as The multiple copied intermediate data frames are spliced ​​together to generate the noise data.

14. An electronic device, comprising: At least one processing unit; as well as At least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the device to perform the method according to any one of claims 1 to 12 when executed by the at least one processing unit.

15. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method according to any one of claims 1 to 12.

Images