Image data processing method and related device

The method addresses the trauma caused by disturbing content in image labeling by using an AI model to evenly distribute and mask such images, improving the labeling experience.

CN113537261BActive Publication Date: 2024-12-03HUAWEI TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010311127.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-04-20
Publication Date
2024-12-03
Estimated Expiration
2040-04-20

AI Technical Summary

Technical Problem

Existing image data processing systems cause psychological trauma to data labelers due to the display of violent, pornographic, or disturbing content, and interventions like time limits or consultations fail to effectively reduce this trauma.

Method used

An image data processing method using a preset AI model to identify suspected content, evenly distribute these images in a sequence, and display them with masks to reduce psychological discomfort.

Benefits of technology

The method reduces the psychological impact on data labelers by evenly distributing disturbing content and using masks, enhancing their predictability and comfort during the labeling process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113537261B_ABST
    Figure CN113537261B_ABST
Patent Text Reader

Abstract

The embodiment of the present application discloses an image data processing method and related equipment, which are used to sort pictures to be annotated so that pictures with preset content can be displayed evenly. The method includes: determining pictures with preset content in K pictures to be annotated through a preset artificial intelligence AI model, obtaining a suspected picture set, K is a positive integer, sorting the K pictures to be annotated according to the suspected picture set, obtaining a target picture sequence, the pictures in the suspected picture set are evenly distributed in the target picture sequence, and displaying K pictures to be annotated according to the target picture sequence, so that the display of pictures to be annotated with preset content that cause discomfort is decentralized and uniform, so that data annotators can achieve psychological predictability, thereby reducing the discomfort of the annotators.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image data processing, and in particular to an image data processing method and related equipment. Background Art

[0002] With the rapid development of the Internet, data is growing explosively, among which image data brings the highest amount of information. Driven by profit, the prosperity of Internet content platforms has led to chaos, and a large amount of illegal and illegal content has flooded the Internet. Among them, bloody / violent / child abuse / disgusting / pornographic image content has led to a poor user experience, user loss, and platform damage. At present, for the review of image content, major platforms in the industry use artificial intelligence (AI) review + manual review. The AI ​​review method can efficiently identify large amounts of repetitive and common content, while manual review can identify and mark new changes in content and some cultural metaphors.

[0003] However, bloody / violent / child abuse / disgusting / pornographic content will cause psychological and physical discomfort to data labelers who conduct manual review. Long-term exposure to these extreme contents will cause data labelers to suffer serious psychological trauma. Major platforms in the industry do not perform special processing on images that cause discomfort, and only a few platforms will intervene. Common intervention methods include providing regular consultations and limiting review time. However, even if regular consultations are provided, trauma has already occurred, and post-intervention cannot reduce the formation of trauma. Limiting the review time will reduce the amount of annotations, reduce the efficiency of annotations, and increase labor costs. Summary of the invention

[0004] The embodiments of the present application provide an image data processing method and related devices for sorting pictures to be annotated so that pictures with preset content can be displayed evenly.

[0005] In a first aspect, the present application provides an image data processing method, comprising:

[0006] Through the preset artificial intelligence AI model, the pictures with preset content in K pictures to be labeled are determined to obtain a suspected picture set, K is a positive integer, and the K pictures to be labeled are sorted according to the suspected picture set to obtain a target picture sequence. The pictures in the suspected picture set are evenly distributed in the target picture sequence, and the K pictures to be labeled are displayed according to the target picture sequence, so that the display of the pictures to be labeled with preset content that cause discomfort is decentralized and uniform, so that data labelers can achieve psychological predictability, thereby reducing the discomfort of labelers.

[0007] Optionally, in an implementation of the first aspect, determining pictures with preset content in K pictures to be labeled by a preset AI model to obtain a suspected picture set includes: determining the probability of the preset content appearing in each picture in the K pictures to be labeled by the AI ​​model, determining pictures in the K pictures to be labeled whose probability of the preset content appearing is greater than or equal to a preset probability value, and obtaining the suspected picture set. The AI ​​model can process K pictures to be labeled simultaneously in large batches, with high efficiency.

[0008] Optionally, in an implementation of the first aspect, the preset probability value is 90%.

[0009] Optionally, in an implementation of the first aspect, the preset content is inappropriate content.

[0010] Optionally, in an implementation of the first aspect, sorting the K to-be-annotated pictures according to the suspected picture set to obtain a target picture sequence includes:

[0011] Divide K to-be-annotated pictures into n groups of to-be-annotated pictures arranged in sequence, determine the number of suspected pictures assigned to each group of to-be-annotated pictures in the n groups of to-be-annotated pictures according to the ratio of suspected pictures, the ratio of the number of pictures in the suspected picture set to K, determine the number of suspected pictures in each group of to-be-annotated pictures in each n groups of to-be-annotated pictures according to the ratio of suspected pictures and k, randomly put pictures in the suspected picture set into each group of to-be-annotated pictures in the n groups of to-be-annotated pictures, randomly put pictures that are not in the suspected picture set among the K to-be-annotated pictures into each group of to-be-annotated pictures in the n groups of to-be-annotated pictures, randomly sort pictures in each group of to-be-annotated pictures in the n groups of to-be-annotated pictures, obtain a target picture sequence, and disperse and evenly display pictures to-be-annotated that cause discomfort with preset content, so that data annotators can achieve psychological predictability, thereby reducing the discomfort of annotators.

[0012] Optionally, in an implementation of the first aspect, before dividing K pictures to be labeled into n groups of pictures to be labeled arranged in sequence, it also includes: determining k according to the proportion of suspected pictures, k is less than K, 1 / k is not less than the proportion of suspected pictures, if K is an integer division of k, then n=K / k, the number of pictures in each group of pictures to be labeled in the n groups of pictures to be labeled is k, if K is not an integer division of k, then n=ceil(K / k), wherein the number of pictures from the first group of pictures to be labeled to the n-1th group of pictures to be labeled in the n groups of pictures to be labeled is k, and the number of pictures in the nth group of pictures to be labeled is mod(K, k), ceil() is a rounding-up function, and mod() is a remainder function, thereby ensuring that there is at least one suspected picture in the k pictures to be labeled, and achieving uniform distribution of the suspected picture set in the K pictures to be labeled.

[0013] Optionally, in an implementation of the first aspect, after randomly sorting the pictures in each group of pictures to be labeled in the n groups of pictures to be labeled to obtain the target picture sequence, the method further includes: performing the following steps P times:

[0014] S1. Randomly determine a first picture and a second picture, where the first picture and the second picture are pictures of different groups in the n groups of pictures to be labeled.

[0015] S2. Calculate the difference abs1 between the number of pictures with suspected content in the first m consecutive pictures to be labeled and the second m consecutive pictures to be labeled in the target picture sequence, respectively. The first m consecutive pictures to be labeled and the second m consecutive pictures to be labeled have no intersection, wherein the first m consecutive pictures to be labeled include the first picture, and the second m consecutive pictures to be labeled include the second picture.

[0016] S3. Exchange the positions of the first image and the second image to obtain a new first continuous m images to be labeled, a new second continuous m images to be labeled, and a new target image sequence, and calculate the difference abs2 between the number of images with suspected content in the new first continuous m images to be labeled and the new second continuous m images to be labeled in the new target image sequence.

[0017] S4. If abs1 is greater than abs2, update the target picture sequence according to the new target picture sequence.

[0018] This allows the images in the suspected image set to be evenly distributed among the K images to be labeled.

[0019] Optionally, in an implementation of the first aspect, the first picture is arranged at a middle position of a first series of m pictures to be labeled, and the second picture is arranged at a middle position of a second series of m pictures to be labeled.

[0020] Optionally, in an implementation of the first aspect, P is equal to K.

[0021] Optionally, in an implementation of the first aspect, before displaying K to-be-annotated pictures in sequence according to the target picture sequence, the method further includes: adding masks to pictures in the suspected picture set to alleviate discomfort for data annotators.

[0022] Optionally, in an implementation of the first aspect, the mask is a colored mask with a transparency of 45%.

[0023] Optionally, in an implementation of the first aspect, after displaying K pictures to be labeled in sequence according to the target picture sequence, it also includes: accepting labeling of any picture of the K pictures to be labeled, and after L times of labeling, displaying a preset pleasant picture to alleviate the discomfort of the data labeler.

[0024] In a second aspect, the present application provides an image data processing device, comprising:

[0025] The AI ​​module is used to determine the pictures with preset content among K pictures to be labeled through a preset artificial intelligence AI model to obtain a suspected picture set, where K is a positive integer.

[0026] The processing module is further used to sort the K to-be-annotated pictures according to the suspected picture set to obtain a target picture sequence, and the pictures in the suspected picture set are evenly distributed in the target picture sequence.

[0027] The display module is used to display K images to be labeled according to the target image sequence.

[0028] Optionally, in an implementation of the second aspect, the AI ​​module is specifically configured to:

[0029] The AI ​​model is used to determine the probability of the preset content appearing in each of the K images to be labeled.

[0030] Determine the K images to be labeled whose probability of appearing the preset content is greater than or equal to the preset probability value, and obtain a set of suspected images.

[0031] Optionally, in an implementation of the second aspect, the processing module is specifically configured to:

[0032] Divide the K images to be labeled into n groups of images to be labeled which are arranged in sequence.

[0033] The number of suspected images allocated to each group of n groups of images to be labeled is determined according to the suspected image ratio, where the suspected image ratio is the ratio of the number of images in the suspected image set to K.

[0034] The number of suspected images in each group of images to be labeled in each of the n groups of images to be labeled is determined according to the suspected image ratio and k.

[0035] The images of the suspected image set are randomly placed into each group of images to be labeled in the n groups of images to be labeled.

[0036] The images that are not in the suspected image set among the K images to be labeled are randomly placed into each group of images to be labeled in the n groups of images to be labeled.

[0037] Randomly sort the images in each group of n groups of images to be labeled to obtain a target image sequence.

[0038] Optionally, in an implementation of the second aspect, the processing module is further used to determine k according to the ratio of suspected pictures, k is less than K, and 1 / k is not less than the ratio of suspected pictures. If K is an integer division of k, then n=K / k, and the number of pictures in each group of n groups of pictures to be labeled is k. If K is not an integer division of k, then n=ceil(K / k), wherein the number of pictures from the first group of pictures to be labeled to the n-1th group of pictures to be labeled in the n groups of pictures to be labeled is k, and the number of pictures in the nth group of pictures to be labeled is mod(K, k), ceil() is a rounding up function, and mod() is a remainder function.

[0039] Optionally, in an implementation of the second aspect, the processing module is further configured to perform the following steps P times:

[0040] S1. Randomly determine a first picture and a second picture, where the first picture and the second picture are pictures of different groups in the n groups of pictures to be labeled.

[0041] S2. Calculate the difference abs1 between the number of pictures with suspected content in the first m consecutive pictures to be labeled and the second m consecutive pictures to be labeled in the target picture sequence, respectively. The first m consecutive pictures to be labeled and the second m consecutive pictures to be labeled have no intersection, wherein the first m consecutive pictures to be labeled include the first picture, and the second m consecutive pictures to be labeled include the second picture.

[0042] S3. Exchange the positions of the first image and the second image to obtain a new first continuous m images to be labeled, a new second continuous m images to be labeled, and a new target image sequence, and calculate the difference abs2 between the number of images with suspected content in the new first continuous m images to be labeled and the new second continuous m images to be labeled in the new target image sequence.

[0043] S4. If abs1 is greater than abs2, update the target picture sequence according to the new target picture sequence.

[0044] Optionally, in an implementation of the second aspect, the display module is further used to add a mask to the pictures in the suspected picture set.

[0045] Optionally, in an implementation of the second aspect, the labeling module is used to accept labeling of any image of the K images to be labeled.

[0046] Optionally, in an implementation of the second aspect, the display module is further used to display a preset pleasant picture after L times of labeling.

[0047] In a third aspect, the present application provides a computer-readable storage medium, including a program, which, when executed on a computer, enables the computer to execute the steps described in any one of the first aspect and possible implementation methods.

[0048] In a fourth aspect, the present application provides an image data processing device, including a processor and a memory, wherein the processor is coupled to the memory, and the memory is used to store a program. The processor is used to execute the program in the memory, so that the image data processing device performs any of the steps in the first aspect and possible implementations described above.

[0049] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0050] Through the preset artificial intelligence AI model, the pictures with preset content in K pictures to be labeled are determined to obtain a suspected picture set, K is a positive integer, and the K pictures to be labeled are sorted according to the suspected picture set to obtain a target picture sequence. The pictures in the suspected picture set are evenly distributed in the target picture sequence, and the K pictures to be labeled are displayed according to the target picture sequence, so that the display of the pictures to be labeled with preset content that cause discomfort is decentralized and uniform, so that data labelers can achieve psychological predictability, thereby reducing the discomfort of labelers. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a schematic diagram of the structure of the image data processing device used in this application;

[0052] Figure 2-1 A schematic diagram of an embodiment of an image data processing method provided by the present application;

[0053] Figure 2-2 Schematic diagram of inappropriate pictures;

[0054] Figure 2-3 A schematic diagram of a cat picture;

[0055] Figure 2-4 This is a schematic diagram identified by the AI ​​model;

[0056] Figure 2-5 A schematic diagram of a method for sorting K images to be labeled provided in an embodiment of the present application;

[0057] Figure 2-6 Schematic diagram of the pictures in the two groups respectively;

[0058] Figure 2-7 Schematic diagrams after the pictures in the two groups are swapped;

[0059] Figure 2-8 Schematic diagram of adding a mask to an image;

[0060] Figure 2-9 A schematic diagram of a pleasant picture;

[0061] Figure 3 A schematic diagram of an embodiment of an image data processing device provided by the present application;

[0062] Figure 4 A schematic diagram of an embodiment of an image data processing device provided in the present application. DETAILED DESCRIPTION

[0063] The embodiments of the present application provide an image data processing method and related devices for sorting and displaying multiple images to be labeled.

[0064] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0065] The embodiments of the present application can be applied to Figure 1 In the image data processing device 100 shown, the image data processing device 100 includes a processor 110, a display 120, a memory 130, a transceiver 140 and an input unit 150 (such as a keyboard, a mouse, a touch screen, etc., which are not limited here).

[0066] The processor 1180 is the control center of the image data processing device 100. It uses various interfaces and lines to connect various parts of the image data processing device 100, and executes various functions of the mobile phone and processes data by running or executing software programs and / or modules stored in the memory 130, and calling data stored in the memory 130, so as to monitor the mobile phone as a whole. Optionally, the processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application programs, etc., and the modem processor mainly processes wireless communications. It can be understood that the above-mentioned modem processor may not be integrated into the processor 110.

[0067] The display 120 can be used to display information input by the user or information provided to the user and various interfaces of the image data processing device 100. The display 120 may include a display panel, and optionally, the display panel may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel may cover the display panel, and when the touch panel detects a touch operation on or near it, it is transmitted to the processor 110 to determine the type of the touch event, and then the processor 110 provides a corresponding visual output on the display panel according to the type of the touch event.

[0068] The memory 130 may include a read-only memory and a random access memory, and provides instructions and data to the processor 110. A portion of the memory 130 may also include a non-volatile random access memory (NVRAM). The memory 130 stores processor and operation instructions, executable modules or data structures, or subsets thereof, or extended sets thereof, wherein the operation instructions may include various operation instructions for implementing various operations.

[0069] The transceiver 140 may be used to receive input digital or character information and generate signal input related to the relevant settings and function control of the terminal device. The transceiver 140 may be used to output digital or character information through the first interface; the transceiver 140 may also be used to send instructions to the disk group through the first interface to modify the data in the disk group; the transceiver 140 may also include a display device such as a display screen.

[0070] The input unit 150 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile phone. Specifically, the input unit 150 may include a touch panel and other input devices. The touch panel, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on or near the touch panel using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel may include a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 110, and can receive and execute commands sent by the processor 110. In addition, the touch panel can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel, the input unit 150 may also include other input devices. Specifically, other input devices may include, but are not limited to, one or more of a physical keyboard, function keys (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, and the like.

[0071] In some possible implementations, the image data processing device 100 may be a terminal device or a server, which is not limited here.

[0072] With the rapid development of the Internet, data is growing explosively, among which images bring the highest amount of information. Driven by profit, the prosperity of Internet content platforms has led to chaos, and a large amount of illegal and illegal content has flooded the Internet. Among them, bloody / violent / child abuse / disgusting / pornographic images have led to poor user experience, user loss, and platform damage. At present, for the review of image content, major platforms in the industry currently use artificial intelligence (AI) review + manual review. The AI ​​review method can efficiently identify large amounts of repetitive and common content, while manual review can identify and mark new changes in content and some cultural metaphors.

[0073] However, bloody / violent / child abuse / disgusting / pornographic content will cause psychological and physical discomfort to data labelers who are manually reviewing. Long-term exposure to these extreme contents will cause data labelers to suffer serious psychological trauma. Major platforms in the industry do not perform special processing on images that cause discomfort, and only a few platforms will intervene. Common intervention methods include providing regular consultations and limiting review time. However, even if regular consultations are provided, trauma has already occurred, and post-intervention cannot reduce the formation of trauma. Limiting the review time will reduce the amount of annotations, reduce the efficiency of annotations, and increase labor costs.

[0074] To do this, refer to Figure 2-1 , the present application proposes an image data processing method, comprising:

[0075] 201. Determine suspected images that are suspected to have preset contents among the K images to be labeled by using a preset AI model to obtain a set of suspected images.

[0076] In the embodiment of the present application, the K images to be annotated are the images that the data annotator actually needs to annotate. For example, images uploaded by users on social networking sites or knowledge sharing platforms. The K images to be annotated can all come from the same platform or from different platforms, which is not limited here. The data annotator can be a staff member of the platform or a staff member of a third-party organization that specializes in reviewing content, which is not limited here.

[0077] In some possible implementations, the preset content may be uncomfortable content, such as disgusting, pornographic, violent, terrifying, and other content that causes psychological or physical discomfort to people. Figure 2-2 In some possible implementations, the preset content may also be certain specific types of content, such as cat images, portraits, cartoons, etc., for example Figure 2-3 The cat images shown are not limited here.

[0078] In the embodiment of the present application, the AI ​​model can be a pre-trained AI model, or an AI model obtained by obtaining a certain number of pictures for training in actual work. Specifically, after obtaining multiple training pictures, the data annotation personnel can annotate the multiple training pictures one by one, and then input the annotated pictures into the AI ​​model to be trained to obtain the trained AI model.

[0079] For example, 10,000 pictures are obtained from a social networking site. First, the data annotators annotate the 10,000 pictures to determine whether each picture has the preset content. After the data annotators annotate each of the 10,000 pictures, they hand them over to the graphics processing unit (GPU) to train the AI ​​model to be trained, and obtain the various parameters of the AI ​​model, that is, to obtain the trained AI model. After the AI ​​model training is completed, the AI ​​model can be used to identify K pictures to be annotated.

[0080] Specifically, AI models can perform content review in the following ways:

[0081] (1) Determine the probability of the preset content appearing in each of the K images to be labeled through the AI ​​model.

[0082] For example, Figure 2-4As shown in the figure, there are 4 pictures to be labeled among the K pictures to be labeled, namely picture a, picture b, picture c and picture d. The AI ​​model can determine which pictures have preset content. For example, the preset content is cat, that is, the AI ​​model needs to identify which pictures have cats in each picture. The probability calculated by the AI ​​model for the 4 pictures is: the probability of the preset content appearing in picture a is 5%; the probability of the preset content appearing in picture b is 93%; the probability of picture c is 95%, and the probability of picture d is 65%.

[0083] (2) Determine the K images to be labeled whose probability of appearing the preset content is greater than or equal to the preset probability value, and obtain a set of suspected images.

[0084] For example, set the preset probability value to 90%. Then, Figure 2-4 Among the four pictures, the probability of the preset content appearing in pictures b and c is greater than or equal to 90%, and the probability of the preset content appearing in pictures a and d is less than 90%. Therefore, the AI ​​model initially determines that pictures b and c are suspected pictures and belong to the suspected picture set, while pictures a and d do not belong to the suspected picture set. It should be noted that the preset probability value can be set manually or as a better experience value obtained through machine learning, which is not limited here.

[0085] From the above, we can see that the AI ​​model recognizes image b as a cat, but it is obviously a tiger; the AI ​​model does not recognize image a as a cat, but it is obviously a cartoon cat. Therefore, the AI ​​model may not accurately identify whether the image has the preset content. The AI ​​model can only perform preliminary recognition on K images to be labeled and obtain a set of suspected images. Then, the suspected images need to be further identified and labeled through manual review.

[0086] In the embodiment of the present application, the number of pictures in the suspected picture set is assumed to be N. For example, assuming K=1000, N=100, that is, 1000 pictures to be labeled are identified by the AI ​​model, and the number of pictures in the suspected picture set determined to be 100.

[0087] 202. Sort the K to-be-annotated images according to the suspected image set to obtain a target image sequence, wherein the images in the suspected image set are evenly distributed in the target image sequence.

[0088] In the embodiment of the present application, K images to be annotated can be sorted according to the suspected image set to obtain a target image sequence, and the images in the suspected image set are evenly distributed in the target image sequence. Finally, K images to be annotated are displayed according to the target image sequence, so that the display of the images to be annotated with preset content that cause discomfort is dispersed and uniform, so that data annotators can achieve psychological predictability, thereby reducing the discomfort of the annotators.

[0089] For details, please refer to Figure 2-5 ,The following describes the various steps of sorting K images to be labeled.

[0090] 2021. Divide the K images to be labeled into n groups of images to be labeled.

[0091] In some possible implementations, K images to be labeled can be divided into n groups of images to be labeled with the same number, and then the images in the suspected image set are evenly distributed to each group of images to be labeled, and then the non-suspected images in the K images to be labeled are evenly distributed to each group of images to be labeled. Finally, the images assigned to each group of images to be labeled (including suspected images and non-suspected images) are randomly arranged to obtain a target image sequence.

[0092] For example, assuming K = 1000, N = 100, then the suspected image set accounts for 10% of the K images to be annotated, so it can be set that for every 10 images that appear, there is one suspected image. Optionally, the suspected image appears at a fixed position among the 10 images, such as the 5th or 10th. For example, let 0 be an image that is not in the suspected image set and 1 be an image that is in the suspected image set, then it can be set as:

[0093] 00000 00001 00000 00001 00000 00001 00000 00001 ……

[0094] In some feasible implementations, it can also be set as:

[0095] 00001 00000 00001 00000 00001 00000 00001 00000 ……

[0096] In some possible implementations, the suspected image may also appear in the first or second image, which is not limited here.

[0097] For another example, suppose 1,000 images to be labeled are divided into 10 groups, each with 100 images to be labeled, and the number of images in the suspected image set is 40. Then, these 40 suspected images can be evenly distributed to these 10 groups of images to be labeled, so that each group of images to be labeled has 40 / 10=4 suspected images. Then, the remaining 960 images are distributed to 10 groups of images to be labeled, with 96 images in each group. Finally, each group of images to be labeled is randomly arranged to obtain the target image sequence.

[0098] In some possible implementations, the K images to be annotated may be divided into groups of images to be annotated with different numbers, for example, some groups may have 101 images, some groups may have 99 images, or some groups may have 150 images, and some groups may have 50 images, which is not limited here.

[0099] In some possible implementations, if the number of pictures in the suspected picture set is 45, then each group of pictures to be labeled can be allocated an average of 45 / 10=4.5 suspected pictures. However, since the allocation of pictures must be in units of 1, that is, the number of suspected pictures allocated to each group of pictures to be labeled must be an integer. To this end, in some possible implementations, 5 pictures can be allocated to some groups and 4 pictures can be allocated to some groups. For example, the odd group is allocated 5 suspected pictures and the even group is allocated 4 suspected pictures. In some possible implementations, the number of groups can also be re-determined, for example, K pictures to be labeled are divided into 5 groups, each with 9 suspected pictures. The reason for determining 5 groups is that 1000 can divide 5, and 45 can also divide 5, so that the K pictures to be labeled and the 45 suspected pictures can be evenly divided into 5 groups, that is, the greatest common divisor of 1000 and 45 is determined.

[0100] However, if the number of pictures in the suspected picture set is 43, since 1000 and 43 are mutually prime, the greatest common divisor is 1. If 1 picture is 1 group, since there are 43 suspected pictures in the suspected picture set, there will be 957 groups that cannot be assigned to suspected pictures, which will cause unevenness. Therefore, in some possible implementations, the grouping method can be determined according to the ratio of suspected pictures.

[0101] Specifically, calculate the proportion of the suspected picture set in the K pictures to be labeled, and obtain the suspected picture ratio N / K, where N is the number of pictures in the suspected picture set. For example, N / K=43 / 1000=4.3%. Then, assign 4 or 5 pictures in the suspected picture set to each group of pictures to be labeled. In some possible implementations, 4 suspected pictures can also be assigned to 7 groups of pictures to be labeled, and 5 suspected pictures can be assigned to the other 3 groups of pictures to be labeled. The group with 5 suspected pictures can be ranked last or in front, or randomly arranged among the 10 groups, or assigned to the 1st, 4th, and 7th groups, or 2nd, 5th, and 8th groups, without limitation here.

[0102] In some possible implementations, assuming that N=4, that is, the number of pictures in the suspected picture set is 4, then the suspected picture ratio is 0.4%. Then, 1 or 0 suspected pictures can be assigned to each group of pictures to be labeled. Then, 6 groups of pictures to be labeled will not be assigned suspected pictures.

[0103] To this end, in order to achieve the above-mentioned various situations, after being divided into n groups, in the group of k pictures to be labeled, at least one suspected picture can be divided, that is, k*N / K>1, the following method can be used to group the K pictures to be labeled:

[0104] 1. Determine k based on N and K.

[0105] It should be noted that, in the embodiment of the present application, k is used to group K pictures to be labeled. For example, K pictures to be labeled are divided into n groups of pictures to be labeled according to k, and the number of pictures in the first n-1 groups of pictures to be labeled is k, and the number of pictures in the nth group of pictures to be labeled is mod(K, k).

[0106] (1) The default value of k is preset as k0.

[0107] First, the default value k0 of k can be determined based on experience. For example, assuming K = 1000, then k can be set to 100. In some possible implementations, k can also be set to 50, 200, 250, or 43, which is not limited here. Optionally, it is preferred to set the k value to a number that is divisible by K, so that it can be divided into n groups of equal number of pictures to be labeled. If K cannot divide k, then the K pictures to be labeled can also be divided into n groups, where the number of pictures in the first n-1 groups to be labeled is k, and the number of pictures in the nth group to be labeled is mod(K, k). In some possible implementations, a group with a number of pictures mod(K, k) can be assigned to any group, such as Group 1, or Group 5, which is not limited here.

[0108] (2) Determine whether N / k*k0 is less than 1.

[0109] It should be noted that N / K is the proportion of the suspected picture set in the K pictures to be labeled, and N / K*k0 indicates how many suspected pictures can be assigned to the k0 pictures to be labeled if k=k0. In order to allocate at least one suspected picture to the group of pictures to be labeled with k pictures, N / K*k must be greater than or equal to 1. Here, let k=k0, calculate N / K*k0, and determine whether it is less than 1. If N / K*k0 is greater than or equal to 1, there is no need to adjust k, that is, k=k0; if N / K*k0 is less than 1, it means that k=k0 is too small and the value of k needs to be increased.

[0110] (3) If N / k*k0 is greater than or equal to 1, then let k=k0; if N / k*k0 is less than 1, then let k=ceil(K / N).

[0111] Specifically, in order to increase k, N / K*k is greater than 1, k is greater than K / N, that is, k=ceil(K / N). To this end, the general formula for determining the value of k is:

[0112]

[0113] For example, assuming K=1000, k0=100, when N=43, since k0*N / K≥1, k=k0=100; for example, assuming K=1000, k0=100, when N=4, since k0*N / K<1, k=ceil(K / N)=250.

[0114] In some possible implementations, in addition to requiring each group of k pictures to have at least one suspected picture, each group of k pictures is also required to have at least one non-suspicious picture. Suppose the number of non-suspicious pictures in the K pictures to be labeled is M, then:

[0115]

[0116] For example, K=1000, N=995, M=5, then since k0*M / K=0.5, k=ceil(K / M)=2000; K=1000, N=997, M=3, then since k0*M / K=0.3, k=ceil(K / M)=334.

[0117] 2. Determine n based on k.

[0118] After the k value is determined, the n value can be determined. Specifically, n=ceil(K / k).

[0119] For example, suppose K=1000, k=100, then n=ceil(K / k)=10; suppose K=1000, k=250, then n=ceil(K / k)=4; suppose K=1000, k=55, then n=ceil(K / k)=19.

[0120] 3. Determine the grouping scheme for the K images to be labeled based on k, n, N, and K.

[0121] In this embodiment of the present application, assuming that K is divisible by k, then the number of pictures in each group of pictures to be labeled in the n groups of labeled pictures is equal to k; assuming that K is not divisible by k, then the number of pictures in each group of pictures to be labeled in the first n-1 groups of labeled pictures is equal to k, and the number of pictures to be labeled in the nth group of pictures to be labeled is mod(K / k).

[0122] For example, if K=1000, k=100, it is divided into K / k=10 groups, and the number of pictures in each of the 10 groups of pictures to be labeled is 100; if K=1000, k=250, it is divided into K / k=4 groups, and the number of pictures in each of the 4 groups of pictures to be labeled is 250; if K=1000, k=55, it is divided into ceil(K / k)=19 groups, and the number of pictures in each of the first 18 groups of pictures to be labeled is 55, and the number of pictures in the nth group of pictures to be labeled is 10.

[0123] 2022. Evenly distribute the suspected image set into n groups of images to be labeled, and distribute the non-suspected images into n groups of images to be labeled, to obtain a target image sequence.

[0124] After the grouping scheme is determined, the images in the suspected image set can be evenly distributed into n groups of images to be labeled.

[0125] For example, K=1000, N=3, k=334, then n=3, each group can be assigned to 1 suspected picture; K=1000, N=995, then M=5, then k=ceil(k / M)=200, that is, n=ceil(K / k)=5, then there are 200 pictures to be labeled in each of the 5 groups of pictures to be labeled, and each group of pictures to be labeled has 1 non-suspected picture and 199 suspected pictures; K=1000, N=50, k=100, then n=10, that is, the number of pictures in each group of pictures to be labeled is 100, and each group of pictures to be labeled can be assigned to 2 suspected pictures.

[0126] For example, if K=1000, N=40, that is, there are 1000 pictures to be labeled, and 40 suspected pictures are initially identified, that is, the number of pictures in the suspected picture set is 40. Then the 1000 pictures to be labeled can be divided into 10 groups, that is, n=10, that is, the 1000 pictures to be labeled are divided into 10 groups, each with 100 pictures, and each group of pictures to be labeled can be assigned 4 suspected pictures. Then fill these 10 groups with pictures from the non-suspected picture set among the 1000 pictures to be labeled, that is, each group is assigned 96 pictures from the non-suspected picture set. Then, randomly arrange the pictures in each group to obtain the arrangement order of the pictures in each group. Finally, get the arrangement order of each group and remember the target picture sequence.

[0127] After dividing the K images to be labeled into n groups, the images in the suspected image set are evenly distributed to the n groups, and then the images in the non-suspected image set are used to fill the remaining number of each group. The suspected images are evenly distributed in each group of images to be labeled.

[0128] 2023. Adjust the position of individual images in the target image sequence.

[0129] In some possible implementations, the suspected images can be further evenly distributed. In an embodiment of the present application, the specific method for further evenly distributing the suspected images is to randomly select two images to be labeled that are not in the same group, calculate the number of suspected images around the two images to be labeled, calculate the difference in number, and obtain abs1; then swap the positions of the two images to be labeled, calculate the number of suspected images around the two images to be labeled, calculate the difference in number, and obtain abs2. Finally, compare abs1 and abs2. If abs1 is greater than abs2, it means that the original distribution is relatively uneven, and the positions of the two images to be labeled need to be swapped; if abs1 is less than or equal to abs2, it means that the original distribution is relatively even, and the two images can remain in their original positions without swapping positions.

[0130] Specifically, the following steps may be performed P times:

[0131] S1. Randomly determine a first picture and a second picture, where the first picture and the second picture are pictures of different groups in the n groups of pictures to be labeled.

[0132] For example, Figure 2-6 The first picture and the second picture are shown, the first picture is in the 1st group, and the second picture is in the 4th group.

[0133] S2. Calculate the difference abs1 between the number of suspected images in the first m consecutive images to be labeled and the number of suspected images in the second m consecutive images to be labeled in the target image sequence, respectively. The first m consecutive images to be labeled and the second m consecutive images to be labeled have no intersection, wherein the first m consecutive images to be labeled include the first image, and the second m consecutive images to be labeled include the second image.

[0134] In some possible implementations, suspected images can be marked as 1, and non-suspected images can be marked as 0. Then, the number of suspected images in the first consecutive m images to be labeled including the first image, and the number of suspected images in the second consecutive m images to be labeled including the second image can be calculated. In some possible implementations, the first image is the median of the first consecutive m images to be labeled, and the second image is the median of the second consecutive m images to be labeled. It should be noted that abs1 represents "unevenness". The larger the value of abs1, the greater the difference in the number of suspected images between the first consecutive m images to be labeled and the second consecutive m images to be labeled.

[0135] For example, the number of suspected images in the first m consecutive images to be labeled is 24, and the number of suspected images in the second m consecutive images to be labeled is 20, then abs1=24-20=4.

[0136] In some possible implementations, the first picture is arranged in the middle of the first m consecutive pictures to be labeled, and the second picture is arranged in the middle of the second m consecutive pictures to be labeled. For example, m=101, that is, 50 pictures are taken before and after the first picture / the second picture, and a total of 101 pictures including the first picture / the second picture. In some possible implementations, m=k can also be set, which is not limited here.

[0137] S3. Exchange the positions of the first image and the second image to obtain new first m consecutive images to be labeled and new second m consecutive images to be labeled, and calculate the difference abs2 between the number of images with suspected images in the new first m consecutive images to be labeled and the number of images in the new second m consecutive images to be labeled.

[0138] In order to further evenly distribute the suspected images in the n groups of images to be labeled, the first image and the second image can be swapped to obtain a new first continuous m images to be labeled, a new second continuous m images to be labeled, and a new target image sequence. That is, after the first image and the second image are swapped, the first image belongs to the new second continuous m images to be labeled, and the second image belongs to the new first continuous m images to be labeled. Then, the difference abs2 between the number of images with suspected images in the new first continuous m images to be labeled and the new second continuous m images to be labeled in the new target image sequence is calculated.

[0139] For example, the number of suspected images in the first m consecutive images to be labeled is 25, and the number of suspected images in the second m consecutive images to be labeled is 19, then abs1=25-19=6; for example, Figure 2-7 As shown, the number of suspected images in the first m consecutive images to be labeled is 23, and the number of suspected images in the second m consecutive images to be labeled is 21, so abs1=23-21=2.

[0140] S4. If abs1 is greater than abs2, update the target picture sequence according to the new target picture sequence.

[0141] If the number of suspected images in the first m consecutive images to be labeled is 25, and the number of suspected images in the second m consecutive images to be labeled is 19, then abs1 = 25-19 = 6. It can be seen that since the unevenness is greater after the first and second images are swapped, the two images retain their original positions.

[0142] For example, the number of suspected images in the first m consecutive images to be labeled is 23, and the number of suspected images in the second m consecutive images to be labeled is 21, then abs1 = 23-21 = 2. It can be seen that since the unevenness becomes smaller after the first image and the second image are swapped, the two images need to be swapped.

[0143] The above steps S1-S4 are performed P times, so that the suspected images in the K images to be labeled are distributed more evenly. In some possible implementations, P=K, where K is an empirical value obtained through experiments.

[0144] 203. Display K to-be-annotated images in sequence according to the target image sequence.

[0145] It should be noted that each of the n groups of images to be annotated has a serial number, namely, group 1, group 2, ..., group n. In the embodiment of the present application, group 1, group 2, ..., group n can be displayed in sequence, and the display of each group is displayed according to the target image sequence determined in the above steps 201-202. In some possible implementations, the images to be annotated can be displayed one by one, or several images, such as 5 or n images, can be displayed at the same time.

[0146] In the embodiment of the present application, the data annotator's annotation is accepted during the process of displaying K pictures to be annotated. Specifically, if the data annotator believes that it is a suspected picture, that is, it has preset content, it is marked as "yes", otherwise it is marked as "no" or not marked. If the preset content is inappropriate content, when it is displayed on the website, this picture is blocked, or it is directly deleted, or the post is directly deleted, which is not limited here.

[0147] In some possible implementations, in order to reduce the discomfort, the suspected image can be added to a mask. Specifically, the mask is a colored mask with a transparency of 45%. Optionally, a colored mask with other degrees of transparency, such as blue or green, can also be selected. For example, Figure 2-8 The mask shown.

[0148] In some possible implementations, if the next picture to be labeled is a suspected picture, it can be announced that the next picture will be a suspected picture, so that the data labeler can be mentally prepared.

[0149] In some possible implementations, after the data annotator annotates the to-be-annotated pictures with preset contents for L times, a preset pleasant picture is displayed. Figure 2-9 As shown, the pleasant pictures may be scenery, beauties, babies, cartoons, etc., and are not limited here.

[0150] Please refer to Figure 3 , the present application also provides an image data processing device 300, comprising:

[0151] The AI ​​module 310 is used to determine the pictures with preset content among K pictures to be labeled through a preset artificial intelligence AI model to obtain a suspected picture set, where K is a positive integer.

[0152] The processing module 320 is further used to sort the K to-be-annotated pictures according to the suspected picture set to obtain a target picture sequence, and the pictures in the suspected picture set are evenly distributed in the target picture sequence.

[0153] The display module 330 is used to display K pictures to be labeled according to the target picture sequence.

[0154] Optionally, the AI ​​module 310 is specifically used for:

[0155] The AI ​​model is used to determine the probability of the preset content appearing in each of the K images to be labeled.

[0156] Determine the K images to be labeled whose probability of appearing the preset content is greater than or equal to the preset probability value, and obtain a set of suspected images.

[0157] Optionally, the processing module 320 is specifically configured to:

[0158] Divide the K images to be labeled into n groups of images to be labeled which are arranged in sequence.

[0159] The number of suspected images allocated to each group of n groups of images to be labeled is determined according to the suspected image ratio, where the suspected image ratio is the ratio of the number of images in the suspected image set to K.

[0160] The number of suspected images in each group of images to be labeled in each of the n groups of images to be labeled is determined according to the suspected image ratio and k.

[0161] The images of the suspected image set are randomly placed into each group of images to be labeled in the n groups of images to be labeled.

[0162] The images that are not in the suspected image set among the K images to be labeled are randomly placed into each group of images to be labeled in the n groups of images to be labeled.

[0163] Randomly sort the images in each group of n groups of images to be labeled to obtain a target image sequence.

[0164] Optionally, the processing module 320 is further used to determine k according to the ratio of suspected pictures, k is less than K, and 1 / k is not less than the ratio of suspected pictures. If K is an integer division of k, then n=K / k, and the number of pictures in each group of n groups of pictures to be labeled is k. If K is not an integer division of k, then n=ceil(K / k), where the number of pictures from the first group of pictures to be labeled to the n-1th group of pictures to be labeled in the n groups of pictures to be labeled is k, and the number of pictures in the nth group of pictures to be labeled is mod(K, k), ceil() is a rounding up function, and mod() is a remainder function.

[0165] Optionally, in an implementation of the second aspect, the processing module 320 is further configured to perform the following steps P times:

[0166] S1. Randomly determine a first picture and a second picture, where the first picture and the second picture are pictures of different groups in the n groups of pictures to be labeled.

[0167] S2. Calculate the difference abs1 between the number of pictures with suspected content in the first m consecutive pictures to be labeled and the second m consecutive pictures to be labeled in the target picture sequence, respectively. The first m consecutive pictures to be labeled and the second m consecutive pictures to be labeled have no intersection, wherein the first m consecutive pictures to be labeled include the first picture, and the second m consecutive pictures to be labeled include the second picture.

[0168] S3. Exchange the positions of the first image and the second image to obtain a new first continuous m images to be labeled, a new second continuous m images to be labeled, and a new target image sequence, and calculate the difference abs2 between the number of images with suspected content in the new first continuous m images to be labeled and the new second continuous m images to be labeled in the new target image sequence.

[0169] S4. If abs1 is greater than abs2, update the target picture sequence according to the new target picture sequence.

[0170] Optionally, the display module 330 is further configured to add a mask to the images in the suspected image set.

[0171] Optionally, the image data processing device 300 further includes a labeling module 340 for accepting a labeling of any of the K images to be labeled.

[0172] Optionally, the display module 330 is further configured to display a preset pleasant picture after L times of labeling.

[0173] Please refer to Figure 4 The present application provides an image data processing device 400, including a processor 410 and a memory 420, wherein the processor 410 is coupled to the memory 420, and the memory 420 is used to store programs. The processor 410 is used to execute the program in the memory 420, so that the image data processing device 400 performs the steps of the method described in FIG. 2 above.

[0174] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0175] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a server, or a data center to another website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0176] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0177] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0178] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0179] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0180] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.

[0181] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for processing image data, characterized in that: include: Determine the images with preset content among K images to be labeled through a preset artificial intelligence AI model to obtain a suspected image set, where K is a positive integer; Sorting the K to-be-annotated pictures according to the suspected picture set to obtain a target picture sequence, wherein the pictures in the suspected picture set are evenly distributed in the target picture sequence; The K pictures to be labeled are displayed according to the target picture sequence.

2. The method according to claim 1, characterized in that: The method of determining the pictures with preset contents among the K pictures to be annotated by using a preset artificial intelligence AI model to obtain a suspected picture set includes: Determine, by using the artificial intelligence AI model, the probability that the preset content appears in each of the K pictures to be annotated; Determine the pictures in which the probability of the preset content appearing in the K pictures to be annotated is greater than or equal to a preset probability value, and obtain the suspected picture set.

3. The method according to claim 2, characterized in that: The preset probability value is 90%.

4. The method according to any one of claims 1 to 3, characterized in that: The preset content is inappropriate content.

5. The method according to claim 4, characterized in that: The step of sorting the K to-be-annotated pictures according to the suspected picture set to obtain a target picture sequence includes: Divide the K to-be-annotated pictures into n groups of to-be-annotated pictures arranged in sequence; Determining the number of suspected pictures assigned to each group of pictures to be labeled in the n groups of pictures to be labeled according to the suspected picture ratio, wherein the suspected picture ratio is the ratio of the number of pictures in the suspected picture set to K; Determine the number of suspected images in each group of images to be labeled in each of the n groups of images to be labeled according to the suspected image ratio and K; Randomly placing the pictures in the suspected picture set into each group of pictures to be labeled in the n groups of pictures to be labeled; Randomly placing the pictures that are not in the suspected picture set among the K pictures to be labeled into each group of pictures to be labeled in the n groups of pictures to be labeled; Randomly sort the pictures in each group of pictures to be labeled in the n groups of pictures to be labeled to obtain the target picture sequence.

6. The method according to claim 5, characterized in that: Before dividing the K to-be-annotated pictures into n groups of to-be-annotated pictures arranged in sequence, the method further includes: Determine k according to the ratio of the suspected image, k is less than K, and 1 / k is not less than the ratio of the suspected image; If K is an integer that divides k, then n=K / k, and the number of pictures in each group of the n groups of pictures to be labeled is k; If K does not divide k, then n=ceil(K / k), wherein the number of pictures from the first group of pictures to be labeled to the n-1th group of pictures to be labeled in the n groups of pictures to be labeled is k, the number of pictures in the nth group of pictures to be labeled is mod(K, k), ceil() is a rounding up function, and mod() is a remainder function.

7. The method according to claim 5, characterized in that: After randomly sorting the pictures in each group of the n groups of pictures to be labeled to obtain the target picture sequence, the method further includes: Perform the following steps P times: S1. Randomly determine a first picture and a second picture, where the first picture and the second picture are pictures of different groups in the n groups of pictures to be labeled; S2, respectively calculating the difference abs1 between the number of pictures having the suspected picture in the first continuous m pictures to be labeled and the second continuous m pictures to be labeled in the target picture sequence, wherein the first continuous m pictures to be labeled and the second continuous m pictures to be labeled have no intersection, wherein the first continuous m pictures to be labeled include the first picture, and the second continuous m pictures to be labeled include the second picture; S3, swapping the positions of the first picture and the second picture to obtain new first continuous m pictures to be labeled, new second continuous m pictures to be labeled, and a new target picture sequence, and respectively calculating the difference abs2 between the number of pictures having the suspected picture in the new first continuous m pictures to be labeled and the new second continuous m pictures to be labeled in the new target picture sequence; S4. If the abs1 is greater than the abs2, updating the target picture sequence according to the new target picture sequence.

8. The method according to claim 7, characterized in that: The first picture is arranged at a middle position of the first m consecutive pictures to be labeled, and the second picture is arranged at a middle position of the second m consecutive pictures to be labeled.

9. The method according to claim 7, characterized in that: P equals K.

10. The method according to claim 4, characterized in that: Before displaying the K to-be-annotated pictures according to the target picture sequence, the method further includes: Add masks to the pictures in the suspected picture set.

11. The method according to claim 10, characterized in that: The mask is a colored mask with 45% transparency.

12. The method according to claim 4, characterized in that: After displaying the K to-be-annotated pictures according to the target picture sequence, the method further includes: Accept the labeling of any of the K pictures to be labeled.

13. The method according to claim 12, characterized in that: The method further comprises: After L times of annotation, a preset pleasant picture is displayed.

14. An image data processing device, characterized in that: include: The AI ​​module is used to determine the pictures with preset content among K pictures to be labeled through a preset artificial intelligence AI model to obtain a suspected picture set, where K is a positive integer; The processing module is further used to sort the K to-be-annotated pictures according to the suspected picture set to obtain a target picture sequence, wherein the pictures in the suspected picture set are evenly distributed in the target picture sequence; A display module is used to display the K to-be-annotated pictures according to the target picture sequence.

15. The device according to claim 14, characterized in that: The AI ​​module is specifically used for: Determine, by using the artificial intelligence AI model, the probability that the preset content appears in each of the K pictures to be annotated; Determine the pictures in which the probability of the preset content appearing in the K pictures to be annotated is greater than or equal to a preset probability value, and obtain the suspected picture set.

16. The device according to claim 14 or 15, characterized in that The processing module is specifically used for: Divide the K to-be-annotated pictures into n groups of to-be-annotated pictures arranged in sequence; Determining the number of suspected pictures assigned to each group of pictures to be labeled in the n groups of pictures to be labeled according to the suspected picture ratio, wherein the suspected picture ratio is the ratio of the number of pictures in the suspected picture set to K; Determine the number of suspected images in each group of images to be labeled in each of the n groups of images to be labeled according to the suspected image ratio and K; Randomly placing the pictures in the suspected picture set into each group of pictures to be labeled in the n groups of pictures to be labeled; Randomly placing the pictures that are not in the suspected picture set among the K pictures to be labeled into each group of pictures to be labeled in the n groups of pictures to be labeled; Randomly sort the pictures in each group of pictures to be labeled in the n groups of pictures to be labeled to obtain the target picture sequence.

17. The device according to claim 16, characterized in that The processing module is further used to determine k according to the suspected image ratio, k is less than K, and 1 / k is not less than the suspected image ratio; if K is divisible by k, then n=K / k, and the number of images in each group of the n groups of images to be labeled is k; if K is not divisible by k, then n=ceil(K / k), wherein the number of images from the first group of images to be labeled to the n-1th group of images to be labeled in the n groups of images to be labeled is k, and the number of images in the nth group of images to be labeled is mod(K, k), ceil() is a rounding up function, and mod() is a remainder function.

18. The device according to claim 16, characterized in that The processing module is further used to perform the following steps P times: S1. Randomly determine a first picture and a second picture, where the first picture and the second picture are pictures of different groups in the n groups of pictures to be labeled; S2, respectively calculating the difference abs1 between the number of pictures having the suspected picture in the first continuous m pictures to be labeled and the second continuous m pictures to be labeled in the target picture sequence, wherein the first continuous m pictures to be labeled and the second continuous m pictures to be labeled have no intersection, wherein the first continuous m pictures to be labeled include the first picture, and the second continuous m pictures to be labeled include the second picture; S3, swapping the positions of the first picture and the second picture to obtain new first continuous m pictures to be labeled, new second continuous m pictures to be labeled, and a new target picture sequence, and respectively calculating the difference abs2 between the number of pictures having the suspected picture in the new first continuous m pictures to be labeled and the new second continuous m pictures to be labeled in the new target picture sequence; S4. If the abs1 is greater than the abs2, updating the target picture sequence according to the new target picture sequence.

19. The device according to claim 14 or 15, characterized in that The display module is also used to add masks to the pictures in the suspected picture set.

20. The device according to claim 14 or 15, characterized in that Also includes: The labeling module is used to accept the labeling of any picture of the K pictures to be labeled.

21. The device according to claim 20, characterized in that The display module is also used to display preset pleasant pictures after L times of labeling.

22. A computer-readable storage medium comprising a program, which, when executed on a computer, causes the computer to execute the method according to any one of claims 1 to 13.

23. An image data processing device, comprising a processor and a memory, wherein the processor is coupled to the memory, wherein: The memory is used to store programs; The processor is configured to execute the program in the memory so that the image data processing device performs the method according to any one of claims 1 to 13.