Image data processing method, device and medium

Abstract

The embodiments of the present application provide an image data processing method, device, and medium, which relate to the fields of computer and artificial intelligence technology. The method includes: obtaining an image to be processed, wherein the image to be processed includes at least one target object; obtaining an image segmentation model, wherein the image segmentation model includes a basic segmentation model and a segmentation refinement model, wherein the basic segmentation model is used to extract basic boundary information of the target object in the image to be processed, and the segmentation refinement model is used to correct the basic boundary information; the target object in the image to be processed is segmented by the image segmentation model to obtain target boundary information. The technical solution of the embodiments of the present application can improve the efficiency of image data processing.

Description

Technical Field

[0001] The present application relates to the field of computer and artificial intelligence technology, and more specifically, to an image data processing method, device and medium. Background Art

[0002] In image data processing scenarios, such as the segmentation of target objects in images, classic segmentation models (such as HRnet, improved RCNN, etc.) are usually used. However, for the segmentation of high-resolution images, classic segmentation models need to use high-resolution images for model training, and the model training process is slow.

[0003] Based on this, how to improve the efficiency of image data processing is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0004] The embodiments of the present application provide an image data processing method, apparatus, computer program product or computer program, and computer-readable medium, which can improve the efficiency of image data processing at least to a certain extent.

[0005] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0006] According to one aspect of an embodiment of the present application, there is provided an image data processing method, comprising: obtaining an image to be processed, the image to be processed comprising at least one target object; obtaining an image segmentation model, the image segmentation model comprising a basic segmentation model and a segmentation refinement model, the basic segmentation model being used to extract basic boundary information of the target object in the image to be processed, the segmentation refinement model being used to correct the basic boundary information; and segmenting the target object in the image to be processed by using the image segmentation model to obtain target boundary information.

[0007] According to one aspect of an embodiment of the present application, an image data processing device is provided, comprising: a first acquisition unit, used to acquire an image to be processed, wherein the image to be processed includes at least one target object; a second acquisition unit, used to acquire an image segmentation model, wherein the image segmentation model includes a basic segmentation model and a segmentation refinement model, wherein the basic segmentation model is used to extract basic boundary information of the target object in the image to be processed, and the segmentation refinement model is used to correct the basic boundary information; a segmentation unit, used to segment the target object in the image to be processed by using the image segmentation model to obtain target boundary information.

[0008] In some embodiments of the present application, based on the aforementioned scheme, the image segmentation model is pre-trained through the following steps: obtaining a basic segmentation model to be trained and a segmentation refinement model to be trained; a fourth acquisition unit is used to obtain a sample image, and obtain annotation information for the boundary of the target object in the sample image, and the annotation information is used to characterize the actual boundary information of each target object in the sample image; a training unit is used to perform model training based on the basic segmentation model to be trained and the segmentation refinement model to be trained, using the sample image and the annotation information to obtain the image segmentation model.

[0009] In some embodiments of the present application, based on the aforementioned solution, the image resolution of the sample image is lower than a predetermined resolution threshold.

[0010] In some embodiments of the present application, based on the aforementioned scheme, the training unit is configured as follows: training the basic segmentation model to be trained through the sample image and the annotation information to obtain the basic segmentation model; obtaining basic boundary information obtained after the basic segmentation model segments the target object in the sample image; training the segmentation refinement model to be trained through the sample image and the basic boundary information to obtain the segmentation refinement model; splicing the basic segmentation model and the segmentation refinement model to obtain the image segmentation model.

[0011] In some embodiments of the present application, based on the aforementioned scheme, the training unit is further configured to: based on the sample image, extract the sample basic boundary information of the target object in the sample image through the basic segmentation model to be trained; according to the error between the sample basic boundary information and the annotation information, reversely update the model parameters in the basic segmentation model to be trained through the loss function to obtain the basic segmentation model.

[0012] In some embodiments of the present application, based on the aforementioned scheme, the segmentation refinement model to be trained includes a multi-cascade network structure composed of multiple single network structures connected in a cascade manner, and the training unit is also configured to: input the sample image and the basic boundary information into the multi-cascade network structure; based on the sample image and the basic boundary information, trigger the multi-cascade network structure to adaptively learn the boundary information of the target object in the sample image to obtain the segmentation refinement model.

[0013] In some embodiments of the present application, based on the aforementioned scheme, the device also includes: a perturbation unit, which is used to perform a perturbation operation on the basic boundary information before inputting the sample image and the basic boundary information into the multi-cascade network structure to obtain at least one perturbed basic boundary information.

[0014] In some embodiments of the present application, based on the aforementioned scheme, the training unit is further configured to: based on the sample image and the basic boundary information, trigger the first single network structure to adaptively learn the boundary information of the target object in the sample image; based on the sample image, the basic boundary information, and the intermediate basic boundary information output by the previous single network structure, trigger each single network structure except the first single network structure to adaptively learn the boundary information of the target object in the sample image, wherein the information scale of the intermediate basic boundary information output by each single network structure in the multi-cascade network structure increases successively.

[0015] In some embodiments of the present application, based on the aforementioned scheme, the multi-cascade network structure includes a first multi-cascade network structure and a second multi-cascade network structure, and the training unit is also configured to: based on the sample image and the basic boundary information, trigger the first multi-cascade network structure to adaptively learn the boundary information of the target object in the sample image from the global perspective of the image, and output the global basic boundary information for the target object in the sample image; based on the sample image and the global basic boundary information, trigger the second multi-cascade network structure to adaptively learn the boundary information of the target object in the sample image from the local perspective of the image.

[0016] In some embodiments of the present application, based on the aforementioned scheme, the training unit is further configured to: in accordance with a predetermined cropping order, sequentially crop from the sample image to obtain image areas of predetermined cropping scales, and crop from the global basic boundary information to obtain local basic boundary information corresponding to the image area; based on the image area and the local basic boundary information, trigger the second multi-cascade network structure to adaptively learn the boundary information of the target object in the sample image.

[0017] In some embodiments of the present application, based on the aforementioned scheme, the training unit is configured to: embed the model structure of the segmentation refinement model to be trained into the image segmentation model to be trained to obtain the image segmentation model to be trained; train the image segmentation model to be trained through the sample image and the annotation information to obtain the image segmentation model.

[0018] In some embodiments of the present application, based on the aforementioned scheme, the image to be processed includes a high-resolution character image, the target object includes a target area of ​​the character in the high-resolution character image, and the resolution of the high-resolution character image is higher than or equal to a predetermined resolution threshold.

[0019] According to one aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the image data processing method described in the above embodiments.

[0020] According to one aspect of an embodiment of the present application, an image data processing device is also provided, characterized in that it includes a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, and the one or more programs include instructions for performing the image data processing method described in the above embodiments.

[0021] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which at least one program code is stored. The at least one program code is loaded and executed by a processor to implement the operations performed by the image data processing method described in the above embodiments.

[0022] In the technical solutions provided in some embodiments of the present application, since the image segmentation model includes a basic segmentation model for extracting basic boundary information of the target object in the image to be processed and a segmentation refinement model for correcting the basic boundary information, the basic segmentation model can be used to perform basic segmentation on images of any resolution including high-resolution images, and the segmentation refinement model can be used to correct the segmentation result output by the basic segmentation to obtain high-quality target boundary information for the image to be processed. Through the solution proposed in the present application, there is no need to train the model through a high-resolution image data set. The image segmentation model used for high-quality segmentation of high-resolution images can be trained only through a low-resolution image data set. Therefore, the efficiency of image data processing can be improved through the image segmentation model.

[0023] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0025] Figure 1 A schematic diagram showing an exemplary system architecture to which the technical solution of the embodiments of the present application can be applied;

[0026] Figure 2 A flowchart of an image data processing method according to an embodiment of the present application is shown;

[0027] Figure 3 A flowchart of a method before acquiring an image segmentation model according to an embodiment of the present application is shown;

[0028] Figure 4 A detailed flow chart of model training using the sample image and the annotation information according to an embodiment of the present application is shown;

[0029] Figure 5 A detailed flow chart showing training of the basic segmentation model to be trained by using the sample image and the annotation information according to an embodiment of the present application is shown;

[0030] Figure 6 A detailed flow chart of training the segmentation refinement model to be trained by using the sample image and the basic boundary information according to an embodiment of the present application is shown;

[0031] Figure 7 A detailed flow chart showing triggering the multi-cascade network structure to adaptively learn the boundary information of the target object in the sample image according to an embodiment of the present application is shown;

[0032] Figure 8 A schematic diagram of a framework for triggering the multi-cascade network structure to adaptively learn boundary information of a target object in the sample image according to an embodiment of the present application is shown;

[0033] Fig. 9 A detailed flow chart showing triggering the multi-cascade network structure to adaptively learn the boundary information of the target object in the sample image according to an embodiment of the present application is shown;

[0034] Fig.10 A detailed flow chart showing triggering the second multi-cascade network structure to adaptively learn the boundary information of the target object in the sample image from the local perspective of the image according to an embodiment of the present application is shown;

[0035] Fig.11 A schematic diagram of a framework for triggering the multi-cascade network structure to adaptively learn boundary information of a target object in the sample image according to an embodiment of the present application is shown;

[0036] Fig.12 A detailed flow chart of model training using the sample image and the annotation information according to an embodiment of the present application is shown;

[0037] Fig.13 A block diagram of an image data processing device according to an embodiment of the present application is shown;

[0038] Fig.14 A block diagram of an image data processing device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0039] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0040] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0041] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0042] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0043] It should be noted that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0044] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the objects used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those shown or described.

[0045] The embodiments in this application involve technologies related to artificial intelligence, that is, fully automated processing of data (such as image data) is achieved through artificial intelligence. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that the machines have the functions of perception, reasoning and decision-making.

[0046] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solution of the embodiments of the present application can be applied is shown.

[0047] like Figure 1 As shown, the system architecture may include terminal devices (such as Figure 1 The embodiment of the present invention is a network 104 and a server 105. The network 104 is a medium for providing a communication link between the terminal device and the server 105. The network 104 may include various connection types, such as a wired communication link, a wireless communication link, etc.

[0048] In one embodiment of the present application, when a user needs to segment at least one target object in an image to be processed, the user can send the image to be processed including at least one target object to the server 105 through a terminal device. After obtaining the image to be processed, the server 105 obtains an image segmentation model, and segments the target object in the image to be processed through the image segmentation model to obtain target boundary information, wherein the image segmentation model includes a basic segmentation model and a segmentation refinement model, the basic segmentation model is used to extract basic boundary information of the target object in the image to be processed, and the segmentation refinement model is used to correct the basic boundary information.

[0049] In this embodiment, since the image segmentation model includes a basic segmentation model for extracting basic boundary information of the target object in the image to be processed and a segmentation refinement model for correcting the basic boundary information, the basic segmentation model can be used to perform basic segmentation on images of any resolution including high-resolution images, and the segmentation result output by the basic segmentation model can be corrected to obtain high-quality target boundary information for the image to be processed. Through the solution proposed in the present application, there is no need to train the model through a high-resolution image data set. The image segmentation model used for high-quality segmentation of high-resolution images can be trained only through a low-resolution image data set. Therefore, the image segmentation model can improve the efficiency of image data processing.

[0050] It should be noted that the image data processing method provided in the embodiment of the present application can be executed by the server 105, and accordingly, the image data processing device is generally arranged in the server 105. However, in other embodiments of the present application, the terminal device can also have similar functions as the server, so as to execute the image data processing solution provided in the embodiment of the present application.

[0051] It should also be noted that Figure 1 The number of terminal devices, networks and servers in the description is only for illustration. According to the implementation requirements, the server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0052] It should be explained that cloud computing as described above is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space and information services as needed. The network that provides resources is called a "cloud". The resources in the "cloud" are infinitely expandable in the eyes of users, and can be obtained at any time, used on demand, and expanded at any time. By establishing a cloud computing resource pool (referred to as a cloud platform, generally referred to as an IaaS (Infrastructure as a Service) platform, various types of virtual resources are deployed in the resource pool for external customers to choose to use. The cloud computing resource pool mainly includes: computing devices (virtualized machines, including operating systems), storage devices, and network devices.

[0053] The implementation details of the technical solution of the embodiment of the present application are described in detail below:

[0054] Figure 2 A flowchart of an image data processing method according to an embodiment of the present application is shown. The image data processing method can be executed by a device having a computing and processing function, such as Figure 1 , or may be performed by the server 105 shown in Figure 1 Refer to the terminal device shown in Figure 2 As shown, the image data processing method at least includes steps 210 to 250, which are described in detail as follows:

[0055] In step 210, an image to be processed is acquired, wherein the image to be processed includes at least one target object.

[0056] In the present application, the image to be processed may include a high-resolution person image, and the target object may include various target areas of the person in the high-resolution person image.

[0057] It should be noted that the resolution of the high-resolution character image is higher than or equal to a predetermined resolution threshold. For example, a character image with a resolution higher than or equal to 1080p can be a high-resolution image.

[0058] It should also be noted that the image to be processed may be a low-resolution image, ie, an image with a resolution lower than a predetermined resolution threshold, in addition to a high-resolution image.

[0059] It should also be noted that the image to be processed may be other types of images besides human images, such as building images, animal images, etc., which are not specifically limited in this application.

[0060] In the present application, the image to be processed includes at least one target object. Taking a high-resolution character image as an example, the target object may include various target areas of the character in the character image. For example, the target area may include the character's head area, skin area, jacket area, T-shirt area, skirt area, pants area, dress area, tie area, etc., wherein there is a specific dividing line between each adjacent target area.

[0061] Continue to refer to Figure 2 In step 230, an image segmentation model is obtained, the image segmentation model includes a basic segmentation model and a segmentation refinement model, the basic segmentation model is used to extract basic boundary information of the target object in the image to be processed, and the segmentation refinement model is used to correct the basic boundary information.

[0062] In the present application, the image segmentation model can be obtained through preliminary model training. Before training the model to be trained, it is necessary to obtain the basic segmentation model to be trained and the segmentation refinement model to be trained. By performing model training on the basic segmentation model to be trained and the segmentation refinement model to be trained, an image segmentation model including the basic segmentation model and the segmentation refinement model can be obtained.

[0063] In one embodiment of the present application, before model training, at least one of the basic segmentation model to be trained and the segmentation refinement model to be trained can be compressed based on any one of a model pruning algorithm, a model distillation algorithm, and a model quantization algorithm.

[0064] In the present application, at least one of the basic segmentation model to be trained and the segmentation refinement model to be trained is compressed. The advantage is that the model volume of the basic segmentation model to be trained and the segmentation refinement model to be trained can be further reduced with little or no loss in model accuracy, thereby further accelerating the calculation speed and saving computer resources.

[0065] Continue to refer to Figure 2 In step 250, the target object in the image to be processed is segmented by the image segmentation model to obtain target boundary information.

[0066] In the present application, in the process of segmenting the target object in the image to be processed by the image segmentation model, the basic segmentation model in the image segmentation model can be used to perform basic segmentation on the image to be processed to obtain basic boundary information. Generally, the basic boundary information is inaccurate. Based on this, the present application further corrects the basic segmentation model through a segmentation refinement model to obtain target boundary information with higher accuracy.

[0067] It should be noted that the boundary information of the target object may actually be a segmentation map corresponding to the image.

[0068] In the present application, the proposed image data processing scheme is mainly implemented based on a pre-trained image segmentation model. In order to enable technical personnel in the field to more clearly understand the image segmentation model proposed in the present application, the training details of the image segmentation model mentioned in the present application will be introduced in detail below.

[0069] In one embodiment of the present application, Figure 2 Before step 230, that is, before obtaining the image segmentation model, the following steps may also be performed: Figure 3 The steps shown are as follows, Figure 3 The steps shown pre-train the image segmentation model.

[0070] See also Figure 3 , shows a flow chart of a method before acquiring an image segmentation model according to an embodiment of the present application. Specifically, it includes steps 220 to 240:

[0071] In step 220, a basic segmentation model to be trained and a segmentation refinement model to be trained are obtained.

[0072] In the present application, the basic segmentation model to be trained may be a PSPNet model (Pyramid SceneParsing Network), whose core module is a pyramid pooling module, which can aggregate contextual information of different regions, thereby improving the ability to obtain global information.

[0073] In the present application, the segmentation refinement model to be trained can be a CascadePSP cascade segmentation optimization model, which can improve and correct local boundaries when possible, and can optimize the boundary details of any given segmentation. Specifically, the input of the model is the original high-resolution image and its coarse segmentation result output on other networks, and the output of the model is a refined segmentation result.

[0074] Continue to refer to Figure 3 In step 240, a sample image is obtained, and annotation information for the boundaries of the target objects in the sample image is obtained, where the annotation information is used to characterize the actual boundary information of each target object in the sample image.

[0075] In some application fields of image segmentation, high-resolution images are usually required to be segmented. However, the model for segmenting high-resolution images generally requires high-resolution sample images for training. The general model trained with low-resolution images cannot perform high-quality image segmentation on high-resolution images. In most cases, there are two main problems with training models with high-resolution sample images. On the one hand, there are fewer data sets of high-resolution sample images, and the cost of constructing high-resolution sample images is high. On the other hand, due to the large amount of data of high-resolution images, the amount of calculation required to train the model with high-resolution sample images is large, resulting in low training efficiency.

[0076] Based on the above reasons, on the one hand, the sample image obtained by the present application is a low-resolution image, that is, the image resolution of the sample image is lower than a predetermined resolution threshold, and the annotation information for the boundary of the target object in the sample image is obtained to characterize the actual boundary information of each target object in the sample image. On the other hand, in addition to obtaining the basic segmentation model to be trained, the present application also obtains the segmentation refinement model to be trained, so that the basic segmentation model trained by the low-resolution sample image and the segmentation refinement model trained by the low-resolution sample image can be applied to the image segmentation of the high-resolution image.

[0077] Continue to refer to Figure 3 In step 260, based on the basic segmentation model to be trained and the segmentation refinement model to be trained, model training is performed through the sample image and the annotation information to obtain the image segmentation model.

[0078] In one embodiment of the present application, the data processing method proposed in the present application can be applied to the field of clothing segmentation. Currently, the industry mostly uses classic segmentation models (HRnet, improved RCNN, etc.) to solve the problem of high-resolution clothing segmentation. However, high-resolution problem models need to use high-resolution images for model training, and the model training process is slow.

[0079] In the field of clothing segmentation, common clothing segmentation datasets such as CIHP and LIP are low-resolution datasets and have many flaws in data annotation. Therefore, there are many problems in directly using public datasets to train general segmentation models. Manually annotating high-resolution clothing data is costly and time-consuming.

[0080] Since the main problem with high-resolution clothing segmentation is that there is no public and more accurate clothing segmentation dataset, and the model training accuracy is not high, the solution proposed by this application for the above problems can be: first obtain low-resolution sample images (for example, use public human body analysis datasets (LIP, CIHP) and clothing analysis datasets (CPP), and mix multiple datasets to improve the generalization ability and training accuracy of the model), and then combine the basic segmentation model (such as the PSPNet segmentation model) with the segmentation refinement model (such as the Cascade-PSP segmentation optimization model), and use a two-stage training method to enable the basic segmentation model to complete the basic segmentation of the image, and the segmentation refinement model corrects the basic segmentation results, avoiding the use of high-resolution data during the training process. Finally, according to the clothing segmentation characteristics, the output results of the final model are sorted, and the edges are more in line with the required segmentation effect.

[0081] In such Figure 3In one embodiment of step 260, based on the basic segmentation model to be trained and the segmentation refinement model to be trained, the model is trained by using the sample image and the annotation information to obtain the image segmentation model. Figure 4 Follow the steps shown.

[0082] See also Figure 4 , shows a detailed flow chart of model training using the sample image and the annotation information according to an embodiment of the present application. Specifically, it includes steps 261 to 264:

[0083] Step 261: Train the basic segmentation model to be trained using the sample image and the annotation information to obtain the basic segmentation model.

[0084] Step 262: Obtain basic boundary information obtained after the basic segmentation model segments the target object in the sample image.

[0085] Step 263: Train the segmentation refinement model to be trained using the sample image and the basic boundary information to obtain the segmentation refinement model.

[0086] Step 264, concatenating the basic segmentation model and the segmentation refinement model to obtain the image segmentation model.

[0087] In such Figure 4 In one embodiment of step 261, the basic segmentation model to be trained is trained by using the sample image and the annotation information to obtain the basic segmentation model, which can be obtained as follows: Figure 5 Follow the steps shown.

[0088] See also Figure 5 , shows a detailed flow chart of training the basic segmentation model to be trained by using the sample image and the annotation information according to an embodiment of the present application. Specifically, it includes steps 2611 to 2612:

[0089] Step 2611: Based on the sample image, extract sample basic boundary information of the target object in the sample image through the basic segmentation model to be trained.

[0090] Step 2612: According to the error between the sample basic boundary information and the annotation information, the model parameters in the basic segmentation model to be trained are reversely updated through a loss function to obtain the basic segmentation model.

[0091] In this embodiment, in the process of extracting the sample basic boundary information of the target object in the sample image through the basic segmentation model to be trained, the feature information in the sample image can be first extracted to obtain a sample feature map, and then the sample feature map is input into the basic segmentation model to be trained, and the basic segmentation model to be trained extracts the sample basic boundary information of the target object in the sample image through the sample feature map. Furthermore, since the annotation information represents the actual boundary information of each target object in the sample image, there will be an error between the sample basic boundary information and the annotation information. Based on this, the model parameters in the basic segmentation model to be trained can be reversely updated through the loss function according to the error to obtain the basic segmentation model.

[0092] In such Figure 4 In step 262 shown, basic boundary information can be obtained after the basic segmentation model segments the target object in the sample image. It should be noted that although the basic boundary information is extracted by a trained basic segmentation model, since the basic segmentation model cannot be trained to be absolutely accurate, the obtained basic boundary information is still not accurate.

[0093] In the present application, the segmentation refinement model to be trained may include a multi-cascade network structure composed of multiple single network structures connected in a cascade manner.

[0094] In such Figure 4 In one embodiment of step 263, the segmentation refinement model to be trained is trained by using the sample image and the basic boundary information to obtain the segmentation refinement model, which can be performed as follows: Figure 6 Follow the steps shown.

[0095] Reference Figure 6 , shows a detailed flow chart of training the segmentation refinement model to be trained by using the sample image and the basic boundary information according to an embodiment of the present application. Specifically, it includes steps 2631 to 2632:

[0096] Step 2631, input the sample image and the basic boundary information into the multi-cascade network structure.

[0097] Step 2632: Based on the sample image and the basic boundary information, trigger the multi-cascade network structure to adaptively learn the boundary information of the target object in the sample image to obtain the segmentation refinement model.

[0098] In such Figure 6In one embodiment of the steps shown, before step 2631, that is, before inputting the sample image and the basic boundary information into the multi-cascade network structure, a perturbation operation may be performed on the basic boundary information to obtain at least one perturbed basic boundary information.

[0099] In the present application, the perturbed basic boundary information may also be used as input to the multi-cascade network structure.

[0100] In the present application, the perturbed basic boundary information obtained by perturbing the basic boundary information output by the basic segmentation model can well train the adaptive optimization capability of the segmentation refinement model.

[0101] In such Figure 6 In one embodiment of step 2632, based on the sample image and the basic boundary information, the multi-cascade network structure is triggered to adaptively learn the boundary information of the target object in the sample image, which can be performed as follows: Figure 7 Follow the steps shown.

[0102] See also Figure 7 , shows a detailed flow chart of triggering the multi-cascade network structure to adaptively learn the boundary information of the target object in the sample image according to an embodiment of the present application. Specifically including steps 26321 to 26322:

[0103] Step 26321, based on the sample image and the basic boundary information, trigger the first single network structure to adaptively learn the boundary information of the target object in the sample image.

[0104] Step 26322, based on the sample image, the basic boundary information, and the intermediate basic boundary information output by the previous single network structure, trigger each single network structure except the first single network structure to adaptively learn the boundary information of the target object in the sample image, wherein the information scale of the intermediate basic boundary information output by each single network structure in the multi-cascade network structure increases successively.

[0105] In this application, different single network structures can perform refined segmentation of images and multiple basic boundary information from different scales, and multi-scale input allows the model to learn different levels of structure and boundary information in the image, thereby allowing the model to perform fine segmentation of images from different scales at the best level.

[0106] Specifically, the sample image and the basic boundary information are input into a first single network structure, which, while adaptively learning the boundary information of the target object in the image, also outputs one or more intermediate basic boundary information of different information scales.

[0107] Furthermore, the one or more intermediate basic boundary information of different information scales can be used together with the sample image and the basic boundary information as input information of the next single network structure cascaded adjacent to the first single network structure, so that the next single network structure can learn the boundary information of a larger information scale in the image and output the intermediate basic boundary information of the larger information scale, and iterate in this way until the last single network structure learns the boundary information in the image.

[0108] For example, if the information scale of the basic boundary information output by the basic segmentation model is 256×256, the first single network structure may output intermediate basic boundary information with an information scale of 32×32, the second single network structure may output intermediate basic boundary information with information scales of 32×32 and 64×64, and the third single network structure may output intermediate basic boundary information with information scales of 64×64 and 256×256.

[0109] In the present application, the information scale of the intermediate basic boundary information output by each single network structure in the multi-cascade network structure increases successively. The advantage is that the boundary information of the image can be gradually and finely learned from coarse scale to fine scale to improve the accuracy of the boundary information in the model-corrected image.

[0110] It should be noted that in the stage where each single network structure learns the boundary information of the target object in the image, due to the different learning scales of the boundary information of the target object in the image, different loss functions can be used to reversely update the parameters in each single network structure. By using a loss function suitable for each single network structure, the model can be better motivated to learn the boundary information of the target object in the image, thereby improving the performance of the model.

[0111] In order to make those skilled in the art better understand the above embodiments, Figure 8 , explained with a specific example.

[0112] See also Figure 8 , shows a schematic diagram of a framework for triggering the multi-cascade network structure to adaptively learn the boundary information of the target object in the sample image according to an embodiment of the present application.

[0113] like Figure 8As shown, a three-cascade network structure 800 is shown. First, the sample image, basic boundary information, and the perturbed basic boundary information are input into the first single network structure in the three-cascade network structure 800. The first single network structure adaptively learns the boundary information in the image and outputs 32×32 intermediate basic boundary information. Then, the sample image, basic boundary information, and the 32×32 intermediate basic boundary information output by the first single network structure are input into the second single network structure in the three-cascade network structure 800. The second single network structure adaptively learns the boundary information in the image and outputs 32×32 intermediate basic boundary information and 64×64 intermediate basic boundary information. Finally, the sample image, basic boundary information, and the 32×32 intermediate basic boundary information and 64×64 intermediate basic boundary information output by the second single network structure are input into the third single network structure in the three-cascade network structure 800. The third single network structure adaptively learns the boundary information in the image and outputs 64×64 intermediate basic boundary information and 256×256 intermediate basic boundary information.

[0114] It should be noted that, in the above-mentioned embodiment, the adaptive learning of the boundary information of the target object in the sample image by the multi-cascade network structure is explained from the internal structure and logic of the multi-cascade network structure. The following will explain the adaptive learning of the boundary information of the target object in the sample image by the multi-cascade network structure from the external logic of the multi-cascade network structure.

[0115] In itself, the number of the multi-cascade network structure can be multiple.

[0116] Specifically, in one embodiment of the present application, the multi-cascade network structure may include a first multi-cascade network structure and a second multi-cascade network structure.

[0117] In such Figure 6 In one embodiment of step 2632, based on the sample image and the basic boundary information, the multi-cascade network structure is triggered to adaptively learn the boundary information of the target object in the sample image, which can be performed as follows: Fig. 9 Follow the steps shown.

[0118] Reference Fig. 9 , shows a detailed flow chart of triggering the multi-cascade network structure to adaptively learn the boundary information of the target object in the sample image according to an embodiment of the present application. Specifically including steps 2633 to 2634:

[0119] Step 2633, based on the sample image and the basic boundary information, trigger the first multi-cascade network structure to adaptively learn the boundary information of the target object in the sample image from the global perspective of the image, and output the global basic boundary information for the target object in the sample image.

[0120] Step 2634, based on the sample image and the global basic boundary information, trigger the second multi-cascade network structure to adaptively learn the boundary information of the target object in the sample image from the local perspective of the image.

[0121] It can be understood that to adaptively learn the boundary information of the target object in the sample image from the global perspective of the image, it is necessary to input the entire sample image and the entire basic boundary information into the first multi-cascade network structure, and to adaptively learn the boundary information of the target object in the sample image from the local perspective of the image, it is necessary to input the local image in the sample image and the local information of the basic boundary information into the second multi-cascade network structure.

[0122] In such Fig. 9 In one embodiment of step 2634, based on the sample image and the global basic boundary information, the second multi-cascade network structure is triggered to adaptively learn the boundary information of the target object in the sample image from the local perspective of the image, which can be performed as follows: Fig.10 Follow the steps shown.

[0123] Reference Fig.10 , shows a detailed flowchart of triggering the second multi-cascade network structure to adaptively learn the boundary information of the target object in the sample image from the local perspective of the image according to an embodiment of the present application. Specifically including steps 26341 to 26342:

[0124] Step 26341, in accordance with a predetermined cropping order, cropping the sample image in sequence to obtain an image region of a predetermined cropping scale, and cropping the global basic boundary information to obtain local basic boundary information corresponding to the image region.

[0125] Step 26342, based on the image area and the local basic boundary information, trigger the second multi-cascade network structure to adaptively learn the boundary information of the target object in the sample image.

[0126] In order to make those skilled in the art better understand the above embodiments, Fig.11 , explained with a specific example.

[0127] See also Fig.11 , shows a schematic diagram of a framework for triggering the multi-cascade network structure to adaptively learn the boundary information of the target object in the sample image according to an embodiment of the present application.

[0128] like Fig.11 As shown,

[0129] like Fig.11As shown, a first cascade network structure 1101 and a second cascade network structure 1102 are shown. First, the sample image, basic boundary information, and perturbed basic boundary information are input into the first cascade network structure 1101. The first cascade network structure 1101 adaptively learns the boundary information in the image and outputs 64×64 intermediate basic boundary information and 256×256 intermediate basic boundary information. Then, the sample image, basic boundary information, and the 64×64 intermediate basic boundary information and 256×256 intermediate basic boundary information output by the first cascade network structure 1101 are input into the second cascade network structure 1102. The second cascade network structure 1102 adaptively learns the boundary information in the image and outputs 256×256 intermediate basic boundary information.

[0130] In the present application, high-resolution segmentation refinement can be performed by using a global step and a local step. Specifically, the global step considers the entire image to learn and correct the boundary information of the target object in the image, while the local step considers the local image to learn and correct the boundary information of the target object in the image, which can efficiently and accurately optimize the boundary information of the target object in the image.

[0131] In such Figure 3 In another embodiment of step 260, based on the basic segmentation model to be trained and the segmentation refinement model to be trained, the image segmentation model is obtained by model training through the sample image and the annotation information. Fig.12 Follow the steps shown.

[0132] See also Fig.12 , shows a detailed flow chart of model training using the sample image and the annotation information according to an embodiment of the present application. Specifically, it includes steps 265 to 266:

[0133] Step 265: embed the model structure of the segmentation refinement model to be trained into the image segmentation model to be trained to obtain the image segmentation model to be trained.

[0134] Step 266: Train the image segmentation model to be trained using the sample image and the annotation information to obtain the image segmentation model.

[0135] In this embodiment, by embedding the model structure of the segmentation refinement model to be trained into the image segmentation model to be trained, it is unnecessary to divide the model into two stages for training, and the memory space occupied by the image segmentation model can be greatly reduced, and the overall model training speed can be improved, shortening the training time.

[0136] In the present application, since the image segmentation model includes a basic segmentation model for extracting basic boundary information of the target object in the image to be processed and a segmentation refinement model for correcting the basic boundary information, the basic segmentation model can be used to perform basic segmentation on images of any resolution including high-resolution images, and the segmentation refinement model can be used to correct the segmentation result output by the basic segmentation to obtain high-quality target boundary information for the image to be processed. Through the solution proposed in the present application, there is no need to train the model through a high-resolution image data set. The image segmentation model used for high-quality segmentation of high-resolution images can be trained only through a low-resolution image data set. Therefore, the image segmentation model can improve the efficiency of image data processing.

[0137] The following describes an apparatus embodiment of the present application, which can be used to execute the image data processing method in the above-mentioned embodiment of the present application. For details not disclosed in the apparatus embodiment of the present application, please refer to the above-mentioned embodiment of the image data processing method of the present application.

[0138] Fig.13 A block diagram of an image data processing device according to an embodiment of the present application is shown.

[0139] Reference Fig.13 As shown, an image data processing device 1300 according to an embodiment of the present application includes: a first acquisition unit 1301 , a second acquisition unit 1302 and a segmentation unit 1303 .

[0140] Among them, the first acquisition unit 1301 is used to acquire the image to be processed, which includes at least one target object; the second acquisition unit 1302 is used to acquire the image segmentation model, which includes a basic segmentation model and a segmentation refinement model, the basic segmentation model is used to extract the basic boundary information of the target object in the image to be processed, and the segmentation refinement model is used to correct the basic boundary information; the segmentation unit 1303 is used to segment the target object in the image to be processed through the image segmentation model to obtain the target boundary information.

[0141] In some embodiments of the present application, based on the aforementioned scheme, the image segmentation model is pre-trained through the following steps: obtaining a basic segmentation model to be trained and a segmentation refinement model to be trained; a fourth acquisition unit is used to obtain a sample image, and obtain annotation information for the boundary of the target object in the sample image, and the annotation information is used to characterize the actual boundary information of each target object in the sample image; a training unit is used to perform model training based on the basic segmentation model to be trained and the segmentation refinement model to be trained, using the sample image and the annotation information to obtain the image segmentation model.

[0142] In some embodiments of the present application, based on the aforementioned solution, the image resolution of the sample image is lower than a predetermined resolution threshold.

[0143] In some embodiments of the present application, based on the aforementioned scheme, the training unit is configured as follows: training the basic segmentation model to be trained through the sample image and the annotation information to obtain the basic segmentation model; obtaining basic boundary information obtained after the basic segmentation model segments the target object in the sample image; training the segmentation refinement model to be trained through the sample image and the basic boundary information to obtain the segmentation refinement model; splicing the basic segmentation model and the segmentation refinement model to obtain the image segmentation model.

[0144] In some embodiments of the present application, based on the aforementioned scheme, the training unit is further configured to: based on the sample image, extract the sample basic boundary information of the target object in the sample image through the basic segmentation model to be trained; according to the error between the sample basic boundary information and the annotation information, reversely update the model parameters in the basic segmentation model to be trained through the loss function to obtain the basic segmentation model.

[0145] In some embodiments of the present application, based on the aforementioned scheme, the segmentation refinement model to be trained includes a multi-cascade network structure composed of multiple single network structures connected in a cascade manner, and the training unit is also configured to: input the sample image and the basic boundary information into the multi-cascade network structure; based on the sample image and the basic boundary information, trigger the multi-cascade network structure to adaptively learn the boundary information of the target object in the sample image to obtain the segmentation refinement model.

[0146] In some embodiments of the present application, based on the aforementioned scheme, the device also includes: a perturbation unit, which is used to perform a perturbation operation on the basic boundary information before inputting the sample image and the basic boundary information into the multi-cascade network structure to obtain at least one perturbed basic boundary information.

[0147] In some embodiments of the present application, based on the aforementioned scheme, the training unit is further configured to: based on the sample image and the basic boundary information, trigger the first single network structure to adaptively learn the boundary information of the target object in the sample image; based on the sample image, the basic boundary information, and the intermediate basic boundary information output by the previous single network structure, trigger each single network structure except the first single network structure to adaptively learn the boundary information of the target object in the sample image, wherein the information scale of the intermediate basic boundary information output by each single network structure in the multi-cascade network structure increases successively.

[0148] In some embodiments of the present application, based on the aforementioned scheme, the multi-cascade network structure includes a first multi-cascade network structure and a second multi-cascade network structure, and the training unit is also configured to: based on the sample image and the basic boundary information, trigger the first multi-cascade network structure to adaptively learn the boundary information of the target object in the sample image from the global perspective of the image, and output the global basic boundary information for the target object in the sample image; based on the sample image and the global basic boundary information, trigger the second multi-cascade network structure to adaptively learn the boundary information of the target object in the sample image from the local perspective of the image.

[0149] In some embodiments of the present application, based on the aforementioned scheme, the training unit is further configured to: in accordance with a predetermined cropping order, sequentially crop from the sample image to obtain image areas of predetermined cropping scales, and crop from the global basic boundary information to obtain local basic boundary information corresponding to the image area; based on the image area and the local basic boundary information, trigger the second multi-cascade network structure to adaptively learn the boundary information of the target object in the sample image.

[0150] In some embodiments of the present application, based on the aforementioned scheme, the training unit is configured to: embed the model structure of the segmentation refinement model to be trained into the image segmentation model to be trained to obtain the image segmentation model to be trained; train the image segmentation model to be trained through the sample image and the annotation information to obtain the image segmentation model.

[0151] In some embodiments of the present application, based on the aforementioned scheme, the image to be processed includes a high-resolution character image, the target object includes a target area of ​​the character in the high-resolution character image, and the resolution of the high-resolution character image is higher than or equal to a predetermined resolution threshold.

[0152] As another aspect, an embodiment of the present application also provides another image data processing device, including a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, and the one or more programs include instructions for performing the image data processing method described in the above embodiments.

[0153] Fig.14 A block diagram of an image data processing device according to an embodiment of the present application is shown. For example, the device 1400 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0154] Reference Fig.14, the device 1400 may include one or more of the following components: a processing component 1402 , a memory 1404 , a power component 1406 , a multimedia component 1408 , an audio component 1410 , an input / output (I / O) interface 1412 , a sensor component 1414 , and a communication component 1416 .

[0155] The processing component 1402 generally controls the overall operation of the device 1400, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1402 may include one or more processors 1418 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 1402 may include one or more modules to facilitate interaction between the processing component 1402 and other components. For example, the processing component 1402 may include a multimedia module to facilitate interaction between the multimedia component 1408 and the processing component 1402.

[0156] The memory 1404 is configured to store various types of data to support operations on the device 1400. Examples of such data include instructions for any application or method operating on the device 1400, contact data, phone book data, messages, pictures, videos, etc. The memory 1404 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0157] The power supply component 1406 provides power to the various components of the device 1400. The power supply component 1406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 1400.

[0158] The multimedia component 1408 includes a screen that provides an output interface between the device 1400 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1408 includes a front camera and / or a rear camera. When the device 1400 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0159] The audio component 1410 is configured to output and / or input audio signals. For example, the audio component 1410 includes a microphone (MIC), and when the device 1400 is in an operating mode, such as a call mode, a recording mode, and a voice information processing mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 1404 or sent via the communication component 1416. In some embodiments, the audio component 1410 also includes a speaker for outputting audio signals.

[0160] I / O interface 1412 provides an interface between processing component 1402 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0161] The sensor assembly 1414 includes one or more sensors for providing various aspects of the status assessment of the device 1400. For example, the sensor assembly 1414 can detect the open / closed state of the device 1400, the relative positioning of components, such as the display and keypad of the device 1400, the sensor assembly 1414 can also search for changes in the position of the device 1400 or a component of the device 1400, the presence or absence of user contact with the device 1400, the orientation or acceleration / deceleration of the device 1400, and the temperature change of the device 1400. The sensor assembly 1414 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 1414 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1414 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0162] The communication component 1416 is configured to facilitate wired or wireless communication between the device 1400 and other devices. The device 1400 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 1416 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1416 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency information processing (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0163] In an exemplary embodiment, the device 1400 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above methods.

[0164] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1404 including instructions, and the instructions can be executed by a processor 1418 of the device 1400 to perform the above-mentioned image data processing method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0165] As another aspect, the present application further provides a computer program product or a computer program, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the image data processing method described in the above embodiment.

[0166] As another aspect, the present application further provides a computer-readable storage medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The computer-readable storage medium stores at least one program code, which is loaded and executed by the processor of the device to implement the operations performed by the image data processing method described in the above embodiment.

[0167] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0168] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0169] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0170] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for processing image data, characterized in that: The method comprises: Acquire an image to be processed, wherein the image to be processed includes at least one target object and is a high-resolution image; Acquire an image segmentation model, the image segmentation model comprising a basic segmentation model and a segmentation refinement model, the basic segmentation model being used to extract basic boundary information of a target object in the image to be processed, and the segmentation refinement model being used to correct the basic boundary information; Segmenting the target object in the image to be processed using the image segmentation model to obtain target boundary information; The image segmentation model is pre-trained through the following steps: Obtaining a basic segmentation model to be trained and a segmentation refinement model to be trained, wherein the basic segmentation model to be trained includes a pyramid pooling module, and the segmentation refinement model to be trained includes a multi-cascade network structure composed of multiple single network structures connected in a cascade manner; Acquire a low-resolution sample image, and acquire annotation information for target object boundaries in the sample image, wherein the annotation information is used to represent actual boundary information of each target object in the sample image; Based on the basic segmentation model to be trained and the segmentation refinement model to be trained, model training is performed using the sample image and the annotation information to obtain the image segmentation model.

2. The method according to claim 1, characterized in that Based on the basic segmentation model to be trained and the segmentation refinement model to be trained, model training is performed using the sample image and the annotation information to obtain the image segmentation model, including: Training the to-be-trained basic segmentation model using the sample image and the annotation information to obtain the basic segmentation model; Acquiring basic boundary information obtained after segmenting the target object in the sample image by the basic segmentation model; Training the to-be-trained segmentation refinement model using the sample image and the basic boundary information to obtain the segmentation refinement model; The basic segmentation model and the segmentation refinement model are spliced ​​to obtain the image segmentation model.

3. The method according to claim 2, characterized in that The step of training the basic segmentation model to be trained by using the sample image and the annotation information to obtain the basic segmentation model includes: Based on the sample image, extracting sample basic boundary information of the target object in the sample image through the basic segmentation model to be trained; According to the error between the sample basic boundary information and the annotation information, the model parameters in the basic segmentation model to be trained are reversely updated through the loss function to obtain the basic segmentation model.

4. The method according to claim 2, characterized in that The segmentation refinement model to be trained includes a multi-cascade network structure composed of multiple single network structures connected in a cascade manner, and the segmentation refinement model to be trained is trained using the sample image and the basic boundary information to obtain the segmentation refinement model, including: Inputting the sample image and the basic boundary information into the multi-cascade network structure; Based on the sample image and the basic boundary information, the multi-cascade network structure is triggered to adaptively learn the boundary information of the target object in the sample image to obtain the segmentation refinement model.

5. The method according to claim 4, characterized in that Before inputting the sample image and the basic boundary information into the multi-cascade network structure, the method further includes: A perturbation operation is performed on the basic boundary information to obtain at least one perturbed basic boundary information.

6. The method according to claim 4, characterized in that The step of triggering the multi-cascade network structure to adaptively learn the boundary information of the target object in the sample image based on the sample image and the basic boundary information includes: Based on the sample image and the basic boundary information, triggering a first single network structure to adaptively learn boundary information of a target object in the sample image; Based on the sample image, the basic boundary information, and the intermediate basic boundary information output by the previous single network structure, each single network structure except the first single network structure is triggered to adaptively learn the boundary information of the target object in the sample image, wherein the information scale of the intermediate basic boundary information output by each single network structure in the multi-cascade network structure increases successively.

7. The method according to claim 4, characterized in that The multi-cascade network structure includes a first multi-cascade network structure and a second multi-cascade network structure, and triggering the multi-cascade network structure to adaptively learn the boundary information of the target object in the sample image based on the sample image and the basic boundary information includes: Based on the sample image and the basic boundary information, triggering the first multi-cascade network structure to adaptively learn the boundary information of the target object in the sample image from a global perspective of the image, and outputting the global basic boundary information for the target object in the sample image; Based on the sample image and the global basic boundary information, the second multi-cascade network structure is triggered to adaptively learn the boundary information of the target object in the sample image from a local perspective of the image.

8. The method according to claim 7, characterized in that The triggering, based on the sample image and the global basic boundary information, of the second multi-cascade network structure to adaptively learn the boundary information of the target object in the sample image from a local perspective of the image includes: In accordance with a predetermined cropping order, the sample image is sequentially cropped to obtain an image region of a predetermined cropping scale, and the global basic boundary information is cropped to obtain local basic boundary information corresponding to the image region; Based on the image area and the local basic boundary information, the second multi-cascade network structure is triggered to adaptively learn the boundary information of the target object in the sample image.

9. The method according to claim 1, characterized in that Based on the basic segmentation model to be trained and the segmentation refinement model to be trained, model training is performed using the sample image and the annotation information to obtain the image segmentation model, including: Embedding the model structure of the segmentation refinement model to be trained into the image segmentation model to be trained to obtain the image segmentation model to be trained; The image segmentation model to be trained is trained using the sample image and the annotation information to obtain the image segmentation model.

10. The method according to any one of claims 1 to 9, characterized in that The image to be processed includes a high-resolution person image, the target object includes a target area of ​​the person in the high-resolution person image, and the resolution of the high-resolution person image is higher than or equal to a predetermined resolution threshold.

11. An image data processing device, characterized in that: The device comprises: A first acquisition unit is used to acquire an image to be processed, wherein the image to be processed includes at least one target object and is a high-resolution image; a second acquisition unit, configured to acquire an image segmentation model, the image segmentation model comprising a basic segmentation model and a segmentation refinement model, the basic segmentation model being used to extract basic boundary information of a target object in the image to be processed, and the segmentation refinement model being used to correct the basic boundary information; a segmentation unit, configured to segment the target object in the image to be processed using the image segmentation model to obtain target boundary information; The image segmentation model is pre-trained through the following steps: Obtaining a basic segmentation model to be trained and a segmentation refinement model to be trained, wherein the basic segmentation model to be trained includes a pyramid pooling module, and the segmentation refinement model to be trained includes a multi-cascade network structure composed of multiple single network structures connected in a cascade manner; Acquire a sample image, and acquire annotation information for target object boundaries in the sample image, wherein the annotation information is used to represent actual boundary information of each target object in the sample image, wherein the sample image is a low-resolution image; Based on the basic segmentation model to be trained and the segmentation refinement model to be trained, model training is performed using the sample image and the annotation information to obtain the image segmentation model.

12. An image data processing device, characterized in that: The invention comprises a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, and the one or more programs include instructions for performing the image data processing method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores at least one program code, and the at least one program code is loaded and executed by a processor to implement the operations performed by the image data processing method according to any one of claims 1 to 10.

14. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium and are suitable for being read and executed by a processor, so that a computer device having the processor executes the image data processing method according to any one of claims 1 to 10.

Images