Self-adaptive image recognition and segmentation method, device and equipment

Abstract

The invention relates to the technical field of image processing, in particular to a self-adaptive image recognition and segmentation method, device and equipment, and the method comprises the steps: firstly constructing an image recognition and segmentation model based on CNN and Transform, then obtaining training image data, training the image recognition and segmentation model through the training image data, and finally obtaining a to-be-segmented image. And inputting a to-be-segmented image into the image identification and segmentation model to complete identification and segmentation of the to-be-segmented image. According to the method, the image recognition segmentation model is constructed through collaborative design of the CNN and the Transform, so that the segmentation precision and the segmentation efficiency are improved, and the labeling cost is reduced through a weak supervision training method.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly relates to an adaptive image recognition and segmentation method, device and equipment. Background Art

[0002] Image segmentation is a core task in computer vision and is widely applied in fields such as medical image analysis, autonomous driving, and industrial inspection. Existing mainstream methods are based on convolutional neural networks (CNNs) or Transformer architectures, but there are the following problems: CNNs have insufficient ability to model long-range dependencies, resulting in inaccurate segmentation edges; Transformers are less efficient in local detail processing and require a large amount of computing resources; existing models need to rely on fully supervised learning, and the annotation cost is high. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide an adaptive image recognition and segmentation method, device and equipment to overcome the problems existing in the current prior art.

[0004] To achieve the above purpose, the present invention adopts the following technical solutions: In the first aspect, the present application provides an adaptive image recognition and segmentation method, including: Constructing an image recognition and segmentation model based on CNN and Transformer; Obtaining training image data and training the image recognition and segmentation model with the training image data; Obtaining an image to be segmented; Inputting the image to be segmented into the image recognition and segmentation model to complete the recognition and segmentation of the image to be segmented.

[0005] Further, in the above method, the step of obtaining training image data and training the image recognition and segmentation model with the training image data includes: Obtaining training images; Constructing a training data set by randomly occluding, perturbing colors, and generating synthetic data with GAN; Performing weakly supervised learning training on the image recognition and segmentation model by optimizing features according to the training data set using self-supervised contrast learning.

[0006] Further, in the above method, the image recognition and segmentation model includes: an adaptive parameter module; The adaptive parameter module is used to dynamically adjust the number of convolutional kernels, attention weights, and network depth according to the complexity of the image input into the image recognition and segmentation model.

[0007] Further, in the above method, the image recognition and segmentation model adopts depthwise separable convolution and channel pruning techniques.

[0008] In a second aspect, the present application provides an adaptive image recognition and segmentation device, including: A model creation module, configured to build an image recognition and segmentation model based on CNN and Transformer; A model training module, configured to obtain training image data and train the image recognition and segmentation model with the training image data; An image acquisition module, configured to acquire an image to be segmented; An image recognition and segmentation module, configured to input the image to be segmented into the image recognition and segmentation model to complete the recognition and segmentation of the image to be segmented.

[0009] In a third aspect, the present application provides an adaptive image recognition and segmentation device, including a processor and a memory, where the processor is connected to the memory: Wherein, the processor is configured to call and execute a program stored in the memory; The memory is configured to store the program, and the program is at least used to execute the adaptive image recognition and segmentation method described in any one of the above.

[0010] The beneficial effects of the present invention are: The present application first builds an image recognition and segmentation model based on CNN and Transformer, then obtains training image data, and trains the image recognition and segmentation model with the training image data. Finally, an image to be segmented is obtained, and the image to be segmented is input into the image recognition and segmentation model to complete the recognition and segmentation of the image to be segmented. In the present application, an image recognition and segmentation model is constructed through the collaborative design of CNN and Transformer, thereby improving the segmentation accuracy and segmentation efficiency, and reducing the annotation cost through a weakly supervised training method. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0012] Figure 1 is a flowchart provided by an embodiment of an adaptive image recognition and segmentation method of the present invention; Figure 2It is a flowchart provided by another embodiment of an adaptive image recognition and segmentation method of the present invention; Figure 3 It is a schematic structural diagram provided by an embodiment of an adaptive image recognition and segmentation device of the present invention; Figure 4 It is a schematic structural diagram provided by an embodiment of an adaptive image recognition and segmentation device of the present invention. Specific embodiments

[0013] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be described in detail below. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope protected by the present invention.

[0014] Figure 1 It is a flowchart provided by an embodiment of an adaptive image recognition and segmentation method of the present invention. Please refer to Figure 1 , this embodiment may include the following steps: S1. Construct an image recognition and segmentation model based on CNN and Transformer; S2. Obtain training image data and train the image recognition and segmentation model with the training image data; S3. Obtain the image to be segmented; S4. Input the image to be segmented into the image recognition and segmentation model to complete the recognition and segmentation of the image to be segmented.

[0015] It can be understood that in this application, an image recognition and segmentation model is first constructed based on CNN and Transformer, then training image data is obtained, and the image recognition and segmentation model is trained with the training image data. Finally, the image to be segmented is obtained, and the image to be segmented is input into the image recognition and segmentation model to complete the recognition and segmentation of the image to be segmented. In this application, an image recognition and segmentation model is constructed through the collaborative design of CNN and Transformer, thereby improving the segmentation accuracy and efficiency, reducing the annotation cost through a weakly supervised training method, combining the local feature extraction ability of CNN and the global context modeling of Transformer, and constructing a multi-scale feature fusion module, that is, the image recognition and segmentation model, which can accurately segment the image and avoid the problem of low efficiency in local detail processing of Transformer.

[0016] Preferably, step S2 includes: Obtain training images; Construct a training dataset by randomly occluding, perturbing colors, and generating synthetic data using GANs. Use self-supervised contrastive learning to optimize features based on the training dataset and perform weakly supervised learning training on the image recognition and segmentation model.

[0017] It can be understood that weakly supervised learning is a technique for constructing a model based on newly generated data. It is a branch of machine learning that uses noisy, limited, or inaccurate sources to label a large amount of training data. Weak supervision encompasses a wide range of methods in which the model is trained using partial, imprecise, or otherwise inaccurate information that is easier to provide than manually labeled data. In this application, an image is first obtained, and then synthetic data is generated by random occlusion, color perturbation, and GANs to construct a training dataset, enhancing the diversity of the training dataset; then, partial annotation data is performed on the images in the training dataset, and self-supervised contrastive learning is used to optimize the feature representation, and weakly supervised learning training is performed on the image recognition and segmentation model.

[0018] Preferably, the image recognition and segmentation model includes: an adaptive parameter module; The adaptive parameter module is used to dynamically adjust the number of convolutional kernels, attention weights, and network depth according to the complexity of the image input to the image recognition and segmentation model.

[0019] Preferably, the image recognition and segmentation model employs depthwise separable convolution and channel pruning techniques.

[0020] It can be understood that, as Figure 2 shown, through the weakly supervised learning training of the model, the adaptive parameter module of the image recognition and segmentation model can dynamically adjust the number of convolutional kernels, attention weights, and network depth according to the complexity of the input image. After the input image is feature-extracted, the adaptive module generates an optimal parameter configuration and outputs a segmentation result.

[0021] The present invention also provides an adaptive image recognition and segmentation device for implementing the above method embodiments. Figure 3 It is a schematic structural diagram provided by an embodiment of an adaptive image recognition and segmentation device of the present invention. As Figure 3 shown, this embodiment includes: A model creation module 1 for constructing an image recognition and segmentation model based on CNN and Transformer; A model training module 2 for obtaining training image data and training the image recognition and segmentation model with the training image data; An image acquisition module 3 for acquiring the image to be segmented; An image recognition and segmentation module 4 for inputting the image to be segmented into the image recognition and segmentation model to complete the recognition and segmentation of the image to be segmented.

[0022] Regarding the device in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be elaborated here.

[0023] The present invention also provides an adaptive image recognition and segmentation device for implementing the above method embodiments. Figure 4 It is a schematic structural diagram provided by an embodiment of an adaptive image recognition and segmentation device of the present invention. As Figure 4 shown, an adaptive image recognition and segmentation device in this embodiment includes a processor 21 and a memory 22, and the processor 21 is connected to the memory 22. Among them, the processor 21 is used to call and execute the program stored in the memory 22; the memory 22 is used to store the program, and the program is at least used to execute an adaptive image recognition and segmentation method in the above embodiments.

[0024] The specific implementation of the adaptive image recognition and segmentation device provided by the embodiments of the present application can refer to the implementation manner of the adaptive image recognition and segmentation method in any of the above embodiments, and will not be elaborated here.

[0025] It can be understood that the same or similar parts in the above embodiments can be referred to each other, and the content not detailed in some embodiments can be seen in the same or similar content in other embodiments.

[0026] It should be noted that in the description of the present invention, terms such as "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, the meaning of "plurality" refers to at least two.

[0027] Any process or method description in the flowchart or described in other ways herein can be understood as representing a module, segment, or part of code including one or more executable instructions for implementing a specific logical function or process, and the scope of the preferred embodiments of the present invention includes additional implementations, where the functions can be executed in a substantially simultaneous manner or in a reverse order according to the involved functions, rather than in the order shown or discussed, which should be understood by those skilled in the technical field to which the embodiments of the present invention belong.

[0028] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits with logic gate circuits for implementing logical functions on data signals, application specific integrated circuits with appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0029] Those of ordinary skill in the art can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0030] In addition, in each embodiment of the present invention, the functional units can be integrated into a processing module, or each unit can exist physically alone, or two or more units can be integrated into one module. The above integrated module can be implemented in the form of hardware or in the form of a software functional module. When the above integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0031] The above-mentioned storage medium can be a read-only memory, a magnetic disk, an optical disk, etc.

[0032] In the description of this specification, the descriptions with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0033] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. An adaptive image recognition and segmentation method, characterized in that, Comprising: Constructing an image recognition and segmentation model based on CNN and Transformer; Obtaining training image data and training the image recognition and segmentation model with the training image data; Obtaining the image to be segmented; Inputting the image to be segmented into the image recognition and segmentation model to complete the recognition and segmentation of the image to be segmented.

2. The method according to claim 1, wherein The obtaining training image data and training the image recognition and segmentation model with the training image data includes: Obtaining training images; Constructing a training dataset by randomly occluding, perturbing colors, and generating synthetic data with GAN; Performing weakly supervised learning training on the image recognition and segmentation model by optimizing features using self-supervised contrast learning according to the training dataset.

3. The method according to claim 2, wherein The image recognition and segmentation model includes: an adaptive parameter module; The adaptive parameter module is used to dynamically adjust the number of convolutional kernels, attention weights, and network depth according to the complexity of the image input into the image recognition and segmentation model.

4. The method according to claim 3, characterized in that, The image recognition and segmentation model adopts depthwise separable convolution and channel pruning techniques.

5. An adaptive image recognition and segmentation device, characterized in that, Comprising: A model creation module for constructing an image recognition and segmentation model based on CNN and Transformer; A model training module for obtaining training image data and training the image recognition and segmentation model with the training image data; An image acquisition module for obtaining the image to be segmented; An image recognition and segmentation module for inputting the image to be segmented into the image recognition and segmentation model to complete the recognition and segmentation of the image to be segmented.

6. An adaptive image recognition and segmentation device, characterized in that, Comprising a processor and a memory, the processor is connected to the memory: Wherein, the processor is used to call and execute the program stored in the memory; The memory is used to store the program, and the program is at least used to execute the adaptive image recognition and segmentation method according to any one of claims 1-4.

Images