Image recognition method and device, equipment and medium

Abstract

The invention relates to an image recognition method and device, equipment and a medium. The method comprises the following steps: acquiring a to-be-detected image of a target object and a plurality of sample images of the object; the target object is the same as the object; performing fine tuning on the target detection model by adopting a plurality of sample images to obtain a target detection fine tuning model; performing image segmentation on the to-be-detected image to obtain a plurality of to-be-detected image blocks; performing target detection on each to-be-detected image block by adopting a target detection fine tuning model to obtain a target detection frame of the to-be-detected image block; and performing semantic segmentation on the target detection frame of each to-be-detected image block, and generating an identification result of the to-be-detected image of the target object according to a segmentation result. By adopting the method and the device, the high-resolution image can be effectively identified, and the small target detection accuracy is high.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image recognition method, device, equipment and medium. Background Art

[0002] With the development of artificial intelligence technology in the field of computer vision, the detection and recognition of objects in images have received increasing attention. Technologies related to the detection and recognition of objects in images include object detection and semantic segmentation. Among them, object detection aims to identify specific objects in images or videos, determine their locations and output object detection frames; semantic segmentation aims to classify and label pixels in images or video frames.

[0003] In the field of image detection, small target objects have always been a difficulty in target detection and semantic segmentation tasks because they have low resolution, blurred images, and carry little information, which is not conducive to target detection.

[0004] At present, the method of detecting small targets in images usually involves first training a small target detection network and a small target semantic segmentation model with a database. In the test phase, the entire input test image is detected and segmented based on the small target detection network and the small target semantic segmentation model. However, when performing semantic segmentation processing on small target objects such as tiny components, small targets are identified in the entire input image. Since small targets are small, they are prone to recognition errors, and the recognition accuracy of small target objects is low. Moreover, for high-resolution images, such as in the field of wafer inspection, the images to be processed are all high-resolution images. Since high-resolution images are generally large, it is difficult to accurately read the data of the original image when performing small target detection, making detection difficult. Summary of the invention

[0005] Based on this, it is necessary to provide an image recognition method, device, equipment and medium that can improve the accuracy of small target detection in response to the above technical problems.

[0006] An image recognition method, comprising:

[0007] Acquire an image of a target object to be tested and a plurality of sample images of the object; the target object is of the same category as the object;

[0008] Fine-tune the target detection model using a plurality of the sample images to obtain a target detection fine-tuning model;

[0009] Performing image segmentation on the image to be tested to obtain a plurality of image blocks to be tested;

[0010] Using the target detection fine-tuning model to perform target detection on each image block to be tested, and obtain a target detection frame of the image block to be tested;

[0011] The target detection frame of each image block to be tested is semantically segmented, and a recognition result of the image to be tested of the target object is generated according to the segmentation result.

[0012] In one embodiment, the fine-tuning of the target detection model using a plurality of the sample images to obtain the target detection fine-tuning model includes:

[0013] Performing image segmentation on the sample image to obtain a plurality of sample image blocks;

[0014] The object detection model is fine-tuned based on the sample image block and the sample image to obtain the object detection fine-tuning model.

[0015] In one embodiment, segmenting the sample image to obtain a plurality of sample image blocks includes:

[0016] Segmenting the sample image with overlap to obtain a plurality of local sample image blocks;

[0017] The size of each local sample image block is adjusted to a preset size to obtain a sample image block after image enhancement.

[0018] In one embodiment, the target detection model is a YOLOX model.

[0019] In one embodiment, segmenting the image to be tested to obtain a plurality of image blocks to be tested includes:

[0020] Segmenting the image to be tested with overlap to obtain multiple local image blocks to be tested;

[0021] The size of each local image block to be tested is adjusted to a preset size to obtain the image block to be tested after image enhancement.

[0022] In one embodiment, the semantic segmentation of the target detection frame of each image block to be tested and the generation of the recognition result of the image to be tested of the target object according to the segmentation result include:

[0023] The images corresponding to the target detection frames of the image blocks to be tested are respectively extracted and enlarged by a preset multiple to obtain the target images of the image blocks to be tested;

[0024] Selecting a preset number of points in the target image and inputting them into a semantic segmentation model to obtain a segmentation mask;

[0025] Based on the segmentation mask, the semantic segmentation model is used to perform semantic segmentation on the target image to obtain a segmentation result of the target image;

[0026] The segmentation results of each image block to be tested corresponding to the target image are integrated into the image to be tested of the target object to obtain a recognition result of the image to be tested of the target object.

[0027] An image recognition device, comprising:

[0028] An image acquisition module, used to acquire an image of a target object to be tested and a plurality of sample images of the object; the target object is of the same category as the object;

[0029] A model fine-tuning module, used to fine-tune the target detection model using a plurality of the sample images to obtain a target detection fine-tuning model;

[0030] An image segmentation module is used to segment the image to be tested to obtain a plurality of image blocks to be tested;

[0031] A target detection module, used to perform target detection on each image block to be tested by using the target detection fine-tuning model to obtain a target detection frame of the image block to be tested;

[0032] The semantic segmentation module is used to perform semantic segmentation on the target detection frame of each image block to be tested, and generate a recognition result of the image to be tested of the target object according to the segmentation result.

[0033] In one embodiment, the image segmentation module is further used to:

[0034] Segmenting the image to be tested with overlap to obtain multiple local image blocks to be tested;

[0035] The size of each local image block to be tested is adjusted to a preset size to obtain the image block to be tested after image enhancement.

[0036] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned image recognition method when executing the computer program.

[0037] A computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the above-mentioned image recognition method when executed by a processor.

[0038] The above-mentioned image recognition method, device, equipment and medium introduce image segmentation, first divide the large image to be tested into small image blocks to be tested, for the small image blocks to be tested, use the target detection fine-tuning model obtained by fine-tuning the target detection model through multiple sample images to perform target detection, and then perform semantic segmentation on the target detection frame of the small image blocks to be tested, and generate the recognition result of the image to be tested of the target object according to the segmentation result. In this way, the image to be tested is divided into multiple small image blocks to be tested through image block processing, which is convenient for the detection input of the high-resolution large image and can effectively recognize the high-resolution image; and, by using the two-stage method of target detection and semantic segmentation in series, first perform target detection on the small image blocks to be tested after the image is divided, complete the target detection of the small image, that is, complete the small target detection, and then perform semantic segmentation based on the small target detection, and realize the detection and recognition of the small target in the whole image to be tested by detecting and identifying the target in the small image blocks to be tested. Since the proportion of the small target in the image to be tested in the image blocks to be tested after the image is divided becomes larger, the recognition is more accurate, so the recognition accuracy of the small target can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the conventional technology, the drawings required for use in the embodiments or the conventional technology descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0040] Figure 1 is a flowchart of an image recognition method in one embodiment;

[0041] Figure 2 A schematic diagram of a fine-tuning dataset and an enhanced fine-tuning dataset in one embodiment;

[0042] Figure 3 A schematic block diagram of an embodiment of fine-tuning an object detection model using a sample image and a sample image block after image enhancement to obtain an object detection fine-tuning model;

[0043] Figure 4 A flowchart of applying an image recognition method to perform target detection and semantic segmentation in one embodiment;

[0044] Figure 5 FIG. 4 is a structural block diagram of an image recognition device in an embodiment. DETAILED DESCRIPTION

[0045] In order to facilitate understanding of the present application, the present application will be described more fully below with reference to the relevant drawings. Embodiments of the present application are provided in the drawings. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0047] In one embodiment, an image recognition method is provided, which can be applied to a computer device, such as a computer, a notebook, a smart terminal, etc. Figure 1 The method includes steps S110 to S190.

[0048] S110: Acquire an image of a target object to be tested and a plurality of sample images of the object.

[0049] The target object is of the same category as the object. The image to be tested and the sample image contain small targets. For example, in the case of wafer detection, the target object is the wafer whose parts need to be identified, and the object is also a wafer; the image to be tested and the sample image of the wafer can be 2000pixel*2000pixel, which contains small targets of less than (10pixel*10pixel), such as wafer device bumps, pads, dies, etc.

[0050] Specifically, the image acquisition device may acquire an image of the target object to obtain the image to be tested, and the image acquisition device may acquire multiple images of the object to obtain a sample image; and the computer device may receive the image to be tested and the sample image sent by the image acquisition device.

[0051] S130: fine-tune the target detection model using multiple sample images to obtain a target detection fine-tuning model.

[0052] The target detection model is a model for target detection based on images. The target detection model is fine-tuned using sample images so that the fine-tuned target detection fine-tuned model is suitable for target detection tasks on sample image blocks.

[0053] S150: Segment the image to be tested to obtain a plurality of image blocks to be tested.

[0054] For target detection on high-resolution images to be tested, it is difficult to read data from the original images to be tested due to the high resolution. For example, in the field of wafer detection, wafer images are basically high-resolution images, and the size basically starts from 2000pixel*2000pixel. By segmenting the image to be tested, image blocks to be tested that are smaller than the image to be tested are obtained, so that the high-resolution image to be tested can be minimized, and the proportion of small targets in the image to be tested in the image blocks to be tested is greater than the proportion of small targets in the image to be tested, which facilitates the detection of small targets. Specifically, image segmentation can be performed based on the number of image blocks to be tested that need to be segmented, or based on the size of the image blocks to be tested that need to be obtained. For example, the image to be tested is segmented into M and N image blocks in the horizontal and vertical directions respectively, and M and N can be predefined as hyperparameters in [M min ,M max ]、[N min , N max ].

[0055] S170: Using the target detection fine-tuning model to perform target detection on each image block to be tested, and obtain a target detection frame of the image block to be tested.

[0056] By performing target detection on each image block to be tested after the image is segmented, a target detection frame corresponding to the image block to be tested is obtained, thereby completing target detection of the small image.

[0057] S190: Perform semantic segmentation on the target detection frame of each image block to be tested, and generate a recognition result of the image to be tested of the target object according to the segmentation result.

[0058] Specifically, an existing semantic segmentation model may be used to perform semantic segmentation on the target detection frame of the image block to be tested, to obtain a segmentation result of the image block to be tested, and then the segmentation results of each image block to be tested are integrated to generate a recognition result of the overall image to be tested.

[0059] The above-mentioned image recognition method introduces image segmentation, firstly divides the large image to be tested into small image blocks to be tested, and for the small image blocks to be tested, uses the target detection fine-tuning model obtained by fine-tuning the target detection model through multiple sample images to perform target detection, and then performs semantic segmentation on the target detection frame of the small image blocks to be tested, and generates the recognition result of the image to be tested of the target object according to the segmentation result. In this way, the image to be tested is divided into multiple small image blocks to be tested through image segmentation processing, which facilitates the detection input of the high-resolution large image and can effectively recognize the high-resolution image; and, by using the two-stage method of target detection and semantic segmentation in series, firstly performs target detection on the small image blocks to be tested after the image is segmented, completes the target detection of the small image, that is, completes the small target detection, and then performs semantic segmentation based on the small target detection, and realizes the detection and recognition of the small target in the whole image to be tested by detecting and identifying the target in the small image blocks to be tested. Since the proportion of the small target in the image to be tested in the image blocks to be tested after the image is segmented increases, the recognition is more accurate, and therefore, the recognition accuracy of the small target can be improved.

[0060] In one embodiment, step S130 includes step (a1) to step (a2).

[0061] Step (a1): Segment the sample image to obtain multiple sample image blocks.

[0062] The sample image and the image to be tested are generally the same size. For a high-resolution sample image, the sample image is segmented to obtain a sample image block smaller than the sample image. Specifically, the size of the sample image block can be equal to the size of the image block to be tested.

[0063] Step (a2): fine-tune the target detection model based on the sample image blocks and the sample images to obtain a target detection fine-tuning model.

[0064] Specifically, the target detection model can be fine-tuned using an existing model fine-tuning training method. For example, in the process of fine-tuning the target detection model, an SGD optimizer can be used, with a learning rate of 0.01, a momentum of 0.9, and a weight decay of 0.0001.

[0065] By dividing the sample image into blocks, the sample image blocks and the original sample image are input into the target detection model for fine-tuning, so that the fine-tuned target detection fine-tuning model can adapt to the target detection task of the target object image whose image size is the size of the sample image block, so as to facilitate the target detection of the image block to be tested of the target object. For example, the target object image can be a wafer image.

[0066] In one embodiment, step (a1) includes: dividing the sample image with overlap to obtain multiple local sample image blocks; adjusting the size of each local sample image block to a preset size to obtain an image enhanced sample image block.

[0067] The sample image is segmented with overlap, that is, the sample image is segmented into overlapping image blocks to obtain overlapping local sample image blocks, such as local sample image blocks adjacent in the horizontal direction overlap in the horizontal direction, and local sample image blocks adjacent in the vertical direction overlap in the vertical direction. The size of each local sample image block is adjusted, such as by enlarging the local sample image block to a preset size, thereby performing image enhancement on the local sample image block to obtain a final enhanced sample image block.

[0068] After the high-resolution original image is segmented, the size of the small target in the original image will become larger. After overlapping and segmenting the sample image to obtain a local sample image block, the local sample image block is adjusted to a preset size, and the small target in the sample image block is enlarged through image enhancement, which is convenient for target detection.

[0069] For example, five thousand 2000 pixel * 2000 pixel high-definition wafer images are taken as sample images to form a fine-tuning data set. There are small targets of less than 10 pixel * 10 pixel in the high-definition wafer images. If conventional image segmentation is performed, the small targets will become smaller after processing, and the model used for target detection cannot learn effective features. In this embodiment, reference Figure 2 , each original high-definition wafer image I1 in the fine-tuning dataset F 、I2 F ,…,I j F Divide into overlapping image blocks P1 F 、P2 F ,…P k F , the original image is divided into M and N image blocks horizontally and vertically respectively; then, all the image blocks are resized to a preset size, such as 500-1000 pixels, at a certain ratio, to obtain a high-definition wafer image I1 F The corresponding final enhanced image blocks I1', I2', ..., I k ', at this time, the corresponding object in the enhanced image block is also larger than that in the original image; the enhanced image blocks obtained after segmentation and enlargement of all the original high-definition wafer images in the fine-tuning dataset and the original images constitute the enhanced fine-tuning dataset. Further, refer to Figure 3 In step (a2), the original sample image and the sample image block after image enhancement are used to fine-tune the target detection model to obtain a target detection fine-tuning model.

[0070] In one embodiment, the object detection model is a YOLOX model. Specifically, the YOLOX model is fine-tuned based on the sample image blocks and the sample images to obtain the object detection fine-tuning model.

[0071] The baseline model used by the YOLOX model is YOLOv3 and is improved on this basis. The backbone of YOLOv3 is the Darknet53 framework. YOLOv3 only has convolutional layers. The size of the feature map is controlled by adjusting the convolution step size, so there is no limit on the size of the input image. YOLOv3 borrows the idea of ​​pyramid feature maps. Small-size feature maps are used to detect large-size problems, while large-size feature maps detect small-size objects. YOLOv3 uses exponential sliding average, cosine annealing to adjust the learning rate, adds an IoU-aware branch, and uses mosaic, RandomHorizontalFlip, ColorJitter and multi-scale for data enhancement.

[0072] The YOLOX model has the following improvements compared to YOLOv3:

[0073] 1. Decoupled head

[0074] In YOLOv3, for the detection task of M, each anchor will correspond to the prediction result of H*W*(M+5), where obj occupies 1 channel, reg occupies 4 channels, and cls occupies M channels. YOLOX uses 1*1 convolution to unify the feature maps of different channel numbers to 256 (the main purpose is to reduce the dimension), and then uses two parallel branches, each of which uses 3*3 convolution. At the same time, the IoU branch is also added to the regression branch.

[0075] Decoupling the prediction branch greatly improves the convergence speed. Compared with the non-decoupled end-to-end approach, decoupling can bring a 4.2% AP improvement.

[0076] 2. Powerful data enhancement

[0077] The YOLOX model adds mosaic and Mixup, but turns them off after 15 epochs to prevent excessive data enhancement.

[0078] 3. Anchor-free architecture

[0079] When using anchors, in order to tune the model, it is necessary to cluster and analyze the data to determine the optimal anchor point, which lacks generalization. The anchor mechanism of the YOLOX model increases the complexity of the detection head and the number of predictions for each image. Using anchor-free can reduce the amount of adjustment parameters and the required usage skills.

[0080] Compared with YOLOv2, YOLOv3, etc., the YOLOX model has significantly improved the speed and accuracy of target detection. In addition, the YOLOX model fine-tuning uses sample image blocks and original sample images, and combines the results of full-size prediction and image block prediction, taking into account both large and small targets, which can improve the overall target detection performance.

[0081] In one embodiment, step S150 includes step (b1) and step (b2).

[0082] Step (b1): Segment the image to be tested with overlap to obtain multiple local image blocks to be tested.

[0083] The image to be tested is segmented with overlap, that is, the image to be tested is segmented into overlapping image blocks to obtain overlapping local image blocks to be tested.

[0084] Step (b2): adjusting the size of each local image block to be tested to a preset size to obtain the image block to be tested after image enhancement.

[0085] The size of each local image block to be tested is adjusted, for example, the local image block to be tested is enlarged to a preset size, thereby performing image enhancement on the local image block to be tested to obtain the final enhanced image block to be tested. The specific implementation of step (b1) and step (b2) can refer to step (a1).

[0086] After overlapping and segmenting the image to be tested to obtain a local image block to be tested, the local image block to be tested is adjusted to a preset size. On the basis of segmenting the image for data reading, small targets in the image block to be tested are enlarged through image enhancement, so as to facilitate more accurate target detection in the subsequent process.

[0087] For example, firstly, the image to be tested I is divided into l=M*N overlapping local image blocks to be tested P1 I 、P2 I 、P3 I ,…P l I Then, each local image block to be tested is enlarged to a size consistent with the sample image block at a certain ratio, for example, to 500-1000 pixels, to obtain the image block to be tested.

[0088] Specifically, in step S170, each image block to be tested is individually input into the target detection fine-tuning model, and the target detection fine-tuning model performs target detection on each image block to be tested in turn to obtain a target detection frame for each image block to be tested. Furthermore, the detected target detection frame can be processed using NMS (non-maximum suppression algorithm) to obtain the final target detection frame. Step S190 performs semantic segmentation on the final target detection frame to improve the accuracy of detection and recognition. In the NMS processing stage, detection frames with an IoU value (Intersection over Union) higher than the matching threshold Tm will be selected, otherwise they will be excluded.

[0089] In one embodiment, step S190 includes steps (c1) to (c4).

[0090] Step (c1): extracting the images corresponding to the target detection frames of the image blocks to be tested respectively and enlarging them by a preset multiple to obtain the target images of the image blocks to be tested.

[0091] The preset multiple can be set according to actual needs. Figure 4 As shown, the image corresponding to the target detection frame is extracted and enlarged by 100 pixels around it to obtain the target image.

[0092] Step (c2): Select a preset number of points in the target image and input them into the semantic segmentation model to obtain a segmentation mask.

[0093] The preset number can be set according to the requirements. Taking the preset number 3 as an example, 3 points are randomly selected around the target in the enlarged target image. The 3 points can be better distributed in the target image and are input into the semantic segmentation model as prompts to obtain the segmentation mask of the component.

[0094] Step (c3): Based on the segmentation mask, a semantic segmentation model is used to perform semantic segmentation on the target image to obtain a segmentation result of the target image.

[0095] Specifically, the segmentation mask and the enlarged target image are input into the semantic segmentation model for semantic recognition, and the segmentation result of the image block to be tested corresponding to the target image is obtained, thereby completing the recognition of the small target.

[0096] Step (c4): integrating the segmentation results of each image block to be tested corresponding to the target image into the image to be tested of the target object, to obtain the recognition result of the image to be tested of the target object.

[0097] Specifically, refer to Figure 4, the position of the image block to be tested in the coordinate system of the image to be tested can be determined, and the segmentation result corresponding to each image block to be tested can be restored to the position of the corresponding image block to be tested in the image to be tested, thereby restoring the recognition result of the small target completed based on the image block to be tested to the original image to be tested.

[0098] When performing semantic segmentation on tiny parts, if the entire large original image is input for automatic segmentation, the accuracy and efficiency are relatively low, and it is difficult to send the prompt. Sending the prompt on the entire image will not get an accurate segmentation result. After completing the target detection of the small image on the segmented image block to be tested, the semantic segmentation of the image block to be tested in the small image is performed, and the small target detection method and the semantic segmentation algorithm are connected in series, which solves the problem of semantic segmentation of small targets on large images and improves the accuracy of semantic recognition. Moreover, by enlarging the target detection frame and extracting a preset number of points to obtain the segmentation mask, semantic recognition can be performed accurately due to the existence of the prompt project. As long as the target detection frame obtained is relatively accurate, this target detection frame is provided to the semantic segmentation model, and objects in various fields can be obtained, such as semiconductor devices, people, animals, etc.

[0099] In one embodiment, the semantic segmentation model adopts the SAM (Segment Anything Model) model. The SAM model has three major features:

[0100] 1. A segmentation task model that can provide prompts;

[0101] 2. Zero-shot migration to new image distributions and tasks;

[0102] 3. Accumulated 11 million images and over 1 billion mask annotations.

[0103] Specifically, the SAM model is inspired by the prompt in NLP (Natural Language Processing). The prompt in the SAM model is a series of points in the foreground or background, a prompt box or unrestricted text; the promptable segmentation model is converted into a legal segmentation mask output according to the prompt.

[0104] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0105] like Figure 5 As shown, the present application provides an image recognition device, including an image acquisition module 510, a model fine-tuning module 530, an image segmentation module 550, a target detection module 570 and a semantic segmentation module 590.

[0106] Among them, the image acquisition module 510 is used to acquire the image to be tested of the target object and multiple sample images of the object; the target object is of the same category as the object. The model fine-tuning module 530 is used to fine-tune the target detection model using multiple sample images to obtain a target detection fine-tuning model. The image segmentation module 550 is used to perform image segmentation on the image to be tested to obtain multiple image blocks to be tested. The target detection module 570 is used to perform target detection on each image block to be tested respectively using the target detection fine-tuning model to obtain the target detection frame of the image block to be tested. The semantic segmentation module 590 is used to perform semantic segmentation on the target detection frame of each image block to be tested, and generate the recognition result of the image to be tested of the target object according to the segmentation result.

[0107] The above-mentioned image recognition device divides the image to be tested into multiple small image blocks to be tested through image segmentation processing, which is convenient for the detection input of large high-resolution images and can effectively identify high-resolution images; moreover, a two-stage method of target detection and semantic segmentation in series is used to first perform target detection on the small image blocks to be tested after the image is segmented to complete the target detection of the small image, that is, to complete the small target detection, and then perform semantic segmentation based on the small target detection. By detecting and identifying the targets in the small image blocks to be tested, the detection and identification of small targets in the overall image to be tested can be achieved. Since the proportion of small targets in the image to be tested in the image blocks to be tested becomes larger after the image is segmented, the identification is more accurate, and therefore the recognition accuracy of small targets can be improved.

[0108] In one embodiment, the model fine-tuning module 530 is further used to segment the sample image to obtain a plurality of sample image blocks; and fine-tune the target detection model based on the sample image blocks and the sample image to obtain a target detection fine-tuning model.

[0109] In one embodiment, the model fine-tuning module 530 performs image segmentation on the sample image to obtain multiple sample image blocks, including: performing overlapping segmentation on the sample image to obtain multiple local sample image blocks; and adjusting the size of each local sample image block to a preset size to obtain an image-enhanced sample image block.

[0110] In one embodiment, the image segmentation module 550 is further used to: segment the image to be tested with overlap to obtain multiple local image blocks to be tested; and adjust the size of each local image block to be tested to a preset size to obtain the image block to be tested after image enhancement.

[0111] In one embodiment, the semantic segmentation module 590 is further used to: extract the images corresponding to the target detection frames of each image block to be tested and enlarge them by a preset multiple to obtain the target images of each image block to be tested; select a preset number of points in the target image and input them into the semantic segmentation model to obtain a segmentation mask; perform semantic segmentation on the target image based on the segmentation mask and using the semantic segmentation model to obtain the segmentation result of the target image; integrate the segmentation results of the target images corresponding to each image block to be tested into the image to be tested of the target object to obtain the recognition result of the image to be tested of the target object.

[0112] For the specific limitations of the image recognition device, please refer to the limitations of the image recognition method above, which will not be repeated here. The various modules in the above-mentioned image recognition device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules. It should be noted that the division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0113] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0114] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that the computer program performs the steps in the above-mentioned method embodiments when executed by a processor.

[0115] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0116] In the description of this specification, the description with reference to the terms "some embodiments", "other embodiments", "ideal embodiments", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic descriptions of the above terms do not necessarily refer to the same embodiment or example.

[0117] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0118] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. An image recognition method, characterized in that: include: Acquire an image of a target object to be tested and multiple sample images of the object; The target object is of the same category as the object; Fine-tune the target detection model using a plurality of the sample images to obtain a target detection fine-tuning model; Performing image segmentation on the image to be tested to obtain a plurality of image blocks to be tested; Using the target detection fine-tuning model to perform target detection on each image block to be tested, and obtain a target detection frame of the image block to be tested; The target detection frame of each image block to be tested is semantically segmented, and a recognition result of the image to be tested of the target object is generated according to the segmentation result.

2. The method according to claim 1, characterized in that: The fine-tuning of the target detection model by using a plurality of sample images to obtain the target detection fine-tuning model includes: Performing image segmentation on the sample image to obtain a plurality of sample image blocks; The object detection model is fine-tuned based on the sample image block and the sample image to obtain the object detection fine-tuning model.

3. The method according to claim 2, characterized in that The step of segmenting the sample image to obtain a plurality of sample image blocks includes: Segmenting the sample image with overlap to obtain a plurality of local sample image blocks; The size of each local sample image block is adjusted to a preset size to obtain a sample image block after image enhancement.

4. The method according to claim 1, characterized in that: The target detection model is the YOLOX model.

5. The method according to claim 1, characterized in that The step of segmenting the image to be tested to obtain a plurality of image blocks to be tested includes: Segmenting the image to be tested with overlap to obtain a plurality of local image blocks to be tested; The size of each local image block to be tested is adjusted to a preset size to obtain the image block to be tested after image enhancement.

6. The method according to any one of claims 1 to 5, characterized in that: The semantic segmentation of the target detection frame of each image block to be tested and generating the recognition result of the image to be tested of the target object according to the segmentation result include: The images corresponding to the target detection frames of the image blocks to be tested are respectively extracted and enlarged by a preset multiple to obtain the target images of the image blocks to be tested; Selecting a preset number of points in the target image and inputting them into a semantic segmentation model to obtain a segmentation mask; Based on the segmentation mask, the semantic segmentation model is used to perform semantic segmentation on the target image to obtain a segmentation result of the target image; The segmentation results of each image block to be tested corresponding to the target image are integrated into the image to be tested of the target object to obtain the recognition result of the image to be tested of the target object.

7. An image recognition device, characterized in that: include: An image acquisition module is used to acquire an image of a target object to be tested and multiple sample images of the object; The target object is of the same category as the object; A model fine-tuning module, used to fine-tune the target detection model using a plurality of the sample images to obtain a target detection fine-tuning model; An image segmentation module is used to segment the image to be tested to obtain a plurality of image blocks to be tested; A target detection module, used to perform target detection on each image block to be tested by using the target detection fine-tuning model to obtain a target detection frame of the image block to be tested; The semantic segmentation module is used to perform semantic segmentation on the target detection frame of each image block to be tested, and generate a recognition result of the image to be tested of the target object according to the segmentation result.

8. The device according to claim 7, characterized in that The image segmentation module is further used for: Segmenting the image to be tested with overlap to obtain multiple local image blocks to be tested; The size of each local image block to be tested is adjusted to a preset size to obtain the image block to be tested after image enhancement.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images